Search in sources :

Example 1 with WordCountValueMapper

use of org.apache.storm.hbase.topology.WordCountValueMapper in project storm by apache.

the class WordCountTrident method buildTopology.

public static StormTopology buildTopology(String hbaseRoot) {
    Fields fields = new Fields("word", "count");
    FixedBatchSpout spout = new FixedBatchSpout(fields, 4, new Values("storm", 1), new Values("trident", 1), new Values("needs", 1), new Values("javadoc", 1));
    spout.setCycle(true);
    TridentHBaseMapper tridentHBaseMapper = new SimpleTridentHBaseMapper().withColumnFamily("cf").withColumnFields(new Fields("word")).withCounterFields(new Fields("count")).withRowKeyField("word");
    HBaseValueMapper rowToStormValueMapper = new WordCountValueMapper();
    HBaseProjectionCriteria projectionCriteria = new HBaseProjectionCriteria();
    projectionCriteria.addColumn(new HBaseProjectionCriteria.ColumnMetaData("cf", "count"));
    HBaseState.Options options = new HBaseState.Options().withConfigKey(hbaseRoot).withDurability(Durability.SYNC_WAL).withMapper(tridentHBaseMapper).withProjectionCriteria(projectionCriteria).withRowToStormValueMapper(rowToStormValueMapper).withTableName("WordCount");
    StateFactory factory = new HBaseStateFactory(options);
    TridentTopology topology = new TridentTopology();
    Stream stream = topology.newStream("spout1", spout);
    stream.partitionPersist(factory, fields, new HBaseUpdater(), new Fields());
    TridentState state = topology.newStaticState(factory);
    stream = stream.stateQuery(state, new Fields("word"), new HBaseQuery(), new Fields("columnName", "columnValue"));
    stream.each(new Fields("word", "columnValue"), new PrintFunction(), new Fields());
    return topology.build();
}
Also used : HBaseUpdater(org.apache.storm.hbase.trident.state.HBaseUpdater) TridentState(org.apache.storm.trident.TridentState) Values(org.apache.storm.tuple.Values) HBaseProjectionCriteria(org.apache.storm.hbase.bolt.mapper.HBaseProjectionCriteria) WordCountValueMapper(org.apache.storm.hbase.topology.WordCountValueMapper) HBaseValueMapper(org.apache.storm.hbase.bolt.mapper.HBaseValueMapper) HBaseStateFactory(org.apache.storm.hbase.trident.state.HBaseStateFactory) FixedBatchSpout(org.apache.storm.trident.testing.FixedBatchSpout) Fields(org.apache.storm.tuple.Fields) SimpleTridentHBaseMapper(org.apache.storm.hbase.trident.mapper.SimpleTridentHBaseMapper) StateFactory(org.apache.storm.trident.state.StateFactory) HBaseStateFactory(org.apache.storm.hbase.trident.state.HBaseStateFactory) HBaseState(org.apache.storm.hbase.trident.state.HBaseState) TridentTopology(org.apache.storm.trident.TridentTopology) Stream(org.apache.storm.trident.Stream) SimpleTridentHBaseMapper(org.apache.storm.hbase.trident.mapper.SimpleTridentHBaseMapper) TridentHBaseMapper(org.apache.storm.hbase.trident.mapper.TridentHBaseMapper) HBaseQuery(org.apache.storm.hbase.trident.state.HBaseQuery)

Aggregations

HBaseProjectionCriteria (org.apache.storm.hbase.bolt.mapper.HBaseProjectionCriteria)1 HBaseValueMapper (org.apache.storm.hbase.bolt.mapper.HBaseValueMapper)1 WordCountValueMapper (org.apache.storm.hbase.topology.WordCountValueMapper)1 SimpleTridentHBaseMapper (org.apache.storm.hbase.trident.mapper.SimpleTridentHBaseMapper)1 TridentHBaseMapper (org.apache.storm.hbase.trident.mapper.TridentHBaseMapper)1 HBaseQuery (org.apache.storm.hbase.trident.state.HBaseQuery)1 HBaseState (org.apache.storm.hbase.trident.state.HBaseState)1 HBaseStateFactory (org.apache.storm.hbase.trident.state.HBaseStateFactory)1 HBaseUpdater (org.apache.storm.hbase.trident.state.HBaseUpdater)1 Stream (org.apache.storm.trident.Stream)1 TridentState (org.apache.storm.trident.TridentState)1 TridentTopology (org.apache.storm.trident.TridentTopology)1 StateFactory (org.apache.storm.trident.state.StateFactory)1 FixedBatchSpout (org.apache.storm.trident.testing.FixedBatchSpout)1 Fields (org.apache.storm.tuple.Fields)1 Values (org.apache.storm.tuple.Values)1