Search in sources :

Example 1 with HBaseCountersBatchBolt

use of backtype.storm.contrib.hbase.bolts.HBaseCountersBatchBolt in project storm-hbase by jrkinley.

the class HBaseCountersBatchTopology method main.

/**
   * @param args
   * @throws InterruptedException 
   */
public static void main(String[] args) throws InterruptedException {
    // Add transactional spout
    MemoryTransactionalSpout spout = new MemoryTransactionalSpout(values, new Fields("shortid", "url", "user", "date"), 3);
    TransactionalTopologyBuilder builder = new TransactionalTopologyBuilder("shorturl-count", "spout", spout, 2);
    // Build TupleTableConifg
    TupleTableConfig ttConfig = new TupleTableConfig("shorturl", "shortid");
    ttConfig.setBatch(false);
    ttConfig.addColumn("data", "clicks");
    ttConfig.addColumn("daily", "date");
    builder.setBolt("hbase-counters", new HBaseCountersBatchBolt(ttConfig), 2).fieldsGrouping("spout", new Fields("shortid"));
    LocalCluster cluster = new LocalCluster();
    Config stormConfig = new Config();
    stormConfig.setDebug(true);
    stormConfig.setMaxSpoutPending(3);
    cluster.submitTopology("hbase-example", stormConfig, builder.buildTopology());
    Thread.sleep(10000);
    cluster.shutdown();
}
Also used : LocalCluster(backtype.storm.LocalCluster) MemoryTransactionalSpout(backtype.storm.testing.MemoryTransactionalSpout) Fields(backtype.storm.tuple.Fields) HBaseCountersBatchBolt(backtype.storm.contrib.hbase.bolts.HBaseCountersBatchBolt) Config(backtype.storm.Config) TupleTableConfig(backtype.storm.contrib.hbase.utils.TupleTableConfig) TupleTableConfig(backtype.storm.contrib.hbase.utils.TupleTableConfig) TransactionalTopologyBuilder(backtype.storm.transactional.TransactionalTopologyBuilder)

Aggregations

Config (backtype.storm.Config)1 LocalCluster (backtype.storm.LocalCluster)1 HBaseCountersBatchBolt (backtype.storm.contrib.hbase.bolts.HBaseCountersBatchBolt)1 TupleTableConfig (backtype.storm.contrib.hbase.utils.TupleTableConfig)1 MemoryTransactionalSpout (backtype.storm.testing.MemoryTransactionalSpout)1 TransactionalTopologyBuilder (backtype.storm.transactional.TransactionalTopologyBuilder)1 Fields (backtype.storm.tuple.Fields)1