Search in sources :

Example 1 with OutputBolt

use of ypf412.storm.bolt.OutputBolt in project storm-hbase by ypf412.

the class OutputTopology method main.

/**
 * HBase Data Output Topology
 * @param args
 * @throws Exception
 */
public static void main(String[] args) throws Exception {
    TopologyBuilder builder = new TopologyBuilder();
    PropConfig pc = new PropConfig("storm.properties");
    int topoWorkers = Integer.valueOf(pc.getProperty("storm.topolopy.workers"));
    int spoutTasks = Integer.valueOf(pc.getProperty("storm.spout.tasks"));
    builder.setSpout("hbaseSpout", new HBaseSpout(), spoutTasks);
    int boltTasks = spoutTasks;
    builder.setBolt("outputBolt", new OutputBolt(), boltTasks).fieldsGrouping("hbaseSpout", new Fields("sharding"));
    Config conf = new Config();
    conf.put(Constants.STORM_PROP_CONF_FILE, "storm.properties");
    conf.put(Constants.HBASE_PROP_CONF_FILE, "hbase.properties");
    if (args != null && args.length > 0) {
        // run on storm cluster
        conf.setNumAckers(1);
        conf.setNumWorkers(topoWorkers);
        StormSubmitter.submitTopology(args[0], conf, builder.createTopology());
    } else {
        // run on local cluster
        conf.setMaxTaskParallelism(3);
        LocalCluster cluster = new LocalCluster();
        cluster.submitTopology("test", conf, builder.createTopology());
        Utils.sleep(10000);
        cluster.killTopology("test");
        cluster.shutdown();
    }
}
Also used : LocalCluster(backtype.storm.LocalCluster) HBaseSpout(ypf412.storm.spout.HBaseSpout) Fields(backtype.storm.tuple.Fields) TopologyBuilder(backtype.storm.topology.TopologyBuilder) Config(backtype.storm.Config) PropConfig(ypf412.storm.util.PropConfig) PropConfig(ypf412.storm.util.PropConfig) OutputBolt(ypf412.storm.bolt.OutputBolt)

Aggregations

Config (backtype.storm.Config)1 LocalCluster (backtype.storm.LocalCluster)1 TopologyBuilder (backtype.storm.topology.TopologyBuilder)1 Fields (backtype.storm.tuple.Fields)1 OutputBolt (ypf412.storm.bolt.OutputBolt)1 HBaseSpout (ypf412.storm.spout.HBaseSpout)1 PropConfig (ypf412.storm.util.PropConfig)1