Examples with LocalCluster backtype.storm.LocalCluster used on opensource projects

Example 31 with LocalCluster

use of backtype.storm.LocalCluster in project jstorm by alibaba.

the class TestTopology method submitTopology.

private static void submitTopology(TopologyBuilder builder) {
    try {
        if (local_mode(conf)) {
            LocalCluster cluster = new LocalCluster();
            cluster.submitTopology(String.valueOf(conf.get("topology.name")), conf, builder.createTopology());
            Thread.sleep(200000);
            cluster.shutdown();
        } else {
            StormSubmitter.submitTopology(String.valueOf(conf.get("topology.name")), conf, builder.createTopology());
        }
    } catch (Exception e) {
        LOG.error(e.getMessage(), e.getCause());
    }
}

Example 32 with LocalCluster

use of backtype.storm.LocalCluster in project jstorm by alibaba.

the class TestTopology method submitTopology.

private static void submitTopology(TopologyBuilder builder) {
    try {
        if (local_mode(conf)) {
            LocalCluster cluster = new LocalCluster();
            cluster.submitTopology(String.valueOf(conf.get("topology.name")), conf, builder.createTopology());
            Thread.sleep(200000);
            cluster.shutdown();
        } else {
            StormSubmitter.submitTopology(String.valueOf(conf.get("topology.name")), conf, builder.createTopology());
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Example 33 with LocalCluster

use of backtype.storm.LocalCluster in project jstorm by alibaba.

the class TridentWindowingInmemoryStoreTopology method test.

public static void test() {
    String[] className = Thread.currentThread().getStackTrace()[1].getClassName().split("\\.");
    String topologyName = className[className.length - 1];
    try {
        WindowsStoreFactory mapState = new InMemoryWindowsStoreFactory();
        List<? extends WindowConfig> list = Arrays.asList(SlidingCountWindow.of(1000, 100), TumblingCountWindow.of(1000), SlidingDurationWindow.of(new BaseWindowedBolt.Duration(6, TimeUnit.SECONDS), new BaseWindowedBolt.Duration(3, TimeUnit.SECONDS)), TumblingDurationWindow.of(new BaseWindowedBolt.Duration(3, TimeUnit.SECONDS)));
        for (WindowConfig windowConfig : list) {
            LocalCluster cluster = new LocalCluster();
            cluster.submitTopology("wordCounter", conf, buildTopology(mapState, windowConfig));
            Utils.sleep(60 * 1000);
            cluster.shutdown();
        }
        for (WindowConfig windowConfig : list) {
            String name = topologyName + "." + windowConfig.getWindowLength() + "." + windowConfig.getSlidingLength();
            JStormHelper.runTopology(buildTopology(mapState, windowConfig), name, conf, 60, new JStormHelper.CheckAckedFail(conf), isLocal);
        }
    } catch (Exception e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        Assert.fail("Failed");
    }
}

Example 34 with LocalCluster

use of backtype.storm.LocalCluster in project jstorm by alibaba.

the class TestSuite method init.

@BeforeClass
public static void init() {
    cluster = new LocalCluster();
    conf = new Config();
    conf.put(Config.TOPOLOGY_MAX_TASK_PARALLELISM, 1);
    esConfig = new EsConfig("test", "127.0.0.1:9300");
}

Example 35 with LocalCluster

use of backtype.storm.LocalCluster in project storm-hbase by ypf412.

the class DumpToHBaseTopology method main.

/**
 * HBase Data Dump to Another HBase Table Topology
 * @param args
 * @throws Exception
 */
public static void main(String[] args) throws Exception {
    TopologyBuilder builder = new TopologyBuilder();
    PropConfig pc = new PropConfig("storm.properties");
    int topoWorkers = Integer.valueOf(pc.getProperty("storm.topolopy.workers"));
    int spoutTasks = Integer.valueOf(pc.getProperty("storm.spout.tasks"));
    builder.setSpout("hbaseSpout", new HBaseSpout(), spoutTasks);
    int boltTasks = spoutTasks;
    builder.setBolt("dumpBolt", new DumpToHBaseBolt(), boltTasks).fieldsGrouping("hbaseSpout", new Fields("sharding"));
    Config conf = new Config();
    conf.put(Constants.STORM_PROP_CONF_FILE, "storm.properties");
    conf.put(Constants.HBASE_PROP_CONF_FILE, "hbase.properties");
    if (args != null && args.length > 0) {
        // run on storm cluster
        conf.setNumAckers(1);
        conf.setNumWorkers(topoWorkers);
        StormSubmitter.submitTopology(args[0], conf, builder.createTopology());
    } else {
        // run on local cluster
        conf.setMaxTaskParallelism(3);
        LocalCluster cluster = new LocalCluster();
        cluster.submitTopology("test", conf, builder.createTopology());
        Utils.sleep(10000);
        cluster.killTopology("test");
        cluster.shutdown();
    }
}