Search in sources :

Example 6 with RandomIntegerSpout

use of org.apache.storm.starter.spout.RandomIntegerSpout in project storm by apache.

the class StatefulTopology method main.

public static void main(String[] args) throws Exception {
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout", new RandomIntegerSpout());
    builder.setBolt("partialsum", new StatefulSumBolt("partial"), 1).shuffleGrouping("spout");
    builder.setBolt("printer", new PrinterBolt(), 2).shuffleGrouping("partialsum");
    builder.setBolt("total", new StatefulSumBolt("total"), 1).shuffleGrouping("printer");
    Config conf = new Config();
    conf.setDebug(false);
    if (args != null && args.length > 0) {
        conf.setNumWorkers(1);
        StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
    } else {
        try (LocalCluster cluster = new LocalCluster();
            LocalTopology topology = cluster.submitTopology("test", conf, builder.createTopology())) {
            Utils.sleep(40000);
        }
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) Config(org.apache.storm.Config) RandomIntegerSpout(org.apache.storm.starter.spout.RandomIntegerSpout) LocalTopology(org.apache.storm.LocalCluster.LocalTopology)

Example 7 with RandomIntegerSpout

use of org.apache.storm.starter.spout.RandomIntegerSpout in project storm by apache.

the class StatefulWindowingTopology method main.

public static void main(String[] args) throws Exception {
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout", new RandomIntegerSpout());
    builder.setBolt("sumbolt", new WindowSumBolt().withWindow(new Count(5), new Count(3)).withMessageIdField("msgid"), 1).shuffleGrouping("spout");
    builder.setBolt("printer", new PrinterBolt(), 1).shuffleGrouping("sumbolt");
    Config conf = new Config();
    conf.setDebug(false);
    //conf.put(Config.TOPOLOGY_STATE_PROVIDER, "org.apache.storm.redis.state.RedisKeyValueStateProvider");
    if (args != null && args.length > 0) {
        conf.setNumWorkers(1);
        StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
    } else {
        try (LocalCluster cluster = new LocalCluster();
            LocalTopology topo = cluster.submitTopology("test", conf, builder.createTopology())) {
            Utils.sleep(40000);
        }
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) Config(org.apache.storm.Config) RandomIntegerSpout(org.apache.storm.starter.spout.RandomIntegerSpout) Count(org.apache.storm.topology.base.BaseWindowedBolt.Count) PrinterBolt(org.apache.storm.starter.bolt.PrinterBolt) LocalTopology(org.apache.storm.LocalCluster.LocalTopology)

Example 8 with RandomIntegerSpout

use of org.apache.storm.starter.spout.RandomIntegerSpout in project storm by apache.

the class AggregateExample method main.

@SuppressWarnings("unchecked")
public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    /**
         * Computes average of the stream of numbers emitted by the spout. Internally the per-partition
         * sum and counts are accumulated and emitted to a downstream task where the partially accumulated
         * results are merged and the final result is emitted.
         */
    builder.newStream(new RandomIntegerSpout(), new ValueMapper<Integer>(0), 2).window(TumblingWindows.of(BaseWindowedBolt.Duration.seconds(5))).filter(x -> x > 0 && x < 500).aggregate(new Avg()).print();
    Config config = new Config();
    if (args.length > 0) {
        config.setNumWorkers(1);
        StormSubmitter.submitTopologyWithProgressBar(args[0], config, builder.build());
    } else {
        try (LocalCluster cluster = new LocalCluster();
            LocalCluster.LocalTopology topo = cluster.submitTopology("test", config, builder.build())) {
            Utils.sleep(60_000);
        }
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) Config(org.apache.storm.Config) RandomIntegerSpout(org.apache.storm.starter.spout.RandomIntegerSpout) StreamBuilder(org.apache.storm.streams.StreamBuilder)

Example 9 with RandomIntegerSpout

use of org.apache.storm.starter.spout.RandomIntegerSpout in project storm by apache.

the class BranchExample method main.

@SuppressWarnings("unchecked")
public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    Stream<Integer>[] evenAndOdd = builder.newStream(new RandomIntegerSpout(), new ValueMapper<Integer>(0)).branch(x -> (x % 2) == 0, x -> (x % 2) == 1);
    evenAndOdd[0].forEach(x -> LOG.info("EVEN> " + x));
    evenAndOdd[1].forEach(x -> LOG.info("ODD > " + x));
    Config config = new Config();
    if (args.length > 0) {
        config.setNumWorkers(1);
        StormSubmitter.submitTopologyWithProgressBar(args[0], config, builder.build());
    } else {
        try (LocalCluster cluster = new LocalCluster();
            LocalCluster.LocalTopology topo = cluster.submitTopology("test", config, builder.build())) {
            Utils.sleep(60_000);
        }
    }
}
Also used : LocalCluster(org.apache.storm.LocalCluster) ValueMapper(org.apache.storm.streams.operations.mappers.ValueMapper) Config(org.apache.storm.Config) RandomIntegerSpout(org.apache.storm.starter.spout.RandomIntegerSpout) Stream(org.apache.storm.streams.Stream) StreamBuilder(org.apache.storm.streams.StreamBuilder)

Aggregations

RandomIntegerSpout (org.apache.storm.starter.spout.RandomIntegerSpout)9 Config (org.apache.storm.Config)7 LocalCluster (org.apache.storm.LocalCluster)7 PrinterBolt (org.apache.storm.starter.bolt.PrinterBolt)5 LocalTopology (org.apache.storm.LocalCluster.LocalTopology)4 SlidingWindowSumBolt (org.apache.storm.starter.bolt.SlidingWindowSumBolt)4 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)4 StreamBuilder (org.apache.storm.streams.StreamBuilder)3 TopologyBuilder (backtype.storm.topology.TopologyBuilder)2 JStormHelper (com.alibaba.starter.utils.JStormHelper)2 BaseWindowedBolt (backtype.storm.topology.base.BaseWindowedBolt)1 Count (backtype.storm.topology.base.BaseWindowedBolt.Count)1 Duration (backtype.storm.topology.base.BaseWindowedBolt.Duration)1 Stream (org.apache.storm.streams.Stream)1 ValueMapper (org.apache.storm.streams.operations.mappers.ValueMapper)1 Tuple3 (org.apache.storm.streams.tuple.Tuple3)1 BaseWindowedBolt (org.apache.storm.topology.base.BaseWindowedBolt)1 Count (org.apache.storm.topology.base.BaseWindowedBolt.Count)1 Duration (org.apache.storm.topology.base.BaseWindowedBolt.Duration)1