Search in sources :

Example 1 with PairStream

use of org.apache.storm.streams.PairStream in project storm by apache.

the class JoinExample method main.

public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    // a stream of (number, square) pairs
    PairStream<Integer, Integer> squares = builder.newStream(new NumberSpout(x -> x * x), new PairValueMapper<>(0, 1));
    // a stream of (number, cube) pairs
    PairStream<Integer, Integer> cubes = builder.newStream(new NumberSpout(x -> x * x * x), new PairValueMapper<>(0, 1));
    // create a windowed stream of five seconds duration
    squares.window(TumblingWindows.of(Duration.seconds(5))).join(cubes).print();
    Config config = new Config();
    String topoName = JoinExample.class.getName();
    if (args.length > 0) {
        topoName = args[0];
    }
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}
Also used : StormSubmitter(org.apache.storm.StormSubmitter) OutputFieldsDeclarer(org.apache.storm.topology.OutputFieldsDeclarer) Duration(org.apache.storm.topology.base.BaseWindowedBolt.Duration) BaseRichSpout(org.apache.storm.topology.base.BaseRichSpout) PairValueMapper(org.apache.storm.streams.operations.mappers.PairValueMapper) PairStream(org.apache.storm.streams.PairStream) StreamBuilder(org.apache.storm.streams.StreamBuilder) TopologyContext(org.apache.storm.task.TopologyContext) Fields(org.apache.storm.tuple.Fields) Utils(org.apache.storm.utils.Utils) Function(org.apache.storm.streams.operations.Function) Values(org.apache.storm.tuple.Values) TumblingWindows(org.apache.storm.streams.windowing.TumblingWindows) Map(java.util.Map) Config(org.apache.storm.Config) SpoutOutputCollector(org.apache.storm.spout.SpoutOutputCollector) Config(org.apache.storm.Config) StreamBuilder(org.apache.storm.streams.StreamBuilder)

Aggregations

Map (java.util.Map)1 Config (org.apache.storm.Config)1 StormSubmitter (org.apache.storm.StormSubmitter)1 SpoutOutputCollector (org.apache.storm.spout.SpoutOutputCollector)1 PairStream (org.apache.storm.streams.PairStream)1 StreamBuilder (org.apache.storm.streams.StreamBuilder)1 Function (org.apache.storm.streams.operations.Function)1 PairValueMapper (org.apache.storm.streams.operations.mappers.PairValueMapper)1 TumblingWindows (org.apache.storm.streams.windowing.TumblingWindows)1 TopologyContext (org.apache.storm.task.TopologyContext)1 OutputFieldsDeclarer (org.apache.storm.topology.OutputFieldsDeclarer)1 BaseRichSpout (org.apache.storm.topology.base.BaseRichSpout)1 Duration (org.apache.storm.topology.base.BaseWindowedBolt.Duration)1 Fields (org.apache.storm.tuple.Fields)1 Values (org.apache.storm.tuple.Values)1 Utils (org.apache.storm.utils.Utils)1