Examples with StreamBuilder org.apache.storm.streams.StreamBuilder used on opensource projects

Example 1 with StreamBuilder

use of org.apache.storm.streams.StreamBuilder in project storm by apache.

the class JoinExample method main.

public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    // a stream of (number, square) pairs
    PairStream<Integer, Integer> squares = builder.newStream(new NumberSpout(x -> x * x), new PairValueMapper<>(0, 1));
    // a stream of (number, cube) pairs
    PairStream<Integer, Integer> cubes = builder.newStream(new NumberSpout(x -> x * x * x), new PairValueMapper<>(0, 1));
    // create a windowed stream of five seconds duration
    squares.window(TumblingWindows.of(Duration.seconds(5))).join(cubes).print();
    Config config = new Config();
    String topoName = JoinExample.class.getName();
    if (args.length > 0) {
        topoName = args[0];
    }
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}

Example 2 with StreamBuilder

use of org.apache.storm.streams.StreamBuilder in project storm by apache.

the class StatefulWordCount method main.

public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    // a stream of words
    builder.newStream(new TestWordSpout(), new ValueMapper<String>(0), 2).window(TumblingWindows.of(BaseWindowedBolt.Duration.seconds(2))).mapToPair(w -> Pair.of(w, 1)).countByKey().updateStateByKey(0L, (state, count) -> state + count).toPairStream().print();
    Config config = new Config();
    // use redis based state store for persistence
    config.put(Config.TOPOLOGY_STATE_PROVIDER, "org.apache.storm.redis.state.RedisKeyValueStateProvider");
    String topoName = "test";
    if (args.length > 0) {
        topoName = args[0];
    }
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}

Example 3 with StreamBuilder

use of org.apache.storm.streams.StreamBuilder in project storm by apache.

the class StateQueryExample method main.

public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    StreamState<String, Long> ss = builder.newStream(new TestWordSpout(), new ValueMapper<String>(0), 2).mapToPair(w -> Pair.of(w, 1)).updateStateByKey(0L, (count, val) -> count + 1);
    /*
         * A stream of words emitted by the QuerySpout is used as
         * the keys to query the state.
         */
    builder.newStream(new QuerySpout(), new ValueMapper<String>(0)).stateQuery(ss).print();
    Config config = new Config();
    // use redis based state store for persistence
    config.put(Config.TOPOLOGY_STATE_PROVIDER, "org.apache.storm.redis.state.RedisKeyValueStateProvider");
    String topoName = "test";
    if (args.length > 0) {
        topoName = args[0];
    }
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}

Example 4 with StreamBuilder

use of org.apache.storm.streams.StreamBuilder in project storm by apache.

the class TypedTupleExample method main.

/**
 * The spout emits sequences of (Integer, Long, Long). TupleValueMapper can be used to extract fields
 * from the values and produce a stream of typed tuple (Tuple3<Integer, Long, Long> in this case.
 */
public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    Stream<Tuple3<Integer, Long, Long>> stream = builder.newStream(new RandomIntegerSpout(), TupleValueMappers.of(0, 1, 2));
    PairStream<Long, Integer> pairs = stream.mapToPair(t -> Pair.of(t.value2 / 10000, t.value1));
    pairs.window(TumblingWindows.of(Count.of(10))).groupByKey().print();
    String topoName = "test";
    if (args.length > 0) {
        topoName = args[0];
    }
    Config config = new Config();
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}

Example 5 with StreamBuilder

use of org.apache.storm.streams.StreamBuilder in project storm by apache.

the class WordCountToBolt method main.

public static void main(String[] args) throws Exception {
    StreamBuilder builder = new StreamBuilder();
    // Redis config parameters for the RedisStoreBolt
    JedisPoolConfig poolConfig = new JedisPoolConfig.Builder().setHost("127.0.0.1").setPort(6379).build();
    // Storm tuple to redis key-value mapper
    RedisStoreMapper storeMapper = new WordCountStoreMapper();
    // The redis bolt (sink)
    IRichBolt redisStoreBolt = new RedisStoreBolt(poolConfig, storeMapper);
    // A stream of words
    builder.newStream(new TestWordSpout(), new ValueMapper<String>(0)).mapToPair(w -> Pair.of(w, 1)).countByKey().to(redisStoreBolt);
    Config config = new Config();
    String topoName = "test";
    if (args.length > 0) {
        topoName = args[0];
    }
    config.setNumWorkers(1);
    StormSubmitter.submitTopologyWithProgressBar(topoName, config, builder.build());
}