Search in sources :

Example 1 with MergerBolt

use of org.apache.flink.storm.tests.operators.MergerBolt in project flink by apache.

the class StormUnionITCase method testProgram.

@Override
protected void testProgram() throws Exception {
    final TopologyBuilder builder = new TopologyBuilder();
    // get input data
    builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
    builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
    builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));
    builder.setBolt(boltId, new MergerBolt()).shuffleGrouping(spoutId1, FiniteRandomSpout.STREAM_PREFIX + 0).shuffleGrouping(spoutId2, FiniteRandomSpout.STREAM_PREFIX + 0).shuffleGrouping(spoutId3, FiniteRandomSpout.STREAM_PREFIX + 0);
    final String[] tokens = this.resultPath.split(":");
    final String outputFile = tokens[tokens.length - 1];
    builder.setBolt(sinkId, new BoltFileSink(outputFile)).shuffleGrouping(boltId);
    // execute program locally
    final FlinkLocalCluster cluster = FlinkLocalCluster.getLocalCluster();
    Config conf = new Config();
    // only required to stabilize integration test
    conf.put(FlinkLocalCluster.SUBMIT_BLOCKING, true);
    cluster.submitTopology(topologyId, conf, FlinkTopology.createTopology(builder));
    cluster.shutdown();
}
Also used : MergerBolt(org.apache.flink.storm.tests.operators.MergerBolt) BoltFileSink(org.apache.flink.storm.util.BoltFileSink) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) FlinkLocalCluster(org.apache.flink.storm.api.FlinkLocalCluster) Config(org.apache.storm.Config) FiniteRandomSpout(org.apache.flink.storm.tests.operators.FiniteRandomSpout)

Aggregations

FlinkLocalCluster (org.apache.flink.storm.api.FlinkLocalCluster)1 FiniteRandomSpout (org.apache.flink.storm.tests.operators.FiniteRandomSpout)1 MergerBolt (org.apache.flink.storm.tests.operators.MergerBolt)1 BoltFileSink (org.apache.flink.storm.util.BoltFileSink)1 Config (org.apache.storm.Config)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1