Search in sources :

Example 1 with FiniteRandomSpout

use of org.apache.flink.storm.tests.operators.FiniteRandomSpout in project flink by apache.

the class StormFieldsGroupingITCase method testProgram.

@Override
protected void testProgram() throws Exception {
    final String[] tokens = this.resultPath.split(":");
    final String outputFile = tokens[tokens.length - 1];
    final TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout(spoutId, new FiniteRandomSpout(0, 10, 2));
    builder.setBolt(boltId, new TaskIdBolt(), 2).fieldsGrouping(spoutId, FiniteRandomSpout.STREAM_PREFIX + 0, new Fields("number"));
    builder.setBolt(sinkId, new BoltFileSink(outputFile)).shuffleGrouping(boltId);
    final FlinkLocalCluster cluster = FlinkLocalCluster.getLocalCluster();
    Config conf = new Config();
    // only required to stabilize integration test
    conf.put(FlinkLocalCluster.SUBMIT_BLOCKING, true);
    cluster.submitTopology(topologyId, conf, FlinkTopology.createTopology(builder));
    cluster.shutdown();
}
Also used : BoltFileSink(org.apache.flink.storm.util.BoltFileSink) Fields(org.apache.storm.tuple.Fields) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) FlinkLocalCluster(org.apache.flink.storm.api.FlinkLocalCluster) Config(org.apache.storm.Config) FiniteRandomSpout(org.apache.flink.storm.tests.operators.FiniteRandomSpout) TaskIdBolt(org.apache.flink.storm.tests.operators.TaskIdBolt)

Example 2 with FiniteRandomSpout

use of org.apache.flink.storm.tests.operators.FiniteRandomSpout in project flink by apache.

the class StormUnionITCase method testProgram.

@Override
protected void testProgram() throws Exception {
    final TopologyBuilder builder = new TopologyBuilder();
    // get input data
    builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
    builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
    builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));
    builder.setBolt(boltId, new MergerBolt()).shuffleGrouping(spoutId1, FiniteRandomSpout.STREAM_PREFIX + 0).shuffleGrouping(spoutId2, FiniteRandomSpout.STREAM_PREFIX + 0).shuffleGrouping(spoutId3, FiniteRandomSpout.STREAM_PREFIX + 0);
    final String[] tokens = this.resultPath.split(":");
    final String outputFile = tokens[tokens.length - 1];
    builder.setBolt(sinkId, new BoltFileSink(outputFile)).shuffleGrouping(boltId);
    // execute program locally
    final FlinkLocalCluster cluster = FlinkLocalCluster.getLocalCluster();
    Config conf = new Config();
    // only required to stabilize integration test
    conf.put(FlinkLocalCluster.SUBMIT_BLOCKING, true);
    cluster.submitTopology(topologyId, conf, FlinkTopology.createTopology(builder));
    cluster.shutdown();
}
Also used : MergerBolt(org.apache.flink.storm.tests.operators.MergerBolt) BoltFileSink(org.apache.flink.storm.util.BoltFileSink) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) FlinkLocalCluster(org.apache.flink.storm.api.FlinkLocalCluster) Config(org.apache.storm.Config) FiniteRandomSpout(org.apache.flink.storm.tests.operators.FiniteRandomSpout)

Aggregations

FlinkLocalCluster (org.apache.flink.storm.api.FlinkLocalCluster)2 FiniteRandomSpout (org.apache.flink.storm.tests.operators.FiniteRandomSpout)2 BoltFileSink (org.apache.flink.storm.util.BoltFileSink)2 Config (org.apache.storm.Config)2 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)2 MergerBolt (org.apache.flink.storm.tests.operators.MergerBolt)1 TaskIdBolt (org.apache.flink.storm.tests.operators.TaskIdBolt)1 Fields (org.apache.storm.tuple.Fields)1