Search in sources :

Example 1 with FiniteInMemorySpout

use of org.apache.flink.storm.util.FiniteInMemorySpout in project flink by apache.

the class ExclamationTopology method buildTopology.

public static TopologyBuilder buildTopology() {
    final TopologyBuilder builder = new TopologyBuilder();
    // get input data
    if (fileInputOutput) {
        // read the text file from given input path
        final String[] tokens = textPath.split(":");
        final String inputFile = tokens[tokens.length - 1];
        builder.setSpout(spoutId, new FiniteFileSpout(inputFile));
    } else {
        builder.setSpout(spoutId, new FiniteInMemorySpout(WordCountData.WORDS));
    }
    builder.setBolt(firstBoltId, new ExclamationBolt(), 3).shuffleGrouping(spoutId);
    builder.setBolt(secondBoltId, new ExclamationBolt(), 2).shuffleGrouping(firstBoltId);
    // emit result
    if (fileInputOutput) {
        // read the text file from given input path
        final String[] tokens = outputPath.split(":");
        final String outputFile = tokens[tokens.length - 1];
        builder.setBolt(sinkId, new BoltFileSink(outputFile, formatter)).shuffleGrouping(secondBoltId);
    } else {
        builder.setBolt(sinkId, new BoltPrintSink(formatter), 4).shuffleGrouping(secondBoltId);
    }
    return builder;
}
Also used : FiniteInMemorySpout(org.apache.flink.storm.util.FiniteInMemorySpout) ExclamationBolt(org.apache.flink.storm.exclamation.operators.ExclamationBolt) BoltFileSink(org.apache.flink.storm.util.BoltFileSink) TopologyBuilder(org.apache.storm.topology.TopologyBuilder) FiniteFileSpout(org.apache.flink.storm.util.FiniteFileSpout) BoltPrintSink(org.apache.flink.storm.util.BoltPrintSink)

Example 2 with FiniteInMemorySpout

use of org.apache.flink.storm.util.FiniteInMemorySpout in project flink by apache.

the class ExclamationWithSpout method getTextDataStream.

private static DataStream<String> getTextDataStream(final StreamExecutionEnvironment env) {
    if (fileOutput) {
        final String[] tokens = textPath.split(":");
        final String inputFile = tokens[tokens.length - 1];
        // set Storm configuration
        StormConfig config = new StormConfig();
        config.put(FiniteFileSpout.INPUT_FILE_PATH, inputFile);
        env.getConfig().setGlobalJobParameters(config);
        return env.addSource(new SpoutWrapper<String>(new FiniteFileSpout(), new String[] { Utils.DEFAULT_STREAM_ID }), TypeExtractor.getForClass(String.class)).setParallelism(1);
    }
    return env.addSource(new SpoutWrapper<String>(new FiniteInMemorySpout(WordCountData.WORDS), new String[] { Utils.DEFAULT_STREAM_ID }), TypeExtractor.getForClass(String.class)).setParallelism(1);
}
Also used : FiniteInMemorySpout(org.apache.flink.storm.util.FiniteInMemorySpout) StormConfig(org.apache.flink.storm.util.StormConfig) SpoutWrapper(org.apache.flink.storm.wrappers.SpoutWrapper) FiniteFileSpout(org.apache.flink.storm.util.FiniteFileSpout)

Aggregations

FiniteFileSpout (org.apache.flink.storm.util.FiniteFileSpout)2 FiniteInMemorySpout (org.apache.flink.storm.util.FiniteInMemorySpout)2 ExclamationBolt (org.apache.flink.storm.exclamation.operators.ExclamationBolt)1 BoltFileSink (org.apache.flink.storm.util.BoltFileSink)1 BoltPrintSink (org.apache.flink.storm.util.BoltPrintSink)1 StormConfig (org.apache.flink.storm.util.StormConfig)1 SpoutWrapper (org.apache.flink.storm.wrappers.SpoutWrapper)1 TopologyBuilder (org.apache.storm.topology.TopologyBuilder)1