Examples with StormConfig org.apache.flink.storm.util.StormConfig used on opensource projects

Example 6 with StormConfig

use of org.apache.flink.storm.util.StormConfig in project flink by apache.

the class ExclamationWithBolt method main.

// *************************************************************************
// PROGRAM
// *************************************************************************
public static void main(final String[] args) throws Exception {
    if (!parseParameters(args)) {
        return;
    }
    // set up the execution environment
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    // set Storm configuration
    StormConfig config = new StormConfig();
    config.put(ExclamationBolt.EXCLAMATION_COUNT, new Integer(exclamationNum));
    env.getConfig().setGlobalJobParameters(config);
    // get input data
    final DataStream<String> text = getTextDataStream(env);
    final DataStream<String> exclaimed = text.transform("StormBoltTokenizer", TypeExtractor.getForObject(""), new BoltWrapper<String, String>(new ExclamationBolt(), new String[] { Utils.DEFAULT_STREAM_ID })).map(new ExclamationMap());
    // emit result
    if (fileOutput) {
        exclaimed.writeAsText(outputPath);
    } else {
        exclaimed.print();
    }
    // execute program
    env.execute("Streaming WordCount with bolt tokenizer");
}

Example 7 with StormConfig

use of org.apache.flink.storm.util.StormConfig in project flink by apache.

the class ExclamationWithSpout method getTextDataStream.

private static DataStream<String> getTextDataStream(final StreamExecutionEnvironment env) {
    if (fileOutput) {
        final String[] tokens = textPath.split(":");
        final String inputFile = tokens[tokens.length - 1];
        // set Storm configuration
        StormConfig config = new StormConfig();
        config.put(FiniteFileSpout.INPUT_FILE_PATH, inputFile);
        env.getConfig().setGlobalJobParameters(config);
        return env.addSource(new SpoutWrapper<String>(new FiniteFileSpout(), new String[] { Utils.DEFAULT_STREAM_ID }), TypeExtractor.getForClass(String.class)).setParallelism(1);
    }
    return env.addSource(new SpoutWrapper<String>(new FiniteInMemorySpout(WordCountData.WORDS), new String[] { Utils.DEFAULT_STREAM_ID }), TypeExtractor.getForClass(String.class)).setParallelism(1);
}

Example 8 with StormConfig

use of org.apache.flink.storm.util.StormConfig in project flink by apache.

the class FlinkClient method addStormConfigToTopology.

@SuppressWarnings({ "unchecked", "rawtypes" })
static void addStormConfigToTopology(FlinkTopology topology, Map conf) throws ClassNotFoundException {
    if (conf != null) {
        ExecutionConfig flinkConfig = topology.getExecutionEnvironment().getConfig();
        flinkConfig.setGlobalJobParameters(new StormConfig(conf));
        // add all registered types to ExecutionConfig
        List<?> registeredClasses = (List<?>) conf.get(Config.TOPOLOGY_KRYO_REGISTER);
        if (registeredClasses != null) {
            for (Object klass : registeredClasses) {
                if (klass instanceof String) {
                    flinkConfig.registerKryoType(Class.forName((String) klass));
                } else {
                    for (Entry<String, String> register : ((Map<String, String>) klass).entrySet()) {
                        flinkConfig.registerTypeWithKryoSerializer(Class.forName(register.getKey()), (Class<? extends Serializer<?>>) Class.forName(register.getValue()));
                    }
                }
            }
        }
    }
}