Search in sources :

Example 11 with Runner

use of com.twitter.heron.streamlet.Runner in project incubator-heron by apache.

the class StreamletCloneTopology method main.

/**
 * All Heron topologies require a main function that defines the topology's behavior
 * at runtime
 */
public static void main(String[] args) throws Exception {
    Builder processingGraphBuilder = Builder.newBuilder();
    /**
     * A supplier streamlet of random GameScore objects is cloned into two
     * separate streamlets.
     */
    List<Streamlet<GameScore>> splitGameScoreStreamlet = processingGraphBuilder.newSource(GameScore::new).clone(2);
    /**
     * Elements in the first cloned streamlet go to the database sink.
     */
    splitGameScoreStreamlet.get(0).toSink(new DatabaseSink());
    /**
     * Elements in the second cloned streamlet go to the logging sink.
     */
    splitGameScoreStreamlet.get(1).toSink(new FormattedLogSink());
    Config config = Config.defaultConfig();
    // Fetches the topology name from the first command-line argument
    String topologyName = StreamletUtils.getTopologyName(args);
    // Finally, the processing graph and configuration are passed to the Runner, which converts
    // the graph into a Heron topology that can be run in a Heron cluster.
    new Runner().run(topologyName, config, processingGraphBuilder);
}
Also used : Runner(com.twitter.heron.streamlet.Runner) Config(com.twitter.heron.streamlet.Config) Builder(com.twitter.heron.streamlet.Builder) Streamlet(com.twitter.heron.streamlet.Streamlet)

Aggregations

Builder (com.twitter.heron.streamlet.Builder)11 Config (com.twitter.heron.streamlet.Config)11 Runner (com.twitter.heron.streamlet.Runner)11 WindowConfig (com.twitter.heron.streamlet.WindowConfig)3 StreamletUtils (com.twitter.heron.examples.streamlet.utils.StreamletUtils)2 Streamlet (com.twitter.heron.streamlet.Streamlet)2 Serializable (java.io.Serializable)2 List (java.util.List)2 Logger (java.util.logging.Logger)2 Collectors (java.util.stream.Collectors)2 IntStream (java.util.stream.IntStream)2 JoinType (com.twitter.heron.streamlet.JoinType)1 KeyValue (com.twitter.heron.streamlet.KeyValue)1 File (java.io.File)1 DecimalFormat (java.text.DecimalFormat)1 Arrays (java.util.Arrays)1 Random (java.util.Random)1 UUID (java.util.UUID)1