Examples with Fields org.apache.heron.api.tuple.Fields used on opensource projects

Example 21 with Fields

use of org.apache.heron.api.tuple.Fields in project heron by twitter.

the class WordCountTopology method main.

/**
 * Main method
 */
public static void main(String[] args) throws AlreadyAliveException, InvalidTopologyException {
    if (args.length < 1) {
        throw new RuntimeException("Specify topology name");
    }
    int parallelism = 1;
    if (args.length > 1) {
        parallelism = Integer.parseInt(args[1]);
    }
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("word", new WordSpout(), parallelism);
    builder.setBolt("consumer", new ConsumerBolt(), parallelism).fieldsGrouping("word", new Fields("word"));
    Config conf = new Config();
    conf.setNumStmgrs(parallelism);
    // configure component resources
    conf.setComponentRam("word", ByteAmount.fromMegabytes(ExampleResources.COMPONENT_RAM_MB));
    conf.setComponentRam("consumer", ByteAmount.fromMegabytes(ExampleResources.COMPONENT_RAM_MB));
    conf.setComponentCpu("word", 1.0);
    conf.setComponentCpu("consumer", 1.0);
    // configure container resources
    conf.setContainerDiskRequested(ExampleResources.getContainerDisk(2 * parallelism, parallelism));
    conf.setContainerRamRequested(ExampleResources.getContainerRam(2 * parallelism, parallelism));
    conf.setContainerCpuRequested(ExampleResources.getContainerCpu(2 * parallelism, parallelism));
    HeronSubmitter.submitTopology(args[0], conf, builder.createTopology());
}

Example 22 with Fields

use of org.apache.heron.api.tuple.Fields in project heron by twitter.

the class SentenceWordCountTopology method main.

public static void main(String[] args) throws Exception {
    String name = "fast-word-count-topology";
    if (args != null && args.length > 0) {
        name = args[0];
    }
    TopologyBuilder builder = new TopologyBuilder();
    builder.setSpout("spout", new FastRandomSentenceSpout(), 1);
    builder.setBolt("split", new SplitSentence(), 2).shuffleGrouping("spout");
    builder.setBolt("count", new WordCount(), 2).fieldsGrouping("split", new Fields("word"));
    Config conf = new Config();
    // component resource configuration
    conf.setComponentRam("spout", ByteAmount.fromMegabytes(512));
    conf.setComponentRam("split", ByteAmount.fromMegabytes(512));
    conf.setComponentRam("count", ByteAmount.fromMegabytes(512));
    // container resource configuration
    conf.setContainerDiskRequested(ByteAmount.fromGigabytes(3));
    conf.setContainerRamRequested(ByteAmount.fromGigabytes(4));
    conf.setContainerCpuRequested(4);
    conf.setNumStmgrs(2);
    HeronSubmitter.submitTopology(name, conf, builder.createTopology());
}

Example 23 with Fields

use of org.apache.heron.api.tuple.Fields in project heron by twitter.

the class BasicTopologyOneTask method buildTopology.

@Override
protected TopologyBuilder buildTopology(TopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(true), 2);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 3).shuffleGrouping("ab-spout");
    return builder;
}

Example 24 with Fields

use of org.apache.heron.api.tuple.Fields in project heron by twitter.

the class BasicTopologyOneTask method buildTopology.

@Override
protected TopologyBuilder buildTopology(TopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(true), 1);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 3).shuffleGrouping("ab-spout");
    return builder;
}

Example 25 with Fields

use of org.apache.heron.api.tuple.Fields in project heron by twitter.

the class StatefulBasicTopologyOneTask method buildStatefulTopology.

@Override
protected TopologyTestTopologyBuilder buildStatefulTopology(TopologyTestTopologyBuilder builder) {
    builder.setSpout("stateful-ab-spout", new StatefulABSpout(true), 1);
    builder.setBolt("stateful-identity-bolt", new StatefulIdentityBolt(new Fields("word")), 3).shuffleGrouping("stateful-ab-spout");
    return builder;
}