Search in sources :

Example 41 with Fields

use of com.twitter.heron.api.tuple.Fields in project incubator-heron by apache.

the class BasicTopologyOneTask method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(true), 1);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 2).shuffleGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 42 with Fields

use of com.twitter.heron.api.tuple.Fields in project incubator-heron by apache.

the class FieldsGrouping method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 1, 400);
    builder.setBolt("count-bolt", new WordCountBolt(), 2).fieldsGrouping("ab-spout", new Fields("word"));
    builder.setBolt("sum-bolt", new CountAggregatorBolt(), 1).noneGrouping("count-bolt");
    return builder;
}
Also used : Fields(com.twitter.heron.api.tuple.Fields) CountAggregatorBolt(com.twitter.heron.integration_test.common.bolt.CountAggregatorBolt) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout) WordCountBolt(com.twitter.heron.integration_test.common.bolt.WordCountBolt)

Example 43 with Fields

use of com.twitter.heron.api.tuple.Fields in project incubator-heron by apache.

the class OneBoltMultiTasks method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 1);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 3).shuffleGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 44 with Fields

use of com.twitter.heron.api.tuple.Fields in project incubator-heron by apache.

the class OneSpoutMultiTasks method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 3);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 1).shuffleGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 45 with Fields

use of com.twitter.heron.api.tuple.Fields in project incubator-heron by apache.

the class TopologyManagerTest method getTestTopology.

/**
 * Construct the test topology
 */
public static TopologyAPI.Topology getTestTopology() {
    TopologyBuilder topologyBuilder = new TopologyBuilder();
    topologyBuilder.setSpout(STREAM_ID, new BaseRichSpout() {

        private static final long serialVersionUID = 5406114907377311020L;

        @Override
        public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
            outputFieldsDeclarer.declare(new Fields(STREAM_ID));
        }

        @Override
        public void open(Map<String, Object> map, TopologyContext topologyContext, SpoutOutputCollector spoutOutputCollector) {
        }

        @Override
        public void nextTuple() {
        }
    }, 2);
    topologyBuilder.setBolt(BOLT_ID, new BaseBasicBolt() {

        private static final long serialVersionUID = 4398578755681473899L;

        @Override
        public void execute(Tuple tuple, BasicOutputCollector basicOutputCollector) {
        }

        @Override
        public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
        }
    }, 2).shuffleGrouping(STREAM_ID);
    Config conf = new Config();
    conf.setDebug(true);
    conf.setMaxSpoutPending(10);
    conf.put(Config.TOPOLOGY_WORKER_CHILDOPTS, "-XX:+HeapDumpOnOutOfMemoryError");
    conf.setComponentRam(STREAM_ID, ByteAmount.fromMegabytes(500));
    conf.setComponentRam(BOLT_ID, ByteAmount.fromGigabytes(1));
    conf.setMessageTimeoutSecs(1);
    return topologyBuilder.createTopology().setName("topology-name").setConfig(conf).setState(TopologyAPI.TopologyState.RUNNING).getTopology();
}
Also used : BaseBasicBolt(com.twitter.heron.api.bolt.BaseBasicBolt) TopologyBuilder(com.twitter.heron.api.topology.TopologyBuilder) Config(com.twitter.heron.api.Config) OutputFieldsDeclarer(com.twitter.heron.api.topology.OutputFieldsDeclarer) BasicOutputCollector(com.twitter.heron.api.bolt.BasicOutputCollector) Fields(com.twitter.heron.api.tuple.Fields) SpoutOutputCollector(com.twitter.heron.api.spout.SpoutOutputCollector) TopologyContext(com.twitter.heron.api.topology.TopologyContext) Tuple(com.twitter.heron.api.tuple.Tuple) BaseRichSpout(com.twitter.heron.api.spout.BaseRichSpout)

Aggregations

Fields (com.twitter.heron.api.tuple.Fields)50 ABSpout (com.twitter.heron.integration_test.common.spout.ABSpout)22 IdentityBolt (com.twitter.heron.integration_test.common.bolt.IdentityBolt)20 Config (com.twitter.heron.api.Config)13 TopologyBuilder (com.twitter.heron.api.topology.TopologyBuilder)13 Tuple (com.twitter.heron.api.tuple.Tuple)8 TopologyArgParser (com.github.ashvina.common.TopologyArgParser)5 TopologyAPI (com.twitter.heron.api.generated.TopologyAPI)5 TopologyContext (com.twitter.heron.api.topology.TopologyContext)5 Values (com.twitter.heron.api.tuple.Values)5 HashMap (java.util.HashMap)5 BaseBasicBolt (com.twitter.heron.api.bolt.BaseBasicBolt)4 BasicOutputCollector (com.twitter.heron.api.bolt.BasicOutputCollector)4 BaseRichSpout (com.twitter.heron.api.spout.BaseRichSpout)4 SpoutOutputCollector (com.twitter.heron.api.spout.SpoutOutputCollector)4 OutputFieldsDeclarer (com.twitter.heron.api.topology.OutputFieldsDeclarer)4 TupleWindow (com.twitter.heron.api.windowing.TupleWindow)4 Map (java.util.Map)4 TupleWindowImpl (com.twitter.heron.api.windowing.TupleWindowImpl)3 ArrayList (java.util.ArrayList)3