Search in sources :

Example 1 with BoltDeclarer

use of com.twitter.heron.api.topology.BoltDeclarer in project heron by twitter.

the class TopologyTests method createTopologyWithConnection.

/**
   * Create Topology proto object using HeronSubmitter API.
   *
   * @param heronConfig desired config params.
   * @param spouts spoutName -> parallelism
   * @param bolts boltName -> parallelism
   * @param connections connect default stream from value to key.
   * @return topology proto.
   */
public static TopologyAPI.Topology createTopologyWithConnection(String topologyName, Config heronConfig, Map<String, Integer> spouts, Map<String, Integer> bolts, Map<String, String> connections) {
    TopologyBuilder builder = new TopologyBuilder();
    BaseRichSpout baseSpout = new BaseRichSpout() {

        private static final long serialVersionUID = -719523487475322625L;

        public void declareOutputFields(OutputFieldsDeclarer declarer) {
            declarer.declare(new Fields("field1"));
        }

        public void open(Map<String, Object> conf, TopologyContext context, SpoutOutputCollector collector) {
        }

        public void nextTuple() {
        }
    };
    BaseBasicBolt basicBolt = new BaseBasicBolt() {

        private static final long serialVersionUID = 2544765902130713628L;

        public void execute(Tuple input, BasicOutputCollector collector) {
        }

        public void declareOutputFields(OutputFieldsDeclarer declarer) {
        }
    };
    for (String spout : spouts.keySet()) {
        builder.setSpout(spout, baseSpout, spouts.get(spout));
    }
    for (String bolt : bolts.keySet()) {
        BoltDeclarer boltDeclarer = builder.setBolt(bolt, basicBolt, bolts.get(bolt));
        if (connections.containsKey(bolt)) {
            boltDeclarer.shuffleGrouping(connections.get(bolt));
        }
    }
    HeronTopology heronTopology = builder.createTopology();
    return heronTopology.setName(topologyName).setConfig(heronConfig).setState(TopologyAPI.TopologyState.RUNNING).getTopology();
}
Also used : BaseBasicBolt(com.twitter.heron.api.bolt.BaseBasicBolt) TopologyBuilder(com.twitter.heron.api.topology.TopologyBuilder) HeronTopology(com.twitter.heron.api.HeronTopology) OutputFieldsDeclarer(com.twitter.heron.api.topology.OutputFieldsDeclarer) BasicOutputCollector(com.twitter.heron.api.bolt.BasicOutputCollector) Fields(com.twitter.heron.api.tuple.Fields) BoltDeclarer(com.twitter.heron.api.topology.BoltDeclarer) SpoutOutputCollector(com.twitter.heron.api.spout.SpoutOutputCollector) TopologyContext(com.twitter.heron.api.topology.TopologyContext) HashMap(java.util.HashMap) Map(java.util.Map) Tuple(com.twitter.heron.api.tuple.Tuple) BaseRichSpout(com.twitter.heron.api.spout.BaseRichSpout)

Aggregations

HeronTopology (com.twitter.heron.api.HeronTopology)1 BaseBasicBolt (com.twitter.heron.api.bolt.BaseBasicBolt)1 BasicOutputCollector (com.twitter.heron.api.bolt.BasicOutputCollector)1 BaseRichSpout (com.twitter.heron.api.spout.BaseRichSpout)1 SpoutOutputCollector (com.twitter.heron.api.spout.SpoutOutputCollector)1 BoltDeclarer (com.twitter.heron.api.topology.BoltDeclarer)1 OutputFieldsDeclarer (com.twitter.heron.api.topology.OutputFieldsDeclarer)1 TopologyBuilder (com.twitter.heron.api.topology.TopologyBuilder)1 TopologyContext (com.twitter.heron.api.topology.TopologyContext)1 Fields (com.twitter.heron.api.tuple.Fields)1 Tuple (com.twitter.heron.api.tuple.Tuple)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1