Search in sources :

Example 1 with Fields

use of com.twitter.heron.api.tuple.Fields in project heron by twitter.

the class GeneralTopologyContextImpl method getOutputToComponentsFields.

public static Map<String, Map<String, Fields>> getOutputToComponentsFields(List<TopologyAPI.OutputStream> outputs) {
    Map<String, Map<String, Fields>> outputFields = new HashMap<>();
    for (TopologyAPI.OutputStream outputStream : outputs) {
        String componentName = outputStream.getStream().getComponentName();
        String streamId = outputStream.getStream().getId();
        Map<String, Fields> componentFields = outputFields.get(componentName);
        if (componentFields == null) {
            componentFields = new HashMap<>();
        }
        // Get the fields of a particular OutputStream
        List<String> retval = new ArrayList<>();
        for (TopologyAPI.StreamSchema.KeyType kt : outputStream.getSchema().getKeysList()) {
            retval.add(kt.getKey());
        }
        // Put it into the map
        componentFields.put(streamId, new Fields(retval));
        outputFields.put(componentName, componentFields);
    }
    return outputFields;
}
Also used : HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) TopologyAPI(com.twitter.heron.api.generated.TopologyAPI) Fields(com.twitter.heron.api.tuple.Fields) Map(java.util.Map) HashMap(java.util.HashMap)

Example 2 with Fields

use of com.twitter.heron.api.tuple.Fields in project heron by twitter.

the class MultiSpoutsMultiTasks method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout-1", new ABSpout(), 3);
    builder.setSpout("ab-spout-2", new ABSpout(), 3);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 1).shuffleGrouping("ab-spout-1").shuffleGrouping("ab-spout-2");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 3 with Fields

use of com.twitter.heron.api.tuple.Fields in project heron by twitter.

the class NonGrouping method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 1);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 3).noneGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 4 with Fields

use of com.twitter.heron.api.tuple.Fields in project heron by twitter.

the class OneSpoutBoltMultiTasks method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 3);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 3).shuffleGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Example 5 with Fields

use of com.twitter.heron.api.tuple.Fields in project heron by twitter.

the class AllGrouping method buildTopology.

@Override
protected TestTopologyBuilder buildTopology(TestTopologyBuilder builder) {
    builder.setSpout("ab-spout", new ABSpout(), 1);
    builder.setBolt("identity-bolt", new IdentityBolt(new Fields("word")), 2).allGrouping("ab-spout");
    return builder;
}
Also used : IdentityBolt(com.twitter.heron.integration_test.common.bolt.IdentityBolt) Fields(com.twitter.heron.api.tuple.Fields) ABSpout(com.twitter.heron.integration_test.common.spout.ABSpout)

Aggregations

Fields (com.twitter.heron.api.tuple.Fields)50 ABSpout (com.twitter.heron.integration_test.common.spout.ABSpout)22 IdentityBolt (com.twitter.heron.integration_test.common.bolt.IdentityBolt)20 Config (com.twitter.heron.api.Config)13 TopologyBuilder (com.twitter.heron.api.topology.TopologyBuilder)13 Tuple (com.twitter.heron.api.tuple.Tuple)8 TopologyArgParser (com.github.ashvina.common.TopologyArgParser)5 TopologyAPI (com.twitter.heron.api.generated.TopologyAPI)5 TopologyContext (com.twitter.heron.api.topology.TopologyContext)5 Values (com.twitter.heron.api.tuple.Values)5 HashMap (java.util.HashMap)5 BaseBasicBolt (com.twitter.heron.api.bolt.BaseBasicBolt)4 BasicOutputCollector (com.twitter.heron.api.bolt.BasicOutputCollector)4 BaseRichSpout (com.twitter.heron.api.spout.BaseRichSpout)4 SpoutOutputCollector (com.twitter.heron.api.spout.SpoutOutputCollector)4 OutputFieldsDeclarer (com.twitter.heron.api.topology.OutputFieldsDeclarer)4 TupleWindow (com.twitter.heron.api.windowing.TupleWindow)4 Map (java.util.Map)4 TupleWindowImpl (com.twitter.heron.api.windowing.TupleWindowImpl)3 ArrayList (java.util.ArrayList)3