Search in sources :

Example 11 with SpoutNode

use of storm.trident.planner.SpoutNode in project storm by nathanmarz.

the class TridentTopology method getMaxParallelism.

private static Integer getMaxParallelism(Set<Group> groups) {
    Integer ret = null;
    for (Group g : groups) {
        if (isSpoutGroup(g)) {
            SpoutNode n = (SpoutNode) g.nodes.iterator().next();
            Map conf = getSpoutComponentConfig(n.spout);
            if (conf == null)
                conf = new HashMap();
            Number maxP = (Number) conf.get(Config.TOPOLOGY_MAX_TASK_PARALLELISM);
            if (maxP != null) {
                if (ret == null)
                    ret = maxP.intValue();
                else
                    ret = Math.min(ret, maxP.intValue());
            }
        }
    }
    return ret;
}
Also used : Group(storm.trident.graph.Group) SpoutNode(storm.trident.planner.SpoutNode) HashMap(java.util.HashMap) Map(java.util.Map) HashMap(java.util.HashMap) TreeMap(java.util.TreeMap)

Example 12 with SpoutNode

use of storm.trident.planner.SpoutNode in project storm by nathanmarz.

the class TridentTopology method genSpoutIds.

private static Map<Node, String> genSpoutIds(Collection<SpoutNode> spoutNodes) {
    Map<Node, String> ret = new HashMap();
    int ctr = 0;
    for (SpoutNode n : spoutNodes) {
        ret.put(n, "spout" + ctr);
        ctr++;
    }
    return ret;
}
Also used : HashMap(java.util.HashMap) SpoutNode(storm.trident.planner.SpoutNode) SpoutNode(storm.trident.planner.SpoutNode) ProcessorNode(storm.trident.planner.ProcessorNode) PartitionNode(storm.trident.planner.PartitionNode) Node(storm.trident.planner.Node)

Aggregations

SpoutNode (storm.trident.planner.SpoutNode)12 Node (storm.trident.planner.Node)10 PartitionNode (storm.trident.planner.PartitionNode)10 ProcessorNode (storm.trident.planner.ProcessorNode)10 HashMap (java.util.HashMap)6 Fields (backtype.storm.tuple.Fields)4 HashSet (java.util.HashSet)4 Set (java.util.Set)4 GroupedStream (storm.trident.fluent.GroupedStream)4 IAggregatableStream (storm.trident.fluent.IAggregatableStream)4 Group (storm.trident.graph.Group)4 LinkedHashMap (java.util.LinkedHashMap)3 Map (java.util.Map)3 TreeMap (java.util.TreeMap)3 IndexedEdge (storm.trident.util.IndexedEdge)3 GlobalStreamId (backtype.storm.generated.GlobalStreamId)2 BoltDeclarer (backtype.storm.topology.BoltDeclarer)2 ArrayList (java.util.ArrayList)2 LinkedHashSet (java.util.LinkedHashSet)2 DefaultDirectedGraph (org.jgrapht.graph.DefaultDirectedGraph)2