Examples with Node storm.trident.planner.Node used on opensource projects

Example 21 with Node

use of storm.trident.planner.Node in project storm by nathanmarz.

the class TridentTopology method getGroupName.

private static String getGroupName(Group g) {
    TreeMap<Integer, String> sortedNames = new TreeMap();
    for (Node n : g.nodes) {
        if (n.name != null) {
            sortedNames.put(n.creationIndex, n.name);
        }
    }
    List<String> names = new ArrayList<String>();
    String prevName = null;
    for (String n : sortedNames.values()) {
        if (prevName == null || !n.equals(prevName)) {
            prevName = n;
            names.add(n);
        }
    }
    return Utils.join(names, "-");
}

Example 22 with Node

use of storm.trident.planner.Node in project storm by nathanmarz.

the class TridentTopology method genSpoutIds.

private static Map<Node, String> genSpoutIds(Collection<SpoutNode> spoutNodes) {
    Map<Node, String> ret = new HashMap();
    int ctr = 0;
    for (SpoutNode n : spoutNodes) {
        ret.put(n, "spout" + ctr);
        ctr++;
    }
    return ret;
}

Example 23 with Node

use of storm.trident.planner.Node in project storm by nathanmarz.

the class TridentTopology method getFixedParallelism.

private static Integer getFixedParallelism(Set<Group> groups) {
    Integer ret = null;
    for (Group g : groups) {
        for (Node n : g.nodes) {
            if (n.stateInfo != null && n.stateInfo.spec.requiredNumPartitions != null) {
                int reqPartitions = n.stateInfo.spec.requiredNumPartitions;
                if (ret != null && ret != reqPartitions) {
                    throw new RuntimeException("Cannot have one group have fixed parallelism of two different values");
                }
                ret = reqPartitions;
            }
        }
    }
    return ret;
}

Example 24 with Node

use of storm.trident.planner.Node in project storm by nathanmarz.

the class Stream method stateQuery.

public Stream stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) {
    projectionValidation(inputFields);
    String stateId = state._node.stateInfo.id;
    Node n = new ProcessorNode(_topology.getUniqueStreamId(), _name, TridentUtils.fieldsConcat(getOutputFields(), functionFields), functionFields, new StateQueryProcessor(stateId, inputFields, function));
    _topology._colocate.get(stateId).add(n);
    return _topology.addSourcedNode(this, n);
}

Example 25 with Node

use of storm.trident.planner.Node in project storm by nathanmarz.

the class TridentTopology method multiReduce.

public Stream multiReduce(List<Fields> inputFields, List<Stream> streams, MultiReducer function, Fields outputFields) {
    List<String> names = new ArrayList<String>();
    for (Stream s : streams) {
        if (s._name != null) {
            names.add(s._name);
        }
    }
    Node n = new ProcessorNode(getUniqueStreamId(), Utils.join(names, "-"), outputFields, outputFields, new MultiReducerProcessor(inputFields, function));
    return addSourcedNode(streams, n);
}