Examples with Node org.apache.storm.trident.planner.Node used on opensource projects

Example 1 with Node

use of org.apache.storm.trident.planner.Node in project storm by apache.

the class Stream method stateQuery.

public Stream stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) {
    projectionValidation(inputFields);
    String stateId = state._node.stateInfo.id;
    Node n = new ProcessorNode(_topology.getUniqueStreamId(), _name, TridentUtils.fieldsConcat(getOutputFields(), functionFields), functionFields, new StateQueryProcessor(stateId, inputFields, function));
    _topology._colocate.get(stateId).add(n);
    return _topology.addSourcedNode(this, n);
}

Example 2 with Node

use of org.apache.storm.trident.planner.Node in project storm by apache.

the class TridentTopology method genSpoutIds.

private static Map<Node, String> genSpoutIds(Collection<SpoutNode> spoutNodes) {
    Map<Node, String> ret = new HashMap<>();
    int ctr = 0;
    for (SpoutNode n : spoutNodes) {
        if (n.type == SpoutNode.SpoutType.BATCH) {
            // if Batch spout then id contains txId
            ret.put(n, "spout-" + n.txId);
        } else if (n.type == SpoutNode.SpoutType.DRPC) {
            //if DRPC spout then id contains function
            ret.put(n, "spout-" + ((DRPCSpout) n.spout).get_function() + ctr);
            ctr++;
        } else {
            ret.put(n, "spout" + ctr);
            ctr++;
        }
    }
    return ret;
}

Example 3 with Node

use of org.apache.storm.trident.planner.Node in project storm by apache.

the class TridentTopology method completeDRPC.

private static void completeDRPC(DefaultDirectedGraph<Node, IndexedEdge> graph, Map<String, List<Node>> colocate, UniqueIdGen gen) {
    List<Set<Node>> connectedComponents = new ConnectivityInspector<>(graph).connectedSets();
    for (Set<Node> g : connectedComponents) {
        checkValidJoins(g);
    }
    TridentTopology helper = new TridentTopology(graph, colocate, gen);
    for (Set<Node> g : connectedComponents) {
        SpoutNode drpcNode = getDRPCSpoutNode(g);
        if (drpcNode != null) {
            Stream lastStream = new Stream(helper, null, getLastAddedNode(g));
            Stream s = new Stream(helper, null, drpcNode);
            helper.multiReduce(s.project(new Fields("return-info")).batchGlobal(), lastStream.batchGlobal(), new ReturnResultsReducer(), new Fields());
        }
    }
}

Example 4 with Node

use of org.apache.storm.trident.planner.Node in project storm by apache.

the class TridentTopology method getGroupName.

private static String getGroupName(Group g) {
    TreeMap<Integer, String> sortedNames = new TreeMap<>();
    for (Node n : g.nodes) {
        if (n.name != null) {
            sortedNames.put(n.creationIndex, n.name);
        }
    }
    List<String> names = new ArrayList<>();
    String prevName = null;
    for (String n : sortedNames.values()) {
        if (prevName == null || !n.equals(prevName)) {
            prevName = n;
            names.add(n);
        }
    }
    return Utils.join(names, "-");
}

Example 5 with Node

use of org.apache.storm.trident.planner.Node in project storm by apache.

the class TridentTopology method newStaticState.

public TridentState newStaticState(StateSpec spec) {
    String stateId = getUniqueStateId();
    Node n = new Node(getUniqueStreamId(), null, new Fields());
    n.stateInfo = new NodeStateInfo(stateId, spec);
    registerNode(n);
    return new TridentState(this, n);
}