Search in sources :

Example 1 with ReturnResultsReducer

use of org.apache.storm.trident.drpc.ReturnResultsReducer in project storm by apache.

the class TridentTopology method completeDRPC.

private static void completeDRPC(DefaultDirectedGraph<Node, IndexedEdge> graph, Map<String, List<Node>> colocate, UniqueIdGen gen) {
    List<Set<Node>> connectedComponents = new ConnectivityInspector<>(graph).connectedSets();
    for (Set<Node> g : connectedComponents) {
        checkValidJoins(g);
    }
    TridentTopology helper = new TridentTopology(graph, colocate, gen);
    for (Set<Node> g : connectedComponents) {
        SpoutNode drpcNode = getDRPCSpoutNode(g);
        if (drpcNode != null) {
            Stream lastStream = new Stream(helper, null, getLastAddedNode(g));
            Stream s = new Stream(helper, null, drpcNode);
            helper.multiReduce(s.project(new Fields("return-info")).batchGlobal(), lastStream.batchGlobal(), new ReturnResultsReducer(), new Fields());
        }
    }
}
Also used : Fields(org.apache.storm.tuple.Fields) SpoutNode(org.apache.storm.trident.planner.SpoutNode) ProcessorNode(org.apache.storm.trident.planner.ProcessorNode) Node(org.apache.storm.trident.planner.Node) PartitionNode(org.apache.storm.trident.planner.PartitionNode) SpoutNode(org.apache.storm.trident.planner.SpoutNode) GroupedStream(org.apache.storm.trident.fluent.GroupedStream) IAggregatableStream(org.apache.storm.trident.fluent.IAggregatableStream) ReturnResultsReducer(org.apache.storm.trident.drpc.ReturnResultsReducer)

Aggregations

ReturnResultsReducer (org.apache.storm.trident.drpc.ReturnResultsReducer)1 GroupedStream (org.apache.storm.trident.fluent.GroupedStream)1 IAggregatableStream (org.apache.storm.trident.fluent.IAggregatableStream)1 Node (org.apache.storm.trident.planner.Node)1 PartitionNode (org.apache.storm.trident.planner.PartitionNode)1 ProcessorNode (org.apache.storm.trident.planner.ProcessorNode)1 SpoutNode (org.apache.storm.trident.planner.SpoutNode)1 Fields (org.apache.storm.tuple.Fields)1