Search in sources :

Example 1 with MultiReducerProcessor

use of storm.trident.planner.processor.MultiReducerProcessor in project jstorm by alibaba.

the class TridentTopology method multiReduce.

public Stream multiReduce(List<Fields> inputFields, List<Stream> streams, MultiReducer function, Fields outputFields) {
    List<String> names = new ArrayList<>();
    for (Stream s : streams) {
        if (s._name != null) {
            names.add(s._name);
        }
    }
    Node n = new ProcessorNode(getUniqueStreamId(), Utils.join(names, "-"), outputFields, outputFields, new MultiReducerProcessor(inputFields, function));
    return addSourcedNode(streams, n);
}
Also used : ProcessorNode(storm.trident.planner.ProcessorNode) SpoutNode(storm.trident.planner.SpoutNode) ProcessorNode(storm.trident.planner.ProcessorNode) PartitionNode(storm.trident.planner.PartitionNode) Node(storm.trident.planner.Node) ArrayList(java.util.ArrayList) GroupedStream(storm.trident.fluent.GroupedStream) IAggregatableStream(storm.trident.fluent.IAggregatableStream) MultiReducerProcessor(storm.trident.planner.processor.MultiReducerProcessor)

Example 2 with MultiReducerProcessor

use of storm.trident.planner.processor.MultiReducerProcessor in project storm by nathanmarz.

the class TridentTopology method multiReduce.

public Stream multiReduce(List<Fields> inputFields, List<Stream> streams, MultiReducer function, Fields outputFields) {
    List<String> names = new ArrayList<String>();
    for (Stream s : streams) {
        if (s._name != null) {
            names.add(s._name);
        }
    }
    Node n = new ProcessorNode(getUniqueStreamId(), Utils.join(names, "-"), outputFields, outputFields, new MultiReducerProcessor(inputFields, function));
    return addSourcedNode(streams, n);
}
Also used : ProcessorNode(storm.trident.planner.ProcessorNode) SpoutNode(storm.trident.planner.SpoutNode) ProcessorNode(storm.trident.planner.ProcessorNode) PartitionNode(storm.trident.planner.PartitionNode) Node(storm.trident.planner.Node) ArrayList(java.util.ArrayList) GroupedStream(storm.trident.fluent.GroupedStream) IAggregatableStream(storm.trident.fluent.IAggregatableStream) MultiReducerProcessor(storm.trident.planner.processor.MultiReducerProcessor)

Aggregations

ArrayList (java.util.ArrayList)2 GroupedStream (storm.trident.fluent.GroupedStream)2 IAggregatableStream (storm.trident.fluent.IAggregatableStream)2 Node (storm.trident.planner.Node)2 PartitionNode (storm.trident.planner.PartitionNode)2 ProcessorNode (storm.trident.planner.ProcessorNode)2 SpoutNode (storm.trident.planner.SpoutNode)2 MultiReducerProcessor (storm.trident.planner.processor.MultiReducerProcessor)2