Examples with GroupedMultiReducerExecutor storm.trident.operation.impl.GroupedMultiReducerExecutor used on opensource projects

Example 1 with GroupedMultiReducerExecutor

use of storm.trident.operation.impl.GroupedMultiReducerExecutor in project storm by nathanmarz.

the class TridentTopology method multiReduce.

public Stream multiReduce(List<Fields> inputFields, List<GroupedStream> groupedStreams, GroupedMultiReducer function, Fields outputFields) {
    List<Fields> fullInputFields = new ArrayList<Fields>();
    List<Stream> streams = new ArrayList<Stream>();
    List<Fields> fullGroupFields = new ArrayList<Fields>();
    for (int i = 0; i < groupedStreams.size(); i++) {
        GroupedStream gs = groupedStreams.get(i);
        Fields groupFields = gs.getGroupFields();
        fullGroupFields.add(groupFields);
        streams.add(gs.toStream().partitionBy(groupFields));
        fullInputFields.add(TridentUtils.fieldsUnion(groupFields, inputFields.get(i)));
    }
    return multiReduce(fullInputFields, streams, new GroupedMultiReducerExecutor(function, fullGroupFields, inputFields), outputFields);
}

Example 2 with GroupedMultiReducerExecutor

use of storm.trident.operation.impl.GroupedMultiReducerExecutor in project jstorm by alibaba.

the class TridentTopology method multiReduce.

public Stream multiReduce(List<Fields> inputFields, List<GroupedStream> groupedStreams, GroupedMultiReducer function, Fields outputFields) {
    List<Fields> fullInputFields = new ArrayList<>();
    List<Stream> streams = new ArrayList<>();
    List<Fields> fullGroupFields = new ArrayList<>();
    for (int i = 0; i < groupedStreams.size(); i++) {
        GroupedStream gs = groupedStreams.get(i);
        Fields groupFields = gs.getGroupFields();
        fullGroupFields.add(groupFields);
        streams.add(gs.toStream().partitionBy(groupFields));
        fullInputFields.add(TridentUtils.fieldsUnion(groupFields, inputFields.get(i)));
    }
    return multiReduce(fullInputFields, streams, new GroupedMultiReducerExecutor(function, fullGroupFields, inputFields), outputFields);
}