Search in sources :

Example 1 with GroupedAggregator

use of storm.trident.operation.impl.GroupedAggregator in project jstorm by alibaba.

the class GroupedStream method partitionAggregate.

@Override
public IAggregatableStream partitionAggregate(Fields inputFields, Aggregator agg, Fields functionFields) {
    Aggregator groupedAgg = new GroupedAggregator(agg, _groupFields, inputFields, functionFields.size());
    Fields allInFields = TridentUtils.fieldsUnion(_groupFields, inputFields);
    Fields allOutFields = TridentUtils.fieldsConcat(_groupFields, functionFields);
    Stream s = _stream.partitionAggregate(allInFields, groupedAgg, allOutFields);
    return new GroupedStream(s, _groupFields);
}
Also used : Fields(backtype.storm.tuple.Fields) GroupedAggregator(storm.trident.operation.impl.GroupedAggregator) CombinerAggregator(storm.trident.operation.CombinerAggregator) Aggregator(storm.trident.operation.Aggregator) ReducerAggregator(storm.trident.operation.ReducerAggregator) GroupedAggregator(storm.trident.operation.impl.GroupedAggregator) Stream(storm.trident.Stream)

Example 2 with GroupedAggregator

use of storm.trident.operation.impl.GroupedAggregator in project storm by nathanmarz.

the class GroupedStream method partitionAggregate.

@Override
public IAggregatableStream partitionAggregate(Fields inputFields, Aggregator agg, Fields functionFields) {
    Aggregator groupedAgg = new GroupedAggregator(agg, _groupFields, inputFields, functionFields.size());
    Fields allInFields = TridentUtils.fieldsUnion(_groupFields, inputFields);
    Fields allOutFields = TridentUtils.fieldsConcat(_groupFields, functionFields);
    Stream s = _stream.partitionAggregate(allInFields, groupedAgg, allOutFields);
    return new GroupedStream(s, _groupFields);
}
Also used : Fields(backtype.storm.tuple.Fields) GroupedAggregator(storm.trident.operation.impl.GroupedAggregator) CombinerAggregator(storm.trident.operation.CombinerAggregator) Aggregator(storm.trident.operation.Aggregator) ReducerAggregator(storm.trident.operation.ReducerAggregator) GroupedAggregator(storm.trident.operation.impl.GroupedAggregator) Stream(storm.trident.Stream)

Aggregations

Fields (backtype.storm.tuple.Fields)2 Stream (storm.trident.Stream)2 Aggregator (storm.trident.operation.Aggregator)2 CombinerAggregator (storm.trident.operation.CombinerAggregator)2 ReducerAggregator (storm.trident.operation.ReducerAggregator)2 GroupedAggregator (storm.trident.operation.impl.GroupedAggregator)2