Search in sources :

Example 1 with GroupTransform

use of com.hazelcast.jet.impl.pipeline.transform.GroupTransform in project hazelcast by hazelcast.

the class GrAggBuilder method buildBatch.

public <A, R, OUT> BatchStage<OUT> buildBatch(@Nonnull AggregateOperation<A, ? extends R> aggrOp, @Nonnull BiFunctionEx<? super K, ? super R, OUT> mapToOutputFn) {
    checkSerializable(mapToOutputFn, "mapToOutputFn");
    List<Transform> upstreamTransforms = toList(upstreamStages, s -> s.transform);
    AbstractTransform transform = new GroupTransform<>(upstreamTransforms, keyFns, aggrOp, mapToOutputFn);
    pipelineImpl.connect(upstreamStages, transform);
    return new BatchStageImpl<>(transform, pipelineImpl);
}
Also used : AbstractTransform(com.hazelcast.jet.impl.pipeline.transform.AbstractTransform) WindowGroupTransform(com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform) GroupTransform(com.hazelcast.jet.impl.pipeline.transform.GroupTransform) WindowGroupTransform(com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform) GroupTransform(com.hazelcast.jet.impl.pipeline.transform.GroupTransform) AbstractTransform(com.hazelcast.jet.impl.pipeline.transform.AbstractTransform) Transform(com.hazelcast.jet.impl.pipeline.transform.Transform)

Example 2 with GroupTransform

use of com.hazelcast.jet.impl.pipeline.transform.GroupTransform in project hazelcast-jet by hazelcast.

the class GrAggBuilder method buildBatch.

@SuppressWarnings("unchecked")
public <A, R, OUT> BatchStage<OUT> buildBatch(@Nonnull AggregateOperation<A, ? extends R> aggrOp, @Nonnull DistributedBiFunction<? super K, ? super R, OUT> mapToOutputFn) {
    List<Transform> upstreamTransforms = upstreamStages.stream().map(s -> s.transform).collect(toList());
    Transform transform = new GroupTransform<>(upstreamTransforms, keyFns, aggrOp, mapToOutputFn);
    pipelineImpl.connect(upstreamTransforms, transform);
    return new BatchStageImpl<>(transform, pipelineImpl);
}
Also used : WindowGroupTransform(com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform) DistributedBiFunction(com.hazelcast.jet.function.DistributedBiFunction) ArrayList(java.util.ArrayList) BatchStage(com.hazelcast.jet.pipeline.BatchStage) Tag.tag(com.hazelcast.jet.datamodel.Tag.tag) KeyedWindowResultFunction(com.hazelcast.jet.function.KeyedWindowResultFunction) AggregateOperation(com.hazelcast.jet.aggregate.AggregateOperation) WindowGroupAggregateBuilder(com.hazelcast.jet.pipeline.WindowGroupAggregateBuilder) StageWithGroupingAndWindow(com.hazelcast.jet.pipeline.StageWithGroupingAndWindow) DistributedFunction(com.hazelcast.jet.function.DistributedFunction) GroupTransform(com.hazelcast.jet.impl.pipeline.transform.GroupTransform) Nonnull(javax.annotation.Nonnull) StreamStage(com.hazelcast.jet.pipeline.StreamStage) WindowDefinition(com.hazelcast.jet.pipeline.WindowDefinition) StageWithGrouping(com.hazelcast.jet.pipeline.StageWithGrouping) JetEventFunctionAdapter.adaptAggregateOperation(com.hazelcast.jet.impl.pipeline.JetEventFunctionAdapter.adaptAggregateOperation) Tag(com.hazelcast.jet.datamodel.Tag) Transform(com.hazelcast.jet.impl.pipeline.transform.Transform) ADAPT_TO_JET_EVENT(com.hazelcast.jet.impl.pipeline.ComputeStageImplBase.ADAPT_TO_JET_EVENT) List(java.util.List) ComputeStageImplBase.ensureJetEvents(com.hazelcast.jet.impl.pipeline.ComputeStageImplBase.ensureJetEvents) Collectors.toList(java.util.stream.Collectors.toList) JetEventFunctionAdapter.adaptKeyFn(com.hazelcast.jet.impl.pipeline.JetEventFunctionAdapter.adaptKeyFn) StreamStageWithGrouping(com.hazelcast.jet.pipeline.StreamStageWithGrouping) GroupAggregateBuilder(com.hazelcast.jet.pipeline.GroupAggregateBuilder) WindowGroupTransform(com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform) GroupTransform(com.hazelcast.jet.impl.pipeline.transform.GroupTransform) WindowGroupTransform(com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform) GroupTransform(com.hazelcast.jet.impl.pipeline.transform.GroupTransform) Transform(com.hazelcast.jet.impl.pipeline.transform.Transform)

Aggregations

GroupTransform (com.hazelcast.jet.impl.pipeline.transform.GroupTransform)2 Transform (com.hazelcast.jet.impl.pipeline.transform.Transform)2 WindowGroupTransform (com.hazelcast.jet.impl.pipeline.transform.WindowGroupTransform)2 AggregateOperation (com.hazelcast.jet.aggregate.AggregateOperation)1 Tag (com.hazelcast.jet.datamodel.Tag)1 Tag.tag (com.hazelcast.jet.datamodel.Tag.tag)1 DistributedBiFunction (com.hazelcast.jet.function.DistributedBiFunction)1 DistributedFunction (com.hazelcast.jet.function.DistributedFunction)1 KeyedWindowResultFunction (com.hazelcast.jet.function.KeyedWindowResultFunction)1 ADAPT_TO_JET_EVENT (com.hazelcast.jet.impl.pipeline.ComputeStageImplBase.ADAPT_TO_JET_EVENT)1 ComputeStageImplBase.ensureJetEvents (com.hazelcast.jet.impl.pipeline.ComputeStageImplBase.ensureJetEvents)1 JetEventFunctionAdapter.adaptAggregateOperation (com.hazelcast.jet.impl.pipeline.JetEventFunctionAdapter.adaptAggregateOperation)1 JetEventFunctionAdapter.adaptKeyFn (com.hazelcast.jet.impl.pipeline.JetEventFunctionAdapter.adaptKeyFn)1 AbstractTransform (com.hazelcast.jet.impl.pipeline.transform.AbstractTransform)1 BatchStage (com.hazelcast.jet.pipeline.BatchStage)1 GroupAggregateBuilder (com.hazelcast.jet.pipeline.GroupAggregateBuilder)1 StageWithGrouping (com.hazelcast.jet.pipeline.StageWithGrouping)1 StageWithGroupingAndWindow (com.hazelcast.jet.pipeline.StageWithGroupingAndWindow)1 StreamStage (com.hazelcast.jet.pipeline.StreamStage)1 StreamStageWithGrouping (com.hazelcast.jet.pipeline.StreamStageWithGrouping)1