Search in sources :

Example 1 with SliceAttachedWindowingStrategy

use of org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy in project flink by apache.

the class StreamExecWindowAggregateBase method createSliceAssigner.

// ------------------------------------------------------------------------------------------
// Utilities
// ------------------------------------------------------------------------------------------
protected SliceAssigner createSliceAssigner(WindowingStrategy windowingStrategy, ZoneId shiftTimeZone) {
    WindowSpec windowSpec = windowingStrategy.getWindow();
    if (windowingStrategy instanceof WindowAttachedWindowingStrategy) {
        int windowEndIndex = ((WindowAttachedWindowingStrategy) windowingStrategy).getWindowEnd();
        // we don't need time attribute to assign windows, use a magic value in this case
        SliceAssigner innerAssigner = createSliceAssigner(windowSpec, Integer.MAX_VALUE, shiftTimeZone);
        return SliceAssigners.windowed(windowEndIndex, innerAssigner);
    } else if (windowingStrategy instanceof SliceAttachedWindowingStrategy) {
        int sliceEndIndex = ((SliceAttachedWindowingStrategy) windowingStrategy).getSliceEnd();
        // we don't need time attribute to assign windows, use a magic value in this case
        SliceAssigner innerAssigner = createSliceAssigner(windowSpec, Integer.MAX_VALUE, shiftTimeZone);
        return SliceAssigners.sliced(sliceEndIndex, innerAssigner);
    } else if (windowingStrategy instanceof TimeAttributeWindowingStrategy) {
        final int timeAttributeIndex;
        if (windowingStrategy.isRowtime()) {
            timeAttributeIndex = ((TimeAttributeWindowingStrategy) windowingStrategy).getTimeAttributeIndex();
        } else {
            timeAttributeIndex = -1;
        }
        return createSliceAssigner(windowSpec, timeAttributeIndex, shiftTimeZone);
    } else {
        throw new UnsupportedOperationException(windowingStrategy + " is not supported yet.");
    }
}
Also used : SliceAssigner(org.apache.flink.table.runtime.operators.window.slicing.SliceAssigner) TimeAttributeWindowingStrategy(org.apache.flink.table.planner.plan.logical.TimeAttributeWindowingStrategy) WindowAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.WindowAttachedWindowingStrategy) SliceAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy) TumblingWindowSpec(org.apache.flink.table.planner.plan.logical.TumblingWindowSpec) WindowSpec(org.apache.flink.table.planner.plan.logical.WindowSpec) HoppingWindowSpec(org.apache.flink.table.planner.plan.logical.HoppingWindowSpec) CumulativeWindowSpec(org.apache.flink.table.planner.plan.logical.CumulativeWindowSpec)

Example 2 with SliceAttachedWindowingStrategy

use of org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy in project flink by apache.

the class TwoStageOptimizedWindowAggregateRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final StreamPhysicalWindowAggregate windowAgg = call.rel(0);
    final RelNode realInput = call.rel(2);
    final WindowingStrategy windowing = windowAgg.windowing();
    RelTraitSet localTraitSet = realInput.getTraitSet().plus(ModifyKindSetTrait.INSERT_ONLY()).plus(UpdateKindTrait.NONE());
    StreamPhysicalLocalWindowAggregate localAgg = new StreamPhysicalLocalWindowAggregate(windowAgg.getCluster(), localTraitSet, realInput, windowAgg.grouping(), windowAgg.aggCalls(), windowing);
    // grouping keys is forwarded by local agg, use indices instead of groupings
    int[] globalGrouping = IntStream.range(0, windowAgg.grouping().length).toArray();
    FlinkRelDistribution globalDistribution = createDistribution(globalGrouping);
    // create exchange if needed
    RelNode newInput = FlinkExpandConversionRule.satisfyDistribution(FlinkConventions.STREAM_PHYSICAL(), localAgg, globalDistribution);
    RelTraitSet globalAggProvidedTraitSet = windowAgg.getTraitSet();
    // we put sliceEnd/windowEnd at the end of local output fields
    int endIndex = localAgg.getRowType().getFieldCount() - 1;
    final WindowingStrategy globalWindowing;
    if (windowing instanceof TimeAttributeWindowingStrategy) {
        globalWindowing = new SliceAttachedWindowingStrategy(windowing.getWindow(), windowing.getTimeAttributeType(), endIndex);
    } else {
        globalWindowing = new WindowAttachedWindowingStrategy(windowing.getWindow(), windowing.getTimeAttributeType(), endIndex);
    }
    StreamPhysicalGlobalWindowAggregate globalAgg = new StreamPhysicalGlobalWindowAggregate(windowAgg.getCluster(), globalAggProvidedTraitSet, newInput, realInput.getRowType(), globalGrouping, windowAgg.aggCalls(), globalWindowing, windowAgg.namedWindowProperties());
    call.transformTo(globalAgg);
}
Also used : StreamPhysicalLocalWindowAggregate(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalLocalWindowAggregate) FlinkRelDistribution(org.apache.flink.table.planner.plan.trait.FlinkRelDistribution) RelNode(org.apache.calcite.rel.RelNode) StreamPhysicalWindowAggregate(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalWindowAggregate) TimeAttributeWindowingStrategy(org.apache.flink.table.planner.plan.logical.TimeAttributeWindowingStrategy) SliceAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy) WindowAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.WindowAttachedWindowingStrategy) RelTraitSet(org.apache.calcite.plan.RelTraitSet) StreamPhysicalGlobalWindowAggregate(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalGlobalWindowAggregate) WindowAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.WindowAttachedWindowingStrategy) TimeAttributeWindowingStrategy(org.apache.flink.table.planner.plan.logical.TimeAttributeWindowingStrategy) WindowingStrategy(org.apache.flink.table.planner.plan.logical.WindowingStrategy) SliceAttachedWindowingStrategy(org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy)

Aggregations

SliceAttachedWindowingStrategy (org.apache.flink.table.planner.plan.logical.SliceAttachedWindowingStrategy)2 TimeAttributeWindowingStrategy (org.apache.flink.table.planner.plan.logical.TimeAttributeWindowingStrategy)2 WindowAttachedWindowingStrategy (org.apache.flink.table.planner.plan.logical.WindowAttachedWindowingStrategy)2 RelTraitSet (org.apache.calcite.plan.RelTraitSet)1 RelNode (org.apache.calcite.rel.RelNode)1 CumulativeWindowSpec (org.apache.flink.table.planner.plan.logical.CumulativeWindowSpec)1 HoppingWindowSpec (org.apache.flink.table.planner.plan.logical.HoppingWindowSpec)1 TumblingWindowSpec (org.apache.flink.table.planner.plan.logical.TumblingWindowSpec)1 WindowSpec (org.apache.flink.table.planner.plan.logical.WindowSpec)1 WindowingStrategy (org.apache.flink.table.planner.plan.logical.WindowingStrategy)1 StreamPhysicalGlobalWindowAggregate (org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalGlobalWindowAggregate)1 StreamPhysicalLocalWindowAggregate (org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalLocalWindowAggregate)1 StreamPhysicalWindowAggregate (org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalWindowAggregate)1 FlinkRelDistribution (org.apache.flink.table.planner.plan.trait.FlinkRelDistribution)1 SliceAssigner (org.apache.flink.table.runtime.operators.window.slicing.SliceAssigner)1