Search in sources :

Example 11 with MergingWindowAssigner

use of org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner in project flink by apache.

the class WindowOperator method onEventTime.

@Override
public void onEventTime(InternalTimer<K, W> timer) throws Exception {
    context.key = timer.getKey();
    context.window = timer.getNamespace();
    MergingWindowSet<W> mergingWindows;
    if (windowAssigner instanceof MergingWindowAssigner) {
        mergingWindows = getMergingWindowSet();
        W stateWindow = mergingWindows.getStateWindow(context.window);
        if (stateWindow == null) {
            // window and therefore the Trigger state, however, so nothing to do.
            return;
        } else {
            windowState.setCurrentNamespace(stateWindow);
        }
    } else {
        windowState.setCurrentNamespace(context.window);
        mergingWindows = null;
    }
    ACC contents = null;
    if (windowState != null) {
        contents = windowState.get();
    }
    if (contents != null) {
        TriggerResult triggerResult = context.onEventTime(timer.getTimestamp());
        if (triggerResult.isFire()) {
            emitWindowContents(context.window, contents);
        }
        if (triggerResult.isPurge()) {
            windowState.clear();
        }
    }
    if (windowAssigner.isEventTime() && isCleanupTime(context.window, timer.getTimestamp())) {
        clearAllState(context.window, windowState, mergingWindows);
    }
    if (mergingWindows != null) {
        // need to make sure to update the merging state in state
        mergingWindows.persist();
    }
}
Also used : TriggerResult(org.apache.flink.streaming.api.windowing.triggers.TriggerResult) MergingWindowAssigner(org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner)

Example 12 with MergingWindowAssigner

use of org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner in project flink by apache.

the class EvictingWindowOperator method onEventTime.

@Override
public void onEventTime(InternalTimer<K, W> timer) throws Exception {
    context.key = timer.getKey();
    context.window = timer.getNamespace();
    evictorContext.key = timer.getKey();
    evictorContext.window = timer.getNamespace();
    MergingWindowSet<W> mergingWindows = null;
    if (windowAssigner instanceof MergingWindowAssigner) {
        mergingWindows = getMergingWindowSet();
        W stateWindow = mergingWindows.getStateWindow(context.window);
        if (stateWindow == null) {
            // window and therefore the Trigger state, however, so nothing to do.
            return;
        } else {
            evictingWindowState.setCurrentNamespace(stateWindow);
        }
    } else {
        evictingWindowState.setCurrentNamespace(context.window);
    }
    Iterable<StreamRecord<IN>> contents = evictingWindowState.get();
    if (contents != null) {
        TriggerResult triggerResult = context.onEventTime(timer.getTimestamp());
        if (triggerResult.isFire()) {
            emitWindowContents(context.window, contents, evictingWindowState);
        }
        if (triggerResult.isPurge()) {
            evictingWindowState.clear();
        }
    }
    if (windowAssigner.isEventTime() && isCleanupTime(context.window, timer.getTimestamp())) {
        clearAllState(context.window, evictingWindowState, mergingWindows);
    }
    if (mergingWindows != null) {
        // need to make sure to update the merging state in state
        mergingWindows.persist();
    }
}
Also used : StreamRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord) TriggerResult(org.apache.flink.streaming.api.windowing.triggers.TriggerResult) MergingWindowAssigner(org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner)

Aggregations

MergingWindowAssigner (org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner)12 StreamRecord (org.apache.flink.streaming.runtime.streamrecord.StreamRecord)9 RichFunction (org.apache.flink.api.common.functions.RichFunction)6 FoldingStateDescriptor (org.apache.flink.api.common.state.FoldingStateDescriptor)6 ListStateDescriptor (org.apache.flink.api.common.state.ListStateDescriptor)6 TypeSerializer (org.apache.flink.api.common.typeutils.TypeSerializer)6 TriggerResult (org.apache.flink.streaming.api.windowing.triggers.TriggerResult)6 StreamElementSerializer (org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer)6 PublicEvolving (org.apache.flink.annotation.PublicEvolving)3 InternalIterableAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableAllWindowFunction)2 InternalIterableWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableWindowFunction)2 InternalSingleValueAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueAllWindowFunction)2 InternalSingleValueWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueWindowFunction)2 IOException (java.io.IOException)1 Internal (org.apache.flink.annotation.Internal)1 BaseAlignedWindowAssigner (org.apache.flink.streaming.api.windowing.assigners.BaseAlignedWindowAssigner)1 InternalIterableProcessAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableProcessAllWindowFunction)1 InternalIterableProcessWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableProcessWindowFunction)1 InternalSingleValueProcessAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueProcessAllWindowFunction)1 InternalSingleValueProcessWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueProcessWindowFunction)1