Search in sources :

Example 6 with InternalIterableAllWindowFunction

use of org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableAllWindowFunction in project flink by apache.

the class AllWindowedStream method apply.

/**
	 * Applies the given window function to each window. The window function is called for each
	 * evaluation of the window for each key individually. The output of the window function is
	 * interpreted as a regular non-windowed stream.
	 *
	 * <p>
	 * Arriving data is incrementally aggregated using the given fold function.
	 *
	 * @param initialValue The initial value of the fold.
	 * @param foldFunction The fold function that is used for incremental aggregation.
	 * @param function The window function.
	 * @param resultType Type information for the result type of the window function
	 * @return The data stream that is the result of applying the window function to the window.
	 *
	 * @deprecated Use {@link #fold(Object, FoldFunction, AllWindowFunction, TypeInformation, TypeInformation)} instead.
	 */
@Deprecated
public <R> SingleOutputStreamOperator<R> apply(R initialValue, FoldFunction<T, R> foldFunction, AllWindowFunction<R, R, W> function, TypeInformation<R> resultType) {
    if (foldFunction instanceof RichFunction) {
        throw new UnsupportedOperationException("ReduceFunction of apply can not be a RichFunction.");
    }
    if (windowAssigner instanceof MergingWindowAssigner) {
        throw new UnsupportedOperationException("Fold cannot be used with a merging WindowAssigner.");
    }
    //clean the closures
    function = input.getExecutionEnvironment().clean(function);
    foldFunction = input.getExecutionEnvironment().clean(foldFunction);
    String callLocation = Utils.getCallLocationName();
    String udfName = "AllWindowedStream." + callLocation;
    String opName;
    KeySelector<T, Byte> keySel = input.getKeySelector();
    OneInputStreamOperator<T, R> operator;
    if (evictor != null) {
        @SuppressWarnings({ "unchecked", "rawtypes" }) TypeSerializer<StreamRecord<T>> streamRecordSerializer = (TypeSerializer<StreamRecord<T>>) new StreamElementSerializer(input.getType().createSerializer(getExecutionEnvironment().getConfig()));
        ListStateDescriptor<StreamRecord<T>> stateDesc = new ListStateDescriptor<>("window-contents", streamRecordSerializer);
        opName = "TriggerWindow(" + windowAssigner + ", " + stateDesc + ", " + trigger + ", " + evictor + ", " + udfName + ")";
        operator = new EvictingWindowOperator<>(windowAssigner, windowAssigner.getWindowSerializer(getExecutionEnvironment().getConfig()), keySel, input.getKeyType().createSerializer(getExecutionEnvironment().getConfig()), stateDesc, new InternalIterableAllWindowFunction<>(new FoldApplyAllWindowFunction<>(initialValue, foldFunction, function, resultType)), trigger, evictor, allowedLateness, lateDataOutputTag);
    } else {
        FoldingStateDescriptor<T, R> stateDesc = new FoldingStateDescriptor<>("window-contents", initialValue, foldFunction, resultType.createSerializer(getExecutionEnvironment().getConfig()));
        opName = "TriggerWindow(" + windowAssigner + ", " + stateDesc + ", " + trigger + ", " + udfName + ")";
        operator = new WindowOperator<>(windowAssigner, windowAssigner.getWindowSerializer(getExecutionEnvironment().getConfig()), keySel, input.getKeyType().createSerializer(getExecutionEnvironment().getConfig()), stateDesc, new InternalSingleValueAllWindowFunction<>(function), trigger, allowedLateness, lateDataOutputTag);
    }
    return input.transform(opName, resultType, operator).forceNonParallel();
}
Also used : StreamRecord(org.apache.flink.streaming.runtime.streamrecord.StreamRecord) RichFunction(org.apache.flink.api.common.functions.RichFunction) ListStateDescriptor(org.apache.flink.api.common.state.ListStateDescriptor) FoldingStateDescriptor(org.apache.flink.api.common.state.FoldingStateDescriptor) MergingWindowAssigner(org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner) InternalSingleValueAllWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueAllWindowFunction) InternalIterableAllWindowFunction(org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableAllWindowFunction) TypeSerializer(org.apache.flink.api.common.typeutils.TypeSerializer) StreamElementSerializer(org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer)

Aggregations

InternalIterableAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalIterableAllWindowFunction)6 RichFunction (org.apache.flink.api.common.functions.RichFunction)5 ListStateDescriptor (org.apache.flink.api.common.state.ListStateDescriptor)5 TypeSerializer (org.apache.flink.api.common.typeutils.TypeSerializer)5 InternalSingleValueAllWindowFunction (org.apache.flink.streaming.runtime.operators.windowing.functions.InternalSingleValueAllWindowFunction)5 StreamElementSerializer (org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer)5 StreamRecord (org.apache.flink.streaming.runtime.streamrecord.StreamRecord)5 PublicEvolving (org.apache.flink.annotation.PublicEvolving)3 FoldingStateDescriptor (org.apache.flink.api.common.state.FoldingStateDescriptor)2 ReducingStateDescriptor (org.apache.flink.api.common.state.ReducingStateDescriptor)2 MergingWindowAssigner (org.apache.flink.streaming.api.windowing.assigners.MergingWindowAssigner)2 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)1 RuntimeContext (org.apache.flink.api.common.functions.RuntimeContext)1 AggregatingStateDescriptor (org.apache.flink.api.common.state.AggregatingStateDescriptor)1 Configuration (org.apache.flink.configuration.Configuration)1 TimeWindow (org.apache.flink.streaming.api.windowing.windows.TimeWindow)1 Collector (org.apache.flink.util.Collector)1 Test (org.junit.Test)1