Examples with FlinkExecutableStagePruningFunction org.apache.beam.runners.flink.translation.functions.FlinkExecutableStagePruningFunction used on opensource projects

Example 1 with FlinkExecutableStagePruningFunction

use of org.apache.beam.runners.flink.translation.functions.FlinkExecutableStagePruningFunction in project beam by apache.

the class FlinkBatchPortablePipelineTranslator method pruneOutput.

private static void pruneOutput(DataSet<RawUnionValue> taggedDataset, BatchTranslationContext context, int unionTag, Coder<WindowedValue<?>> outputCoder, String collectionId) {
    TypeInformation<WindowedValue<?>> outputType = new CoderTypeInformation<>(outputCoder, context.getPipelineOptions());
    FlinkExecutableStagePruningFunction pruningFunction = new FlinkExecutableStagePruningFunction(unionTag, context.getPipelineOptions());
    FlatMapOperator<RawUnionValue, WindowedValue<?>> pruningOperator = new FlatMapOperator<>(taggedDataset, outputType, pruningFunction, String.format("ExtractOutput[%s]", unionTag));
    context.addDataSet(collectionId, pruningOperator);
}