Search in sources :

Example 1 with RowtimeAttribute

use of org.apache.flink.table.runtime.groupwindow.RowtimeAttribute in project flink by apache.

the class BatchExecPythonGroupWindowAggregate method createPythonOneInputTransformation.

private OneInputTransformation<RowData, RowData> createPythonOneInputTransformation(Transformation<RowData> inputTransform, RowType inputRowType, RowType outputRowType, int maxLimitSize, long windowSize, long slideSize, Configuration pythonConfig, ExecNodeConfig config) {
    int[] namePropertyTypeArray = Arrays.stream(namedWindowProperties).mapToInt(p -> {
        WindowProperty property = p.getProperty();
        if (property instanceof WindowStart) {
            return 0;
        }
        if (property instanceof WindowEnd) {
            return 1;
        }
        if (property instanceof RowtimeAttribute) {
            return 2;
        }
        throw new TableException("Unexpected property " + property);
    }).toArray();
    Tuple2<int[], PythonFunctionInfo[]> aggInfos = CommonPythonUtil.extractPythonAggregateFunctionInfosFromAggregateCall(aggCalls);
    int[] pythonUdafInputOffsets = aggInfos.f0;
    PythonFunctionInfo[] pythonFunctionInfos = aggInfos.f1;
    OneInputStreamOperator<RowData, RowData> pythonOperator = getPythonGroupWindowAggregateFunctionOperator(config, pythonConfig, inputRowType, outputRowType, maxLimitSize, windowSize, slideSize, namePropertyTypeArray, pythonUdafInputOffsets, pythonFunctionInfos);
    return ExecNodeUtil.createOneInputTransformation(inputTransform, createTransformationName(config), createTransformationDescription(config), pythonOperator, InternalTypeInfo.of(outputRowType), inputTransform.getParallelism());
}
Also used : Arrays(java.util.Arrays) InputProperty(org.apache.flink.table.planner.plan.nodes.exec.InputProperty) Tuple2(org.apache.flink.api.java.tuple.Tuple2) RowtimeAttribute(org.apache.flink.table.runtime.groupwindow.RowtimeAttribute) RowType(org.apache.flink.table.types.logical.RowType) Constructor(java.lang.reflect.Constructor) ExecNode(org.apache.flink.table.planner.plan.nodes.exec.ExecNode) ExecNodeUtil(org.apache.flink.table.planner.plan.nodes.exec.utils.ExecNodeUtil) WindowEnd(org.apache.flink.table.runtime.groupwindow.WindowEnd) ManagedMemoryUseCase(org.apache.flink.core.memory.ManagedMemoryUseCase) CodeGeneratorContext(org.apache.flink.table.planner.codegen.CodeGeneratorContext) Projection(org.apache.flink.table.connector.Projection) ProjectionCodeGenerator(org.apache.flink.table.planner.codegen.ProjectionCodeGenerator) WindowCodeGenerator(org.apache.flink.table.planner.codegen.agg.batch.WindowCodeGenerator) ExecNodeContext(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeContext) WindowStart(org.apache.flink.table.runtime.groupwindow.WindowStart) RowData(org.apache.flink.table.data.RowData) PlannerBase(org.apache.flink.table.planner.delegation.PlannerBase) CommonPythonUtil(org.apache.flink.table.planner.plan.nodes.exec.utils.CommonPythonUtil) SingleTransformationTranslator(org.apache.flink.table.planner.plan.nodes.exec.SingleTransformationTranslator) ExecNodeConfig(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeConfig) Configuration(org.apache.flink.configuration.Configuration) TableException(org.apache.flink.table.api.TableException) PythonFunctionInfo(org.apache.flink.table.functions.python.PythonFunctionInfo) OneInputTransformation(org.apache.flink.streaming.api.transformations.OneInputTransformation) InvocationTargetException(java.lang.reflect.InvocationTargetException) InternalTypeInfo(org.apache.flink.table.runtime.typeutils.InternalTypeInfo) ExecEdge(org.apache.flink.table.planner.plan.nodes.exec.ExecEdge) LogicalWindow(org.apache.flink.table.planner.plan.logical.LogicalWindow) AggregateCall(org.apache.calcite.rel.core.AggregateCall) ExecNodeBase(org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase) GeneratedProjection(org.apache.flink.table.runtime.generated.GeneratedProjection) Transformation(org.apache.flink.api.dag.Transformation) OneInputStreamOperator(org.apache.flink.streaming.api.operators.OneInputStreamOperator) ExecutionConfigOptions(org.apache.flink.table.api.config.ExecutionConfigOptions) WindowProperty(org.apache.flink.table.runtime.groupwindow.WindowProperty) Collections(java.util.Collections) NamedWindowProperty(org.apache.flink.table.runtime.groupwindow.NamedWindowProperty) RowtimeAttribute(org.apache.flink.table.runtime.groupwindow.RowtimeAttribute) PythonFunctionInfo(org.apache.flink.table.functions.python.PythonFunctionInfo) WindowProperty(org.apache.flink.table.runtime.groupwindow.WindowProperty) NamedWindowProperty(org.apache.flink.table.runtime.groupwindow.NamedWindowProperty) TableException(org.apache.flink.table.api.TableException) RowData(org.apache.flink.table.data.RowData) WindowStart(org.apache.flink.table.runtime.groupwindow.WindowStart) WindowEnd(org.apache.flink.table.runtime.groupwindow.WindowEnd)

Aggregations

Constructor (java.lang.reflect.Constructor)1 InvocationTargetException (java.lang.reflect.InvocationTargetException)1 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 AggregateCall (org.apache.calcite.rel.core.AggregateCall)1 Transformation (org.apache.flink.api.dag.Transformation)1 Tuple2 (org.apache.flink.api.java.tuple.Tuple2)1 Configuration (org.apache.flink.configuration.Configuration)1 ManagedMemoryUseCase (org.apache.flink.core.memory.ManagedMemoryUseCase)1 OneInputStreamOperator (org.apache.flink.streaming.api.operators.OneInputStreamOperator)1 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)1 TableException (org.apache.flink.table.api.TableException)1 ExecutionConfigOptions (org.apache.flink.table.api.config.ExecutionConfigOptions)1 Projection (org.apache.flink.table.connector.Projection)1 RowData (org.apache.flink.table.data.RowData)1 PythonFunctionInfo (org.apache.flink.table.functions.python.PythonFunctionInfo)1 CodeGeneratorContext (org.apache.flink.table.planner.codegen.CodeGeneratorContext)1 ProjectionCodeGenerator (org.apache.flink.table.planner.codegen.ProjectionCodeGenerator)1 WindowCodeGenerator (org.apache.flink.table.planner.codegen.agg.batch.WindowCodeGenerator)1 PlannerBase (org.apache.flink.table.planner.delegation.PlannerBase)1