Search in sources :

Example 6 with DataViewSpec

use of org.apache.flink.table.runtime.dataview.DataViewSpec in project flink by apache.

the class CommonPythonUtil method extractDataViewSpecs.

public static DataViewSpec[] extractDataViewSpecs(int index, DataType accType) {
    if (!(accType instanceof FieldsDataType)) {
        return new DataViewSpec[0];
    }
    FieldsDataType compositeAccType = (FieldsDataType) accType;
    if (includesDataView(compositeAccType)) {
        LogicalType logicalType = compositeAccType.getLogicalType();
        if (logicalType instanceof RowType) {
            List<DataType> childrenDataTypes = compositeAccType.getChildren();
            return IntStream.range(0, childrenDataTypes.size()).mapToObj(i -> {
                DataType childDataType = childrenDataTypes.get(i);
                LogicalType childLogicalType = childDataType.getLogicalType();
                if ((childLogicalType instanceof RowType) && includesDataView((FieldsDataType) childDataType)) {
                    throw new TableException("For Python AggregateFunction, DataView cannot be used in the" + " nested columns of the accumulator. ");
                } else if ((childLogicalType instanceof StructuredType) && ListView.class.isAssignableFrom(((StructuredType) childLogicalType).getImplementationClass().get())) {
                    return new ListViewSpec("agg" + index + "$" + ((RowType) logicalType).getFieldNames().get(i), i, childDataType.getChildren().get(0));
                } else if ((childLogicalType instanceof StructuredType) && MapView.class.isAssignableFrom(((StructuredType) childLogicalType).getImplementationClass().get())) {
                    return new MapViewSpec("agg" + index + "$" + ((RowType) logicalType).getFieldNames().get(i), i, childDataType.getChildren().get(0), false);
                }
                return null;
            }).filter(Objects::nonNull).toArray(DataViewSpec[]::new);
        } else {
            throw new TableException("For Python AggregateFunction you can only use DataView in " + "Row type.");
        }
    } else {
        return new DataViewSpec[0];
    }
}
Also used : FieldsDataType(org.apache.flink.table.types.FieldsDataType) TableException(org.apache.flink.table.api.TableException) MapViewSpec(org.apache.flink.table.runtime.dataview.MapViewSpec) DataViewSpec(org.apache.flink.table.runtime.dataview.DataViewSpec) ListViewSpec(org.apache.flink.table.runtime.dataview.ListViewSpec) LogicalType(org.apache.flink.table.types.logical.LogicalType) RowType(org.apache.flink.table.types.logical.RowType) DataType(org.apache.flink.table.types.DataType) FieldsDataType(org.apache.flink.table.types.FieldsDataType) MapView(org.apache.flink.table.api.dataview.MapView) StructuredType(org.apache.flink.table.types.logical.StructuredType)

Aggregations

DataViewSpec (org.apache.flink.table.runtime.dataview.DataViewSpec)6 PythonAggregateFunctionInfo (org.apache.flink.table.functions.python.PythonAggregateFunctionInfo)4 RowType (org.apache.flink.table.types.logical.RowType)4 Configuration (org.apache.flink.configuration.Configuration)3 TableException (org.apache.flink.table.api.TableException)3 RowData (org.apache.flink.table.data.RowData)3 ArrayList (java.util.ArrayList)2 Transformation (org.apache.flink.api.dag.Transformation)2 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)2 MapView (org.apache.flink.table.api.dataview.MapView)2 ExecEdge (org.apache.flink.table.planner.plan.nodes.exec.ExecEdge)2 AggregateInfoList (org.apache.flink.table.planner.plan.utils.AggregateInfoList)2 ListViewSpec (org.apache.flink.table.runtime.dataview.ListViewSpec)2 MapViewSpec (org.apache.flink.table.runtime.dataview.MapViewSpec)2 RowDataKeySelector (org.apache.flink.table.runtime.keyselector.RowDataKeySelector)2 DataType (org.apache.flink.table.types.DataType)2 LogicalType (org.apache.flink.table.types.logical.LogicalType)2 StructuredType (org.apache.flink.table.types.logical.StructuredType)2 InvocationTargetException (java.lang.reflect.InvocationTargetException)1 Method (java.lang.reflect.Method)1