Search in sources :

Example 1 with MAP

use of org.apache.flink.table.types.logical.LogicalTypeRoot.MAP in project flink by apache.

the class ValuesOperationFactory method findCommonTypeAtPosition.

private DataType findCommonTypeAtPosition(List<List<ResolvedExpression>> resolvedRows, int i) {
    List<LogicalType> typesAtIPosition = extractLogicalTypesAtPosition(resolvedRows, i);
    LogicalType logicalType = LogicalTypeMerging.findCommonType(typesAtIPosition).orElseThrow(() -> {
        Set<DataType> columnTypes = resolvedRows.stream().map(row -> row.get(i).getOutputDataType()).collect(Collectors.toCollection(LinkedHashSet::new));
        return new ValidationException(String.format("Types in fromValues(...) must have a common super type. Could not find a common type" + " for all rows at column %d.\n" + "Could not find a common super type for types: %s", i, columnTypes));
    });
    return TypeConversions.fromLogicalToDataType(logicalType);
}
Also used : IntStream(java.util.stream.IntStream) DataType(org.apache.flink.table.types.DataType) QueryOperation(org.apache.flink.table.operations.QueryOperation) ApiExpressionUtils.valueLiteral(org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral) KeyValueDataType(org.apache.flink.table.types.KeyValueDataType) CallExpression(org.apache.flink.table.expressions.CallExpression) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) LogicalTypeMerging(org.apache.flink.table.types.logical.utils.LogicalTypeMerging) Expression(org.apache.flink.table.expressions.Expression) ExpressionDefaultVisitor(org.apache.flink.table.expressions.ExpressionDefaultVisitor) ArrayList(java.util.ArrayList) NULL(org.apache.flink.table.types.logical.LogicalTypeRoot.NULL) ResolvedExpression(org.apache.flink.table.expressions.ResolvedExpression) FieldsDataType(org.apache.flink.table.types.FieldsDataType) ExpressionResolver(org.apache.flink.table.expressions.resolver.ExpressionResolver) MAP(org.apache.flink.table.types.logical.LogicalTypeRoot.MAP) LogicalTypeCasts.supportsExplicitCast(org.apache.flink.table.types.logical.utils.LogicalTypeCasts.supportsExplicitCast) ValuesQueryOperation(org.apache.flink.table.operations.ValuesQueryOperation) Nullable(javax.annotation.Nullable) LinkedHashSet(java.util.LinkedHashSet) ROW(org.apache.flink.table.types.logical.LogicalTypeRoot.ROW) FunctionDefinition(org.apache.flink.table.functions.FunctionDefinition) BuiltInFunctionDefinitions(org.apache.flink.table.functions.BuiltInFunctionDefinitions) TableException(org.apache.flink.table.api.TableException) Set(java.util.Set) ValueLiteralExpression(org.apache.flink.table.expressions.ValueLiteralExpression) Collectors(java.util.stream.Collectors) List(java.util.List) CollectionDataType(org.apache.flink.table.types.CollectionDataType) LogicalType(org.apache.flink.table.types.logical.LogicalType) ARRAY(org.apache.flink.table.types.logical.LogicalTypeRoot.ARRAY) ValidationException(org.apache.flink.table.api.ValidationException) Optional(java.util.Optional) Internal(org.apache.flink.annotation.Internal) TypeConversions(org.apache.flink.table.types.utils.TypeConversions) Collections(java.util.Collections) ValidationException(org.apache.flink.table.api.ValidationException) LogicalType(org.apache.flink.table.types.logical.LogicalType) DataType(org.apache.flink.table.types.DataType) KeyValueDataType(org.apache.flink.table.types.KeyValueDataType) FieldsDataType(org.apache.flink.table.types.FieldsDataType) CollectionDataType(org.apache.flink.table.types.CollectionDataType)

Example 2 with MAP

use of org.apache.flink.table.types.logical.LogicalTypeRoot.MAP in project flink by apache.

the class ValuesOperationFactory method create.

/**
 * Creates a valid {@link ValuesQueryOperation} operation.
 *
 * <p>It derives a row type based on {@link LogicalTypeMerging}. It flattens any row
 * constructors. It does not flatten ROWs which are a result of e.g. a function call.
 *
 * <p>The resulting schema can be provided manually. If it is not, the schema will be
 * automatically derived from the types of the expressions.
 */
QueryOperation create(@Nullable ResolvedSchema expectedSchema, List<ResolvedExpression> resolvedExpressions, ExpressionResolver.PostResolverFactory postResolverFactory) {
    List<List<ResolvedExpression>> resolvedRows = unwrapFromRowConstructor(resolvedExpressions);
    if (expectedSchema != null) {
        verifyAllSameSize(resolvedRows, expectedSchema.getColumnCount());
    }
    ResolvedSchema schema = Optional.ofNullable(expectedSchema).orElseGet(() -> extractSchema(resolvedRows));
    List<List<ResolvedExpression>> castedExpressions = resolvedRows.stream().map(row -> convertTopLevelExpressionToExpectedRowType(postResolverFactory, schema.getColumnDataTypes(), row)).collect(Collectors.toList());
    return new ValuesQueryOperation(castedExpressions, schema);
}
Also used : IntStream(java.util.stream.IntStream) DataType(org.apache.flink.table.types.DataType) QueryOperation(org.apache.flink.table.operations.QueryOperation) ApiExpressionUtils.valueLiteral(org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral) KeyValueDataType(org.apache.flink.table.types.KeyValueDataType) CallExpression(org.apache.flink.table.expressions.CallExpression) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema) LogicalTypeMerging(org.apache.flink.table.types.logical.utils.LogicalTypeMerging) Expression(org.apache.flink.table.expressions.Expression) ExpressionDefaultVisitor(org.apache.flink.table.expressions.ExpressionDefaultVisitor) ArrayList(java.util.ArrayList) NULL(org.apache.flink.table.types.logical.LogicalTypeRoot.NULL) ResolvedExpression(org.apache.flink.table.expressions.ResolvedExpression) FieldsDataType(org.apache.flink.table.types.FieldsDataType) ExpressionResolver(org.apache.flink.table.expressions.resolver.ExpressionResolver) MAP(org.apache.flink.table.types.logical.LogicalTypeRoot.MAP) LogicalTypeCasts.supportsExplicitCast(org.apache.flink.table.types.logical.utils.LogicalTypeCasts.supportsExplicitCast) ValuesQueryOperation(org.apache.flink.table.operations.ValuesQueryOperation) Nullable(javax.annotation.Nullable) LinkedHashSet(java.util.LinkedHashSet) ROW(org.apache.flink.table.types.logical.LogicalTypeRoot.ROW) FunctionDefinition(org.apache.flink.table.functions.FunctionDefinition) BuiltInFunctionDefinitions(org.apache.flink.table.functions.BuiltInFunctionDefinitions) TableException(org.apache.flink.table.api.TableException) Set(java.util.Set) ValueLiteralExpression(org.apache.flink.table.expressions.ValueLiteralExpression) Collectors(java.util.stream.Collectors) List(java.util.List) CollectionDataType(org.apache.flink.table.types.CollectionDataType) LogicalType(org.apache.flink.table.types.logical.LogicalType) ARRAY(org.apache.flink.table.types.logical.LogicalTypeRoot.ARRAY) ValidationException(org.apache.flink.table.api.ValidationException) Optional(java.util.Optional) Internal(org.apache.flink.annotation.Internal) TypeConversions(org.apache.flink.table.types.utils.TypeConversions) Collections(java.util.Collections) ValuesQueryOperation(org.apache.flink.table.operations.ValuesQueryOperation) ArrayList(java.util.ArrayList) List(java.util.List) ResolvedSchema(org.apache.flink.table.catalog.ResolvedSchema)

Example 3 with MAP

use of org.apache.flink.table.types.logical.LogicalTypeRoot.MAP in project flink by apache.

the class LogicalTypeMerging method findCommonType.

/**
 * Returns the most common, more general {@link LogicalType} for a given set of types. If such a
 * type exists, all given types can be casted to this more general type.
 *
 * <p>For example: {@code [INT, BIGINT, DECIMAL(2, 2)]} would lead to {@code DECIMAL(21, 2)}.
 *
 * <p>This class aims to be compatible with the SQL standard. It is inspired by Apache Calcite's
 * {@code SqlTypeFactoryImpl#leastRestrictive} method.
 */
public static Optional<LogicalType> findCommonType(List<LogicalType> types) {
    Preconditions.checkArgument(types.size() > 0, "List of types must not be empty.");
    // collect statistics first
    boolean hasRawType = false;
    boolean hasNullType = false;
    boolean hasNullableTypes = false;
    for (LogicalType type : types) {
        final LogicalTypeRoot typeRoot = type.getTypeRoot();
        if (typeRoot == RAW) {
            hasRawType = true;
        } else if (typeRoot == NULL) {
            hasNullType = true;
        }
        if (type.isNullable()) {
            hasNullableTypes = true;
        }
    }
    final List<LogicalType> normalizedTypes = types.stream().map(t -> t.copy(true)).collect(Collectors.toList());
    LogicalType foundType = findCommonNullableType(normalizedTypes, hasRawType, hasNullType);
    if (foundType == null) {
        foundType = findCommonCastableType(normalizedTypes);
    }
    if (foundType != null) {
        final LogicalType typeWithNullability = foundType.copy(hasNullableTypes);
        // NULL is reserved for untyped literals only
        if (typeWithNullability.is(NULL)) {
            return Optional.empty();
        }
        return Optional.of(typeWithNullability);
    }
    return Optional.empty();
}
Also used : Arrays(java.util.Arrays) INTERVAL(org.apache.flink.table.types.logical.LogicalTypeFamily.INTERVAL) AbstractList(java.util.AbstractList) MapType(org.apache.flink.table.types.logical.MapType) BINARY(org.apache.flink.table.types.logical.LogicalTypeRoot.BINARY) CharType(org.apache.flink.table.types.logical.CharType) MULTISET(org.apache.flink.table.types.logical.LogicalTypeRoot.MULTISET) NULL(org.apache.flink.table.types.logical.LogicalTypeRoot.NULL) DecimalType(org.apache.flink.table.types.logical.DecimalType) LogicalTypeChecks.getScale(org.apache.flink.table.types.logical.utils.LogicalTypeChecks.getScale) HOUR_TO_MINUTE(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.HOUR_TO_MINUTE) Map(java.util.Map) TimeType(org.apache.flink.table.types.logical.TimeType) DECIMAL(org.apache.flink.table.types.logical.LogicalTypeRoot.DECIMAL) DOUBLE(org.apache.flink.table.types.logical.LogicalTypeRoot.DOUBLE) LocalZonedTimestampType(org.apache.flink.table.types.logical.LocalZonedTimestampType) INTERVAL_YEAR_MONTH(org.apache.flink.table.types.logical.LogicalTypeRoot.INTERVAL_YEAR_MONTH) DayTimeIntervalType(org.apache.flink.table.types.logical.DayTimeIntervalType) TIMESTAMP_WITHOUT_TIME_ZONE(org.apache.flink.table.types.logical.LogicalTypeRoot.TIMESTAMP_WITHOUT_TIME_ZONE) SECOND(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.SECOND) Preconditions(org.apache.flink.util.Preconditions) Collectors(java.util.stream.Collectors) NullType(org.apache.flink.table.types.logical.NullType) HOUR_TO_SECOND(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.HOUR_TO_SECOND) EXACT_NUMERIC(org.apache.flink.table.types.logical.LogicalTypeFamily.EXACT_NUMERIC) LegacyTypeInformationType(org.apache.flink.table.types.logical.LegacyTypeInformationType) List(java.util.List) LogicalType(org.apache.flink.table.types.logical.LogicalType) VARCHAR(org.apache.flink.table.types.logical.LogicalTypeRoot.VARCHAR) ARRAY(org.apache.flink.table.types.logical.LogicalTypeRoot.ARRAY) Optional(java.util.Optional) MONTH(org.apache.flink.table.types.logical.YearMonthIntervalType.YearMonthResolution.MONTH) DAY(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.DAY) LogicalTypeChecks.getPrecision(org.apache.flink.table.types.logical.utils.LogicalTypeChecks.getPrecision) YEAR(org.apache.flink.table.types.logical.YearMonthIntervalType.YearMonthResolution.YEAR) TIMESTAMP_WITH_TIME_ZONE(org.apache.flink.table.types.logical.LogicalTypeRoot.TIMESTAMP_WITH_TIME_ZONE) LogicalTypeCasts.supportsImplicitCast(org.apache.flink.table.types.logical.utils.LogicalTypeCasts.supportsImplicitCast) IntStream(java.util.stream.IntStream) BINARY_STRING(org.apache.flink.table.types.logical.LogicalTypeFamily.BINARY_STRING) TIMESTAMP(org.apache.flink.table.types.logical.LogicalTypeFamily.TIMESTAMP) BinaryType(org.apache.flink.table.types.logical.BinaryType) APPROXIMATE_NUMERIC(org.apache.flink.table.types.logical.LogicalTypeFamily.APPROXIMATE_NUMERIC) VARBINARY(org.apache.flink.table.types.logical.LogicalTypeRoot.VARBINARY) CHAR(org.apache.flink.table.types.logical.LogicalTypeRoot.CHAR) MINUTE_TO_SECOND(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.MINUTE_TO_SECOND) LogicalTypeChecks.getLength(org.apache.flink.table.types.logical.utils.LogicalTypeChecks.getLength) HashMap(java.util.HashMap) MINUTE(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.MINUTE) RowType(org.apache.flink.table.types.logical.RowType) ArrayList(java.util.ArrayList) TimestampType(org.apache.flink.table.types.logical.TimestampType) DoubleType(org.apache.flink.table.types.logical.DoubleType) DAY_TO_SECOND(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.DAY_TO_SECOND) CHARACTER_STRING(org.apache.flink.table.types.logical.LogicalTypeFamily.CHARACTER_STRING) MAP(org.apache.flink.table.types.logical.LogicalTypeRoot.MAP) YearMonthIntervalType(org.apache.flink.table.types.logical.YearMonthIntervalType) YEAR_TO_MONTH(org.apache.flink.table.types.logical.YearMonthIntervalType.YearMonthResolution.YEAR_TO_MONTH) NUMERIC(org.apache.flink.table.types.logical.LogicalTypeFamily.NUMERIC) Nullable(javax.annotation.Nullable) ROW(org.apache.flink.table.types.logical.LogicalTypeRoot.ROW) MultisetType(org.apache.flink.table.types.logical.MultisetType) TIME(org.apache.flink.table.types.logical.LogicalTypeFamily.TIME) DateType(org.apache.flink.table.types.logical.DateType) VarCharType(org.apache.flink.table.types.logical.VarCharType) ArrayType(org.apache.flink.table.types.logical.ArrayType) DATE(org.apache.flink.table.types.logical.LogicalTypeRoot.DATE) RAW(org.apache.flink.table.types.logical.LogicalTypeRoot.RAW) DAY_TO_MINUTE(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.DAY_TO_MINUTE) DayTimeResolution(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution) YearMonthResolution(org.apache.flink.table.types.logical.YearMonthIntervalType.YearMonthResolution) DATETIME(org.apache.flink.table.types.logical.LogicalTypeFamily.DATETIME) TIMESTAMP_WITH_LOCAL_TIME_ZONE(org.apache.flink.table.types.logical.LogicalTypeRoot.TIMESTAMP_WITH_LOCAL_TIME_ZONE) VarBinaryType(org.apache.flink.table.types.logical.VarBinaryType) Internal(org.apache.flink.annotation.Internal) ZonedTimestampType(org.apache.flink.table.types.logical.ZonedTimestampType) HOUR(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.HOUR) DAY_TO_HOUR(org.apache.flink.table.types.logical.DayTimeIntervalType.DayTimeResolution.DAY_TO_HOUR) LogicalTypeRoot(org.apache.flink.table.types.logical.LogicalTypeRoot) Collections(java.util.Collections) INTERVAL_DAY_TIME(org.apache.flink.table.types.logical.LogicalTypeRoot.INTERVAL_DAY_TIME) LogicalType(org.apache.flink.table.types.logical.LogicalType) LogicalTypeRoot(org.apache.flink.table.types.logical.LogicalTypeRoot)

Aggregations

ArrayList (java.util.ArrayList)3 Collections (java.util.Collections)3 List (java.util.List)3 Optional (java.util.Optional)3 Collectors (java.util.stream.Collectors)3 IntStream (java.util.stream.IntStream)3 Nullable (javax.annotation.Nullable)3 Internal (org.apache.flink.annotation.Internal)3 LogicalType (org.apache.flink.table.types.logical.LogicalType)3 ARRAY (org.apache.flink.table.types.logical.LogicalTypeRoot.ARRAY)3 MAP (org.apache.flink.table.types.logical.LogicalTypeRoot.MAP)3 NULL (org.apache.flink.table.types.logical.LogicalTypeRoot.NULL)3 ROW (org.apache.flink.table.types.logical.LogicalTypeRoot.ROW)3 LinkedHashSet (java.util.LinkedHashSet)2 Set (java.util.Set)2 TableException (org.apache.flink.table.api.TableException)2 ValidationException (org.apache.flink.table.api.ValidationException)2 ResolvedSchema (org.apache.flink.table.catalog.ResolvedSchema)2 ApiExpressionUtils.valueLiteral (org.apache.flink.table.expressions.ApiExpressionUtils.valueLiteral)2 CallExpression (org.apache.flink.table.expressions.CallExpression)2