Search in sources :

Example 6 with NamedTypeSignature

use of com.facebook.presto.common.type.NamedTypeSignature in project presto by prestodb.

the class RowParametricType method createType.

@Override
public Type createType(List<TypeParameter> parameters) {
    checkArgument(!parameters.isEmpty(), "Row type must have at least one parameter");
    checkArgument(parameters.stream().allMatch(parameter -> parameter.getKind() == ParameterKind.NAMED_TYPE), "Expected only named types as a parameters, got %s", parameters);
    List<TypeSignatureParameter> typeSignatureParameters = parameters.stream().map(TypeParameter::getNamedType).map(parameter -> TypeSignatureParameter.of(new NamedTypeSignature(parameter.getName(), parameter.getType().getTypeSignature()))).collect(toList());
    List<RowType.Field> fields = parameters.stream().map(TypeParameter::getNamedType).map(parameter -> new RowType.Field(parameter.getName().map(RowFieldName::getName), parameter.getType())).collect(toList());
    return RowType.createWithTypeSignature(new TypeSignature(StandardTypes.ROW, typeSignatureParameters), fields);
}
Also used : StandardTypes(com.facebook.presto.common.type.StandardTypes) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) TypeSignature(com.facebook.presto.common.type.TypeSignature) ParametricType(com.facebook.presto.common.type.ParametricType) List(java.util.List) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Collectors.toList(java.util.stream.Collectors.toList) ParameterKind(com.facebook.presto.common.type.ParameterKind) RowFieldName(com.facebook.presto.common.type.RowFieldName) TypeParameter(com.facebook.presto.common.type.TypeParameter) Type(com.facebook.presto.common.type.Type) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) RowType(com.facebook.presto.common.type.RowType) TypeParameter(com.facebook.presto.common.type.TypeParameter) TypeSignature(com.facebook.presto.common.type.TypeSignature) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature)

Example 7 with NamedTypeSignature

use of com.facebook.presto.common.type.NamedTypeSignature in project presto by prestodb.

the class CreateTypeTask method execute.

@Override
public ListenableFuture<?> execute(CreateType statement, TransactionManager transactionManager, Metadata metadata, AccessControl accessControl, Session session, List<Expression> parameters, WarningCollector warningCollector) {
    TypeSignature signature;
    if (statement.getDistinctType().isPresent()) {
        signature = new TypeSignature(statement.getDistinctType().get());
    } else {
        List<TypeSignatureParameter> typeParameters = Streams.zip(statement.getParameterNames().stream(), statement.getParameterTypes().stream(), (name, type) -> TypeSignatureParameter.of(new NamedTypeSignature(Optional.of(new RowFieldName(name, false)), parseTypeSignature(type)))).collect(toImmutableList());
        signature = new TypeSignature(ROW, typeParameters);
    }
    UserDefinedType userDefinedType = new UserDefinedType(QualifiedObjectName.valueOf(statement.getTypeName().toString()), signature);
    metadata.getFunctionAndTypeManager().addUserDefinedType(userDefinedType);
    return immediateFuture(null);
}
Also used : WarningCollector(com.facebook.presto.spi.WarningCollector) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) ROW(com.facebook.presto.common.type.StandardTypes.ROW) TypeSignature(com.facebook.presto.common.type.TypeSignature) Inject(javax.inject.Inject) Objects.requireNonNull(java.util.Objects.requireNonNull) QualifiedObjectName(com.facebook.presto.common.QualifiedObjectName) UserDefinedType(com.facebook.presto.common.type.UserDefinedType) TransactionManager(com.facebook.presto.transaction.TransactionManager) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) Futures.immediateFuture(com.google.common.util.concurrent.Futures.immediateFuture) Session(com.facebook.presto.Session) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) CreateType(com.facebook.presto.sql.tree.CreateType) Streams(com.google.common.collect.Streams) String.format(java.lang.String.format) SqlParser(com.facebook.presto.sql.parser.SqlParser) List(java.util.List) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) Expression(com.facebook.presto.sql.tree.Expression) Optional(java.util.Optional) RowFieldName(com.facebook.presto.common.type.RowFieldName) Metadata(com.facebook.presto.metadata.Metadata) AccessControl(com.facebook.presto.security.AccessControl) TypeSignature(com.facebook.presto.common.type.TypeSignature) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) TypeSignature.parseTypeSignature(com.facebook.presto.common.type.TypeSignature.parseTypeSignature) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) RowFieldName(com.facebook.presto.common.type.RowFieldName) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) UserDefinedType(com.facebook.presto.common.type.UserDefinedType)

Example 8 with NamedTypeSignature

use of com.facebook.presto.common.type.NamedTypeSignature in project presto by prestodb.

the class ColumnIOConverter method constructField.

public static Optional<Field> constructField(Type type, ColumnIO columnIO) {
    if (columnIO == null) {
        return Optional.empty();
    }
    boolean required = columnIO.getType().getRepetition() != OPTIONAL;
    int repetitionLevel = columnRepetitionLevel(columnIO);
    int definitionLevel = columnDefinitionLevel(columnIO);
    if (ROW.equals(type.getTypeSignature().getBase())) {
        GroupColumnIO groupColumnIO = (GroupColumnIO) columnIO;
        List<Type> parameters = type.getTypeParameters();
        ImmutableList.Builder<Optional<Field>> fieldsBuilder = ImmutableList.builder();
        List<TypeSignatureParameter> fields = type.getTypeSignature().getParameters();
        boolean structHasParameters = false;
        for (int i = 0; i < fields.size(); i++) {
            NamedTypeSignature namedTypeSignature = fields.get(i).getNamedTypeSignature();
            String name = namedTypeSignature.getName().get().toLowerCase(Locale.ENGLISH);
            Optional<Field> field = constructField(parameters.get(i), lookupColumnByName(groupColumnIO, name));
            structHasParameters |= field.isPresent();
            fieldsBuilder.add(field);
        }
        if (structHasParameters) {
            return Optional.of(new GroupField(type, repetitionLevel, definitionLevel, required, fieldsBuilder.build()));
        }
        return Optional.empty();
    } else if (MAP.equals(type.getTypeSignature().getBase())) {
        GroupColumnIO groupColumnIO = (GroupColumnIO) columnIO;
        MapType mapType = (MapType) type;
        GroupColumnIO keyValueColumnIO = getMapKeyValueColumn(groupColumnIO);
        if (keyValueColumnIO.getChildrenCount() != 2) {
            return Optional.empty();
        }
        Optional<Field> keyField = constructField(mapType.getKeyType(), keyValueColumnIO.getChild(0));
        Optional<Field> valueField = constructField(mapType.getValueType(), keyValueColumnIO.getChild(1));
        return Optional.of(new GroupField(type, repetitionLevel, definitionLevel, required, ImmutableList.of(keyField, valueField)));
    } else if (ARRAY.equals(type.getTypeSignature().getBase())) {
        GroupColumnIO groupColumnIO = (GroupColumnIO) columnIO;
        List<Type> types = type.getTypeParameters();
        if (groupColumnIO.getChildrenCount() != 1) {
            return Optional.empty();
        }
        Optional<Field> field = constructField(types.get(0), getArrayElementColumn(groupColumnIO.getChild(0)));
        return Optional.of(new GroupField(type, repetitionLevel, definitionLevel, required, ImmutableList.of(field)));
    }
    PrimitiveColumnIO primitiveColumnIO = (PrimitiveColumnIO) columnIO;
    RichColumnDescriptor column = new RichColumnDescriptor(primitiveColumnIO.getColumnDescriptor(), columnIO.getType().asPrimitiveType());
    return Optional.of(new PrimitiveField(type, repetitionLevel, definitionLevel, required, column, primitiveColumnIO.getId()));
}
Also used : Optional(java.util.Optional) ImmutableList(com.google.common.collect.ImmutableList) GroupField(com.facebook.presto.parquet.GroupField) RichColumnDescriptor(com.facebook.presto.parquet.RichColumnDescriptor) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) MapType(com.facebook.presto.common.type.MapType) GroupField(com.facebook.presto.parquet.GroupField) PrimitiveField(com.facebook.presto.parquet.PrimitiveField) Field(com.facebook.presto.parquet.Field) MapType(com.facebook.presto.common.type.MapType) Type(com.facebook.presto.common.type.Type) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) PrimitiveField(com.facebook.presto.parquet.PrimitiveField)

Example 9 with NamedTypeSignature

use of com.facebook.presto.common.type.NamedTypeSignature in project presto by prestodb.

the class TypeConverter method toHiveTypeInfo.

private static TypeInfo toHiveTypeInfo(Type type) {
    if (BOOLEAN.equals(type)) {
        return HIVE_BOOLEAN.getTypeInfo();
    }
    if (BIGINT.equals(type)) {
        return HIVE_LONG.getTypeInfo();
    }
    if (INTEGER.equals(type)) {
        return HIVE_INT.getTypeInfo();
    }
    if (SMALLINT.equals(type)) {
        return HIVE_SHORT.getTypeInfo();
    }
    if (TINYINT.equals(type)) {
        return HIVE_BYTE.getTypeInfo();
    }
    if (REAL.equals(type)) {
        return HIVE_FLOAT.getTypeInfo();
    }
    if (DOUBLE.equals(type)) {
        return HIVE_DOUBLE.getTypeInfo();
    }
    if (type instanceof VarcharType) {
        VarcharType varcharType = (VarcharType) type;
        if (varcharType.isUnbounded()) {
            return HIVE_STRING.getTypeInfo();
        }
        if (varcharType.getLengthSafe() <= HiveVarchar.MAX_VARCHAR_LENGTH) {
            return getVarcharTypeInfo(varcharType.getLengthSafe());
        }
        throw new PrestoException(NOT_SUPPORTED, format("Unsupported Hive type: %s. Supported VARCHAR types: VARCHAR(<=%d), VARCHAR.", type, HiveVarchar.MAX_VARCHAR_LENGTH));
    }
    if (type instanceof CharType) {
        CharType charType = (CharType) type;
        int charLength = charType.getLength();
        if (charLength <= HiveChar.MAX_CHAR_LENGTH) {
            return getCharTypeInfo(charLength);
        }
        throw new PrestoException(NOT_SUPPORTED, format("Unsupported Hive type: %s. Supported CHAR types: CHAR(<=%d).", type, HiveChar.MAX_CHAR_LENGTH));
    }
    if (VARBINARY.equals(type)) {
        return HIVE_BINARY.getTypeInfo();
    }
    if (DATE.equals(type)) {
        return HIVE_DATE.getTypeInfo();
    }
    if (TIMESTAMP.equals(type)) {
        return HIVE_TIMESTAMP.getTypeInfo();
    }
    if (type instanceof DecimalType) {
        DecimalType decimalType = (DecimalType) type;
        return new DecimalTypeInfo(decimalType.getPrecision(), decimalType.getScale());
    }
    if (isArrayType(type)) {
        TypeInfo elementType = toHiveTypeInfo(type.getTypeParameters().get(0));
        return getListTypeInfo(elementType);
    }
    if (isMapType(type)) {
        TypeInfo keyType = toHiveTypeInfo(type.getTypeParameters().get(0));
        TypeInfo valueType = toHiveTypeInfo(type.getTypeParameters().get(1));
        return getMapTypeInfo(keyType, valueType);
    }
    if (isRowType(type)) {
        ImmutableList.Builder<String> fieldNames = ImmutableList.builder();
        for (TypeSignatureParameter parameter : type.getTypeSignature().getParameters()) {
            if (!parameter.isNamedTypeSignature()) {
                throw new IllegalArgumentException(format("Expected all parameters to be named type, but got %s", parameter));
            }
            NamedTypeSignature namedTypeSignature = parameter.getNamedTypeSignature();
            if (!namedTypeSignature.getName().isPresent()) {
                throw new PrestoException(NOT_SUPPORTED, format("Anonymous row type is not supported in Hive. Please give each field a name: %s", type));
            }
            fieldNames.add(namedTypeSignature.getName().get());
        }
        return getStructTypeInfo(fieldNames.build(), type.getTypeParameters().stream().map(TypeConverter::toHiveTypeInfo).collect(toList()));
    }
    throw new PrestoException(NOT_SUPPORTED, format("Unsupported Hive type: %s", type));
}
Also used : VarcharType(com.facebook.presto.common.type.VarcharType) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) ImmutableList(com.google.common.collect.ImmutableList) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) PrestoException(com.facebook.presto.spi.PrestoException) TypeInfoFactory.getMapTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getMapTypeInfo) TypeInfoFactory.getListTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getListTypeInfo) TypeInfoFactory.getCharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getCharTypeInfo) TypeInfoFactory.getStructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getStructTypeInfo) TypeInfoFactory.getVarcharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getVarcharTypeInfo) DecimalTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) DecimalTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) DecimalType(com.facebook.presto.common.type.DecimalType) CharType(com.facebook.presto.common.type.CharType)

Example 10 with NamedTypeSignature

use of com.facebook.presto.common.type.NamedTypeSignature in project presto by prestodb.

the class TestRowParametricType method testTypeSignatureRoundTrip.

@Test
public void testTypeSignatureRoundTrip() {
    FunctionAndTypeManager functionAndTypeManager = createTestFunctionAndTypeManager();
    TypeSignature typeSignature = new TypeSignature(ROW, TypeSignatureParameter.of(new NamedTypeSignature(Optional.of(new RowFieldName("col1", false)), new TypeSignature(BIGINT))), TypeSignatureParameter.of(new NamedTypeSignature(Optional.of(new RowFieldName("col2", true)), new TypeSignature(DOUBLE))));
    List<TypeParameter> parameters = typeSignature.getParameters().stream().map(parameter -> TypeParameter.of(parameter, functionAndTypeManager)).collect(Collectors.toList());
    Type rowType = RowParametricType.ROW.createType(parameters);
    assertEquals(rowType.getTypeSignature(), typeSignature);
}
Also used : FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) TypeSignatureParameter(com.facebook.presto.common.type.TypeSignatureParameter) Assert.assertEquals(com.facebook.presto.testing.assertions.Assert.assertEquals) Test(org.testng.annotations.Test) ROW(com.facebook.presto.common.type.StandardTypes.ROW) Collectors(java.util.stream.Collectors) FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) TypeSignature(com.facebook.presto.common.type.TypeSignature) DOUBLE(com.facebook.presto.common.type.StandardTypes.DOUBLE) List(java.util.List) Optional(java.util.Optional) RowFieldName(com.facebook.presto.common.type.RowFieldName) TypeParameter(com.facebook.presto.common.type.TypeParameter) BIGINT(com.facebook.presto.common.type.StandardTypes.BIGINT) Type(com.facebook.presto.common.type.Type) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) TypeSignature(com.facebook.presto.common.type.TypeSignature) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) TypeParameter(com.facebook.presto.common.type.TypeParameter) Type(com.facebook.presto.common.type.Type) FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) RowFieldName(com.facebook.presto.common.type.RowFieldName) NamedTypeSignature(com.facebook.presto.common.type.NamedTypeSignature) Test(org.testng.annotations.Test)

Aggregations

NamedTypeSignature (com.facebook.presto.common.type.NamedTypeSignature)14 TypeSignatureParameter (com.facebook.presto.common.type.TypeSignatureParameter)13 RowFieldName (com.facebook.presto.common.type.RowFieldName)10 TypeSignature (com.facebook.presto.common.type.TypeSignature)10 ImmutableList (com.google.common.collect.ImmutableList)9 Type (com.facebook.presto.common.type.Type)6 PrestoException (com.facebook.presto.spi.PrestoException)5 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)5 List (java.util.List)4 Optional (java.util.Optional)4 CharType (com.facebook.presto.common.type.CharType)3 DecimalType (com.facebook.presto.common.type.DecimalType)3 VarcharType (com.facebook.presto.common.type.VarcharType)3 DecimalTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo)3 TypeInfo (org.apache.hadoop.hive.serde2.typeinfo.TypeInfo)3 MapType (com.facebook.presto.common.type.MapType)2 RowType (com.facebook.presto.common.type.RowType)2 StandardTypes (com.facebook.presto.common.type.StandardTypes)2 ROW (com.facebook.presto.common.type.StandardTypes.ROW)2 TypeParameter (com.facebook.presto.common.type.TypeParameter)2