Search in sources :

Example 36 with TypeInfoUtils.getTypeInfoFromTypeString

use of org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfoFromTypeString in project hive by apache.

the class VectorizedBatchUtil method convertToStandardStructObjectInspector.

public static StandardStructObjectInspector convertToStandardStructObjectInspector(StructObjectInspector structObjectInspector) throws HiveException {
    List<? extends StructField> fields = structObjectInspector.getAllStructFieldRefs();
    List<ObjectInspector> oids = new ArrayList<ObjectInspector>();
    ArrayList<String> columnNames = new ArrayList<String>();
    for (StructField field : fields) {
        TypeInfo typeInfo = TypeInfoUtils.getTypeInfoFromTypeString(field.getFieldObjectInspector().getTypeName());
        ObjectInspector standardWritableObjectInspector = TypeInfoUtils.getStandardWritableObjectInspectorFromTypeInfo(typeInfo);
        oids.add(standardWritableObjectInspector);
        columnNames.add(field.getFieldName());
    }
    return ObjectInspectorFactory.getStandardStructObjectInspector(columnNames, oids);
}
Also used : UnionObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.UnionObjectInspector) StandardStructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector) ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) MapObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector) ListObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector) PrimitiveObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector) StructField(org.apache.hadoop.hive.serde2.objectinspector.StructField) ArrayList(java.util.ArrayList) MapTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.MapTypeInfo) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) PrimitiveTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo) VarcharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.VarcharTypeInfo) ListTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo) DecimalTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) UnionTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.UnionTypeInfo) CharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.CharTypeInfo)

Example 37 with TypeInfoUtils.getTypeInfoFromTypeString

use of org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfoFromTypeString in project hive by apache.

the class VectorizedBatchUtil method createColumnVector.

public static ColumnVector createColumnVector(String typeName) {
    typeName = typeName.toLowerCase();
    // Allow undecorated CHAR and VARCHAR to support scratch column type names.
    if (typeName.equals("char") || typeName.equals("varchar")) {
        return new BytesColumnVector(VectorizedRowBatch.DEFAULT_SIZE);
    }
    TypeInfo typeInfo = (TypeInfo) TypeInfoUtils.getTypeInfoFromTypeString(typeName);
    return createColumnVector(typeInfo);
}
Also used : MapTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.MapTypeInfo) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) PrimitiveTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo) VarcharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.VarcharTypeInfo) ListTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo) DecimalTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) UnionTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.UnionTypeInfo) CharTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.CharTypeInfo)

Example 38 with TypeInfoUtils.getTypeInfoFromTypeString

use of org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfoFromTypeString in project druid by druid-io.

the class OrcHadoopInputRowParser method initialize.

private void initialize() {
    if (typeString == null) {
        typeString = typeStringFromParseSpec(parseSpec);
    }
    TypeInfo typeInfo = TypeInfoUtils.getTypeInfoFromTypeString(typeString);
    Preconditions.checkArgument(typeInfo instanceof StructTypeInfo, String.format("typeString should be struct type but not [%s]", typeString));
    Properties table = getTablePropertiesFromStructTypeInfo((StructTypeInfo) typeInfo);
    serde.initialize(new Configuration(), table);
    try {
        oip = (StructObjectInspector) serde.getObjectInspector();
    } catch (SerDeException e) {
        e.printStackTrace();
    }
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) Properties(java.util.Properties) StructTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo) TypeInfo(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo) SerDeException(org.apache.hadoop.hive.serde2.SerDeException)

Example 39 with TypeInfoUtils.getTypeInfoFromTypeString

use of org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfoFromTypeString in project hive by apache.

the class TypeInfoUtils method typeInfosFromStructObjectInspector.

public static ArrayList<TypeInfo> typeInfosFromStructObjectInspector(StructObjectInspector structObjectInspector) {
    List<? extends StructField> fields = structObjectInspector.getAllStructFieldRefs();
    ArrayList<TypeInfo> typeInfoList = new ArrayList<TypeInfo>(fields.size());
    for (StructField field : fields) {
        TypeInfo typeInfo = TypeInfoUtils.getTypeInfoFromTypeString(field.getFieldObjectInspector().getTypeName());
        typeInfoList.add(typeInfo);
    }
    return typeInfoList;
}
Also used : StructField(org.apache.hadoop.hive.serde2.objectinspector.StructField) ArrayList(java.util.ArrayList)

Example 40 with TypeInfoUtils.getTypeInfoFromTypeString

use of org.apache.hadoop.hive.serde2.typeinfo.TypeInfoUtils.getTypeInfoFromTypeString in project drill by apache.

the class HivePartitionDescriptor method getVectorType.

@Override
public TypeProtos.MajorType getVectorType(SchemaPath column, PlannerSettings plannerSettings) {
    HiveScan hiveScan = (HiveScan) scanRel.getGroupScan();
    String partitionName = column.getAsNamePart().getName();
    Map<String, String> partitionNameTypeMap = hiveScan.hiveReadEntry.table.getPartitionNameTypeMap();
    String hiveType = partitionNameTypeMap.get(partitionName);
    PrimitiveTypeInfo primitiveTypeInfo = (PrimitiveTypeInfo) TypeInfoUtils.getTypeInfoFromTypeString(hiveType);
    TypeProtos.MinorType partitionType = HiveUtilities.getMinorTypeFromHivePrimitiveTypeInfo(primitiveTypeInfo, plannerSettings.getOptions());
    return TypeProtos.MajorType.newBuilder().setMode(TypeProtos.DataMode.OPTIONAL).setMinorType(partitionType).build();
}
Also used : HiveScan(org.apache.drill.exec.store.hive.HiveScan) TypeProtos(org.apache.drill.common.types.TypeProtos) PrimitiveTypeInfo(org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo)

Aggregations

TypeInfo (org.apache.hadoop.hive.serde2.typeinfo.TypeInfo)51 PrimitiveTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo)27 ArrayList (java.util.ArrayList)24 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)23 StructTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo)20 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)16 DecimalTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.DecimalTypeInfo)15 CharTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.CharTypeInfo)12 VarcharTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.VarcharTypeInfo)12 ListTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo)11 MapTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.MapTypeInfo)11 UnionTypeInfo (org.apache.hadoop.hive.serde2.typeinfo.UnionTypeInfo)11 ListObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector)10 MapObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector)10 FieldSchema (org.apache.hadoop.hive.metastore.api.FieldSchema)8 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)8 HashMap (java.util.HashMap)7 ExprNodeDesc (org.apache.hadoop.hive.ql.plan.ExprNodeDesc)7 PrimitiveCategory (org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector.PrimitiveCategory)7 StructField (org.apache.hadoop.hive.serde2.objectinspector.StructField)7