Search in sources :

Example 56 with ListObjectInspector

use of org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector in project metacat by Netflix.

the class HiveTypeConverter method getCanonicalType.

/**
     * Returns the canonical type.
     *
     * @param fieldInspector inspector
     * @return type
     */
Type getCanonicalType(final ObjectInspector fieldInspector) {
    switch(fieldInspector.getCategory()) {
        case PRIMITIVE:
            return getPrimitiveType(fieldInspector);
        case MAP:
            final MapObjectInspector mapObjectInspector = TypeUtils.checkType(fieldInspector, MapObjectInspector.class, "fieldInspector");
            final Type keyType = getCanonicalType(mapObjectInspector.getMapKeyObjectInspector());
            final Type valueType = getCanonicalType(mapObjectInspector.getMapValueObjectInspector());
            if (keyType == null || valueType == null) {
                return null;
            }
            return TypeRegistry.getTypeRegistry().getParameterizedType(TypeEnum.MAP, ImmutableList.of(keyType.getTypeSignature(), valueType.getTypeSignature()), ImmutableList.of());
        case LIST:
            final ListObjectInspector listObjectInspector = TypeUtils.checkType(fieldInspector, ListObjectInspector.class, "fieldInspector");
            final Type elementType = getCanonicalType(listObjectInspector.getListElementObjectInspector());
            if (elementType == null) {
                return null;
            }
            return TypeRegistry.getTypeRegistry().getParameterizedType(TypeEnum.ARRAY, ImmutableList.of(elementType.getTypeSignature()), ImmutableList.of());
        case STRUCT:
            final StructObjectInspector structObjectInspector = TypeUtils.checkType(fieldInspector, StructObjectInspector.class, "fieldInspector");
            final List<TypeSignature> fieldTypes = new ArrayList<>();
            final List<Object> fieldNames = new ArrayList<>();
            for (StructField field : structObjectInspector.getAllStructFieldRefs()) {
                fieldNames.add(field.getFieldName());
                final Type fieldType = getCanonicalType(field.getFieldObjectInspector());
                if (fieldType == null) {
                    return null;
                }
                fieldTypes.add(fieldType.getTypeSignature());
            }
            return TypeRegistry.getTypeRegistry().getParameterizedType(TypeEnum.ROW, fieldTypes, fieldNames);
        default:
            log.info("Currently unsupported type {}, returning Unknown type", fieldInspector.getTypeName());
            return BaseType.UNKNOWN;
    }
}
Also used : DecimalType(com.netflix.metacat.common.type.DecimalType) Type(com.netflix.metacat.common.type.Type) CharType(com.netflix.metacat.common.type.CharType) VarcharType(com.netflix.metacat.common.type.VarcharType) ParametricType(com.netflix.metacat.common.type.ParametricType) BaseType(com.netflix.metacat.common.type.BaseType) RowType(com.netflix.metacat.common.type.RowType) MapType(com.netflix.metacat.common.type.MapType) TypeSignature(com.netflix.metacat.common.type.TypeSignature) StructField(org.apache.hadoop.hive.serde2.objectinspector.StructField) MapObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector) ListObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector) ArrayList(java.util.ArrayList) StandardStructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)

Example 57 with ListObjectInspector

use of org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector in project asterixdb by apache.

the class HiveRecordParser method parseUnorderedList.

private void parseUnorderedList(AUnorderedListType uoltype, Object obj, ListObjectInspector oi) throws IOException {
    UnorderedListBuilder unorderedListBuilder = getUnorderedListBuilder();
    IAType itemType = null;
    if (uoltype != null)
        itemType = uoltype.getItemType();
    byte tagByte = itemType.getTypeTag().serialize();
    unorderedListBuilder.reset(uoltype);
    int n = oi.getListLength(obj);
    for (int i = 0; i < n; i++) {
        Object element = oi.getListElement(obj, i);
        ObjectInspector eoi = oi.getListElementObjectInspector();
        if (element == null) {
            throw new RuntimeDataException(ErrorCode.PARSER_HIVE_NULL_VALUE_IN_LIST);
        }
        listItemBuffer.reset();
        final DataOutput dataOutput = listItemBuffer.getDataOutput();
        dataOutput.writeByte(tagByte);
        parseItem(itemType, element, eoi, dataOutput, true);
        unorderedListBuilder.addItem(listItemBuffer);
    }
    unorderedListBuilder.write(fieldValueBuffer.getDataOutput(), true);
}
Also used : DataOutput(java.io.DataOutput) ListObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector) IntObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector) BooleanObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector) ShortObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.ShortObjectInspector) ObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector) LongObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector) StructObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector) FloatObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.FloatObjectInspector) StringObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspector) ByteObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.ByteObjectInspector) DoubleObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.DoubleObjectInspector) TimestampObjectInspector(org.apache.hadoop.hive.serde2.objectinspector.primitive.TimestampObjectInspector) UnorderedListBuilder(org.apache.asterix.builders.UnorderedListBuilder) IAType(org.apache.asterix.om.types.IAType) RuntimeDataException(org.apache.asterix.common.exceptions.RuntimeDataException)

Aggregations

ListObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector)48 ObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector)35 StructObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.StructObjectInspector)34 MapObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.MapObjectInspector)26 PrimitiveObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector)25 ArrayList (java.util.ArrayList)23 LongObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.LongObjectInspector)17 TimestampObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.TimestampObjectInspector)17 DoubleObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.DoubleObjectInspector)16 IntObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.IntObjectInspector)16 ShortObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.ShortObjectInspector)16 ByteObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.ByteObjectInspector)15 FloatObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.FloatObjectInspector)15 StringObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.StringObjectInspector)15 BooleanObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector)14 HiveDecimalObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.HiveDecimalObjectInspector)14 List (java.util.List)13 BinaryObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.BinaryObjectInspector)13 DateObjectInspector (org.apache.hadoop.hive.serde2.objectinspector.primitive.DateObjectInspector)13 Map (java.util.Map)11