Search in sources :

Example 1 with StructTypeMapping

use of org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping in project parquet-mr by apache.

the class SchemaConverter method fromParquetGroup.

/**
 * @param type parquet types
 * @param name overrides parquet.getName()
 * @return the mapping
 */
private TypeMapping fromParquetGroup(GroupType type, String name) {
    OriginalType ot = type.getOriginalType();
    if (ot == null) {
        List<TypeMapping> typeMappings = fromParquet(type.getFields());
        Field arrowField = new Field(name, type.isRepetition(OPTIONAL), new Struct_(), fields(typeMappings));
        return new StructTypeMapping(arrowField, type, typeMappings);
    } else {
        switch(ot) {
            case LIST:
                List3Levels list3Levels = new List3Levels(type);
                TypeMapping child = fromParquet(list3Levels.getElement(), null, list3Levels.getElement().getRepetition());
                Field arrowField = new Field(name, type.isRepetition(OPTIONAL), new ArrowType.List(), asList(child.getArrowField()));
                return new ListTypeMapping(arrowField, list3Levels, child);
            default:
                throw new UnsupportedOperationException("Unsupported type " + type);
        }
    }
}
Also used : OriginalType(org.apache.parquet.schema.OriginalType) Field(org.apache.arrow.vector.types.pojo.Field) ListTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping) ArrowType(org.apache.arrow.vector.types.pojo.ArrowType) PrimitiveTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.PrimitiveTypeMapping) RepeatedTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.RepeatedTypeMapping) UnionTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.UnionTypeMapping) ListTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping) StructTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping) TypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.TypeMapping) StructTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping) Struct_(org.apache.arrow.vector.types.pojo.ArrowType.Struct_)

Aggregations

ArrowType (org.apache.arrow.vector.types.pojo.ArrowType)1 Struct_ (org.apache.arrow.vector.types.pojo.ArrowType.Struct_)1 Field (org.apache.arrow.vector.types.pojo.Field)1 ListTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping)1 PrimitiveTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.PrimitiveTypeMapping)1 RepeatedTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.RepeatedTypeMapping)1 StructTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping)1 TypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.TypeMapping)1 UnionTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.UnionTypeMapping)1 OriginalType (org.apache.parquet.schema.OriginalType)1