Search in sources :

Example 1 with Struct

use of org.apache.arrow.vector.types.pojo.ArrowType.Struct in project parquet-mr by apache.

the class SchemaConverter method fromParquetGroup.

/**
 * @param type parquet types
 * @param name overrides parquet.getName()
 * @return the mapping
 */
private TypeMapping fromParquetGroup(GroupType type, String name) {
    LogicalTypeAnnotation logicalType = type.getLogicalTypeAnnotation();
    if (logicalType == null) {
        List<TypeMapping> typeMappings = fromParquet(type.getFields());
        Field arrowField = new Field(name, type.isRepetition(OPTIONAL), new Struct(), fields(typeMappings));
        return new StructTypeMapping(arrowField, type, typeMappings);
    } else {
        return logicalType.accept(new LogicalTypeAnnotation.LogicalTypeAnnotationVisitor<TypeMapping>() {

            @Override
            public Optional<TypeMapping> visit(LogicalTypeAnnotation.ListLogicalTypeAnnotation listLogicalType) {
                List3Levels list3Levels = new List3Levels(type);
                TypeMapping child = fromParquet(list3Levels.getElement(), null, list3Levels.getElement().getRepetition());
                Field arrowField = new Field(name, type.isRepetition(OPTIONAL), new ArrowType.List(), asList(child.getArrowField()));
                return of(new ListTypeMapping(arrowField, list3Levels, child));
            }
        }).orElseThrow(() -> new UnsupportedOperationException("Unsupported type " + type));
    }
}
Also used : ArrowType(org.apache.arrow.vector.types.pojo.ArrowType) StructTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping) Struct(org.apache.arrow.vector.types.pojo.ArrowType.Struct) Field(org.apache.arrow.vector.types.pojo.Field) LogicalTypeAnnotation(org.apache.parquet.schema.LogicalTypeAnnotation) ListTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping) PrimitiveTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.PrimitiveTypeMapping) RepeatedTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.RepeatedTypeMapping) UnionTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.UnionTypeMapping) ListTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping) StructTypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping) TypeMapping(org.apache.parquet.arrow.schema.SchemaMapping.TypeMapping)

Aggregations

ArrowType (org.apache.arrow.vector.types.pojo.ArrowType)1 Struct (org.apache.arrow.vector.types.pojo.ArrowType.Struct)1 Field (org.apache.arrow.vector.types.pojo.Field)1 ListTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.ListTypeMapping)1 PrimitiveTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.PrimitiveTypeMapping)1 RepeatedTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.RepeatedTypeMapping)1 StructTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.StructTypeMapping)1 TypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.TypeMapping)1 UnionTypeMapping (org.apache.parquet.arrow.schema.SchemaMapping.UnionTypeMapping)1 LogicalTypeAnnotation (org.apache.parquet.schema.LogicalTypeAnnotation)1