Examples with PrimitiveColumnIO parquet.io.PrimitiveColumnIO used on opensource projects

Example 1 with PrimitiveColumnIO

use of parquet.io.PrimitiveColumnIO in project presto by prestodb.

the class ParquetReader method initializeColumnReaders.

private void initializeColumnReaders() {
    for (PrimitiveColumnIO columnIO : getColumns(fileSchema, requestedSchema)) {
        ColumnDescriptor descriptor = columnIO.getColumnDescriptor();
        RichColumnDescriptor column = new RichColumnDescriptor(descriptor.getPath(), columnIO.getType().asPrimitiveType(), descriptor.getMaxRepetitionLevel(), descriptor.getMaxDefinitionLevel());
        columnReadersMap.put(column, ParquetColumnReader.createReader(column));
    }
}

Example 2 with PrimitiveColumnIO

use of parquet.io.PrimitiveColumnIO in project presto by prestodb.

the class ParquetReader method nextBatch.

public int nextBatch() {
    if (nextRowInGroup >= currentGroupRowCount && !advanceToNextRowGroup()) {
        return -1;
    }
    batchSize = toIntExact(min(MAX_VECTOR_LENGTH, currentGroupRowCount - nextRowInGroup));
    nextRowInGroup += batchSize;
    currentPosition += batchSize;
    for (PrimitiveColumnIO columnIO : getColumns(fileSchema, requestedSchema)) {
        ColumnDescriptor descriptor = columnIO.getColumnDescriptor();
        RichColumnDescriptor column = new RichColumnDescriptor(descriptor.getPath(), columnIO.getType().asPrimitiveType(), descriptor.getMaxRepetitionLevel(), descriptor.getMaxDefinitionLevel());
        ParquetColumnReader columnReader = columnReadersMap.get(column);
        columnReader.prepareNextRead(batchSize);
    }
    return batchSize;
}

Example 3 with PrimitiveColumnIO

use of parquet.io.PrimitiveColumnIO in project presto by prestodb.

the class ParquetTypeUtils method getDescriptor.

public static Optional<RichColumnDescriptor> getDescriptor(MessageType fileSchema, MessageType requestedSchema, List<String> path) {
    checkArgument(path.size() >= 1, "Parquet nested path should have at least one component");
    int level = path.size();
    for (PrimitiveColumnIO columnIO : getColumns(fileSchema, requestedSchema)) {
        ColumnIO[] fields = columnIO.getPath();
        if (fields.length <= level) {
            continue;
        }
        if (fields[level].getName().equalsIgnoreCase(path.get(level - 1))) {
            boolean match = true;
            for (int i = 0; i < level - 1; i++) {
                if (!fields[i + 1].getName().equalsIgnoreCase(path.get(i))) {
                    match = false;
                }
            }
            if (match) {
                ColumnDescriptor descriptor = columnIO.getColumnDescriptor();
                return Optional.of(new RichColumnDescriptor(descriptor.getPath(), columnIO.getType().asPrimitiveType(), descriptor.getMaxRepetitionLevel(), descriptor.getMaxDefinitionLevel()));
            }
        }
    }
    return empty();
}