Search in sources :

Example 1 with DefinitionLevelIterable

use of com.facebook.presto.parquet.writer.levels.DefinitionLevelIterable in project presto by prestodb.

the class StructColumnWriter method writeBlock.

@Override
public void writeBlock(ColumnChunk columnChunk) throws IOException {
    ColumnarRow columnarRow = toColumnarRow(columnChunk.getBlock());
    checkArgument(columnarRow.getFieldCount() == columnWriters.size(), "ColumnarRow field size %s is not equal to columnWriters size %s", columnarRow.getFieldCount(), columnWriters.size());
    List<DefinitionLevelIterable> defLevelIterables = ImmutableList.<DefinitionLevelIterable>builder().addAll(columnChunk.getDefinitionLevelIterables()).add(DefinitionLevelIterables.of(columnarRow, maxDefinitionLevel)).build();
    List<RepetitionLevelIterable> repLevelIterables = ImmutableList.<RepetitionLevelIterable>builder().addAll(columnChunk.getRepetitionLevelIterables()).add(RepetitionLevelIterables.of(columnChunk.getBlock())).build();
    for (int i = 0; i < columnWriters.size(); ++i) {
        ColumnWriter columnWriter = columnWriters.get(i);
        Block block = columnarRow.getField(i);
        columnWriter.writeBlock(new ColumnChunk(block, defLevelIterables, repLevelIterables));
    }
}
Also used : ColumnarRow.toColumnarRow(com.facebook.presto.common.block.ColumnarRow.toColumnarRow) ColumnarRow(com.facebook.presto.common.block.ColumnarRow) RepetitionLevelIterable(com.facebook.presto.parquet.writer.levels.RepetitionLevelIterable) Block(com.facebook.presto.common.block.Block) DefinitionLevelIterable(com.facebook.presto.parquet.writer.levels.DefinitionLevelIterable)

Example 2 with DefinitionLevelIterable

use of com.facebook.presto.parquet.writer.levels.DefinitionLevelIterable in project presto by prestodb.

the class MapColumnWriter method writeBlock.

@Override
public void writeBlock(ColumnChunk columnChunk) throws IOException {
    ColumnarMap columnarMap = ColumnarMap.toColumnarMap(columnChunk.getBlock());
    ImmutableList<DefinitionLevelIterable> defLevelIterables = ImmutableList.<DefinitionLevelIterable>builder().addAll(columnChunk.getDefinitionLevelIterables()).add(DefinitionLevelIterables.of(columnarMap, maxDefinitionLevel)).build();
    ImmutableList<RepetitionLevelIterable> repLevelIterables = ImmutableList.<RepetitionLevelIterable>builder().addAll(columnChunk.getRepetitionLevelIterables()).add(RepetitionLevelIterables.of(columnarMap, maxRepetitionLevel)).build();
    keyWriter.writeBlock(new ColumnChunk(columnarMap.getKeysBlock(), defLevelIterables, repLevelIterables));
    valueWriter.writeBlock(new ColumnChunk(columnarMap.getValuesBlock(), defLevelIterables, repLevelIterables));
}
Also used : RepetitionLevelIterable(com.facebook.presto.parquet.writer.levels.RepetitionLevelIterable) ColumnarMap(com.facebook.presto.common.block.ColumnarMap) DefinitionLevelIterable(com.facebook.presto.parquet.writer.levels.DefinitionLevelIterable)

Aggregations

DefinitionLevelIterable (com.facebook.presto.parquet.writer.levels.DefinitionLevelIterable)2 RepetitionLevelIterable (com.facebook.presto.parquet.writer.levels.RepetitionLevelIterable)2 Block (com.facebook.presto.common.block.Block)1 ColumnarMap (com.facebook.presto.common.block.ColumnarMap)1 ColumnarRow (com.facebook.presto.common.block.ColumnarRow)1 ColumnarRow.toColumnarRow (com.facebook.presto.common.block.ColumnarRow.toColumnarRow)1