Search in sources :

Example 1 with ChunkReadSupport

use of water.parser.parquet.ChunkReadSupport in project h2o-3 by h2oai.

the class VecParquetReader method initReader.

private void initReader() throws IOException {
    assert reader == null;
    List<BlockMetaData> blocks = metadata.getBlocks();
    MessageType fileSchema = metadata.getFileMetaData().getSchema();
    reader = new InternalParquetRecordReader<>(new ChunkReadSupport(writer, chunkSchema));
    Configuration conf = VecFileSystem.makeConfiguration(vec);
    reader.initialize(fileSchema, metadata.getFileMetaData().getKeyValueMetaData(), VecFileSystem.VEC_PATH, blocks, conf);
}
Also used : BlockMetaData(org.apache.parquet.hadoop.metadata.BlockMetaData) Configuration(org.apache.hadoop.conf.Configuration) MessageType(org.apache.parquet.schema.MessageType) ChunkReadSupport(water.parser.parquet.ChunkReadSupport)

Aggregations

Configuration (org.apache.hadoop.conf.Configuration)1 BlockMetaData (org.apache.parquet.hadoop.metadata.BlockMetaData)1 MessageType (org.apache.parquet.schema.MessageType)1 ChunkReadSupport (water.parser.parquet.ChunkReadSupport)1