Search in sources :

Example 6 with DataPageV1

use of org.apache.parquet.column.page.DataPageV1 in project parquet-mr by apache.

the class ColumnReaderImpl method readPage.

private void readPage() {
    LOG.debug("loading page");
    DataPage page = pageReader.readPage();
    page.accept(new DataPage.Visitor<Void>() {

        @Override
        public Void visit(DataPageV1 dataPageV1) {
            readPageV1(dataPageV1);
            return null;
        }

        @Override
        public Void visit(DataPageV2 dataPageV2) {
            readPageV2(dataPageV2);
            return null;
        }
    });
}
Also used : DataPage(org.apache.parquet.column.page.DataPage) DataPageV2(org.apache.parquet.column.page.DataPageV2) DataPageV1(org.apache.parquet.column.page.DataPageV1)

Example 7 with DataPageV1

use of org.apache.parquet.column.page.DataPageV1 in project drill by axbaretto.

the class ParquetRecordReaderTest method validateContains.

private void validateContains(MessageType schema, PageReadStore pages, String[] path, int values, BytesInput bytes) throws IOException {
    PageReader pageReader = pages.getPageReader(schema.getColumnDescription(path));
    DataPageV1 page = (DataPageV1) pageReader.readPage();
    assertEquals(values, page.getValueCount());
    assertArrayEquals(bytes.toByteArray(), page.getBytes().toByteArray());
}
Also used : PageReader(org.apache.parquet.column.page.PageReader) DataPageV1(org.apache.parquet.column.page.DataPageV1)

Example 8 with DataPageV1

use of org.apache.parquet.column.page.DataPageV1 in project hive by apache.

the class BaseVectorizedColumnReader method readPage.

protected void readPage() throws IOException {
    DataPage page = pageReader.readPage();
    if (page == null) {
        return;
    }
    page.accept(new DataPage.Visitor<Void>() {

        @Override
        public Void visit(DataPageV1 dataPageV1) {
            readPageV1(dataPageV1);
            return null;
        }

        @Override
        public Void visit(DataPageV2 dataPageV2) {
            readPageV2(dataPageV2);
            return null;
        }
    });
}
Also used : DataPage(org.apache.parquet.column.page.DataPage) DataPageV2(org.apache.parquet.column.page.DataPageV2) DataPageV1(org.apache.parquet.column.page.DataPageV1)

Example 9 with DataPageV1

use of org.apache.parquet.column.page.DataPageV1 in project drill by apache.

the class ParquetRecordReaderTest method validateContains.

private void validateContains(MessageType schema, PageReadStore pages, String[] path, int values, BytesInput bytes) throws IOException {
    PageReader pageReader = pages.getPageReader(schema.getColumnDescription(path));
    DataPageV1 page = (DataPageV1) pageReader.readPage();
    assertEquals(values, page.getValueCount());
    assertArrayEquals(bytes.toByteArray(), page.getBytes().toByteArray());
}
Also used : PageReader(org.apache.parquet.column.page.PageReader) DataPageV1(org.apache.parquet.column.page.DataPageV1)

Aggregations

DataPageV1 (org.apache.parquet.column.page.DataPageV1)9 DataPage (org.apache.parquet.column.page.DataPage)6 DataPageV2 (org.apache.parquet.column.page.DataPageV2)5 PageReader (org.apache.parquet.column.page.PageReader)4 ParquetDictionary (org.apache.flink.formats.parquet.vector.ParquetDictionary)1 WritableIntVector (org.apache.flink.table.data.columnar.vector.writable.WritableIntVector)1 DictionaryPage (org.apache.parquet.column.page.DictionaryPage)1 Statistics (org.apache.parquet.column.statistics.Statistics)1 ParquetEncodingException (org.apache.parquet.io.ParquetEncodingException)1