Search in sources :

Example 1 with DataPageHeaderV2

use of org.apache.parquet.format.DataPageHeaderV2 in project parquet-mr by apache.

the class ParquetMetadataConverter method newDataPageV2Header.

private PageHeader newDataPageV2Header(int uncompressedSize, int compressedSize, int valueCount, int nullCount, int rowCount, org.apache.parquet.column.statistics.Statistics<?> statistics, org.apache.parquet.column.Encoding dataEncoding, int rlByteLength, int dlByteLength) {
    // TODO: pageHeader.crc = ...;
    DataPageHeaderV2 dataPageHeaderV2 = new DataPageHeaderV2(valueCount, nullCount, rowCount, getEncoding(dataEncoding), dlByteLength, rlByteLength);
    if (!statistics.isEmpty()) {
        dataPageHeaderV2.setStatistics(toParquetStatistics(statistics));
    }
    PageHeader pageHeader = new PageHeader(PageType.DATA_PAGE_V2, uncompressedSize, compressedSize);
    pageHeader.setData_page_header_v2(dataPageHeaderV2);
    return pageHeader;
}
Also used : PageHeader(org.apache.parquet.format.PageHeader) Util.writePageHeader(org.apache.parquet.format.Util.writePageHeader) DictionaryPageHeader(org.apache.parquet.format.DictionaryPageHeader) DataPageHeader(org.apache.parquet.format.DataPageHeader) DataPageHeaderV2(org.apache.parquet.format.DataPageHeaderV2)

Aggregations

DataPageHeader (org.apache.parquet.format.DataPageHeader)1 DataPageHeaderV2 (org.apache.parquet.format.DataPageHeaderV2)1 DictionaryPageHeader (org.apache.parquet.format.DictionaryPageHeader)1 PageHeader (org.apache.parquet.format.PageHeader)1 Util.writePageHeader (org.apache.parquet.format.Util.writePageHeader)1