Search in sources :

Example 1 with DataPageHeader

use of org.apache.parquet.format.DataPageHeader in project parquet-mr by apache.

the class ParquetMetadataConverter method newDataPageHeader.

private PageHeader newDataPageHeader(int uncompressedSize, int compressedSize, int valueCount, org.apache.parquet.column.statistics.Statistics statistics, org.apache.parquet.column.Encoding rlEncoding, org.apache.parquet.column.Encoding dlEncoding, org.apache.parquet.column.Encoding valuesEncoding) {
    PageHeader pageHeader = new PageHeader(PageType.DATA_PAGE, uncompressedSize, compressedSize);
    // TODO: pageHeader.crc = ...;
    pageHeader.setData_page_header(new DataPageHeader(valueCount, getEncoding(valuesEncoding), getEncoding(dlEncoding), getEncoding(rlEncoding)));
    if (!statistics.isEmpty()) {
        pageHeader.getData_page_header().setStatistics(toParquetStatistics(statistics));
    }
    return pageHeader;
}
Also used : PageHeader(org.apache.parquet.format.PageHeader) Util.writePageHeader(org.apache.parquet.format.Util.writePageHeader) DictionaryPageHeader(org.apache.parquet.format.DictionaryPageHeader) DataPageHeader(org.apache.parquet.format.DataPageHeader) DataPageHeader(org.apache.parquet.format.DataPageHeader)

Aggregations

DataPageHeader (org.apache.parquet.format.DataPageHeader)1 DictionaryPageHeader (org.apache.parquet.format.DictionaryPageHeader)1 PageHeader (org.apache.parquet.format.PageHeader)1 Util.writePageHeader (org.apache.parquet.format.Util.writePageHeader)1