Search in sources :

Example 1 with ParquetEncoding

use of com.facebook.presto.parquet.ParquetEncoding in project presto by prestodb.

the class PredicateUtils method readDictionaryPage.

private static Optional<DictionaryPage> readDictionaryPage(byte[] data, CompressionCodecName codecName) {
    try {
        ByteArrayInputStream inputStream = new ByteArrayInputStream(data);
        PageHeader pageHeader = Util.readPageHeader(inputStream);
        if (pageHeader.type != PageType.DICTIONARY_PAGE) {
            return Optional.empty();
        }
        Slice compressedData = wrappedBuffer(data, data.length - inputStream.available(), pageHeader.getCompressed_page_size());
        DictionaryPageHeader dicHeader = pageHeader.getDictionary_page_header();
        ParquetEncoding encoding = getParquetEncoding(Encoding.valueOf(dicHeader.getEncoding().name()));
        int dictionarySize = dicHeader.getNum_values();
        return Optional.of(new DictionaryPage(decompress(codecName, compressedData, pageHeader.getUncompressed_page_size()), dictionarySize, encoding));
    } catch (IOException ignored) {
        return Optional.empty();
    }
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) DictionaryPageHeader(org.apache.parquet.format.DictionaryPageHeader) PageHeader(org.apache.parquet.format.PageHeader) Slice(io.airlift.slice.Slice) DictionaryPageHeader(org.apache.parquet.format.DictionaryPageHeader) IOException(java.io.IOException) DictionaryPage(com.facebook.presto.parquet.DictionaryPage) ParquetEncoding(com.facebook.presto.parquet.ParquetEncoding) ParquetTypeUtils.getParquetEncoding(com.facebook.presto.parquet.ParquetTypeUtils.getParquetEncoding)

Aggregations

DictionaryPage (com.facebook.presto.parquet.DictionaryPage)1 ParquetEncoding (com.facebook.presto.parquet.ParquetEncoding)1 ParquetTypeUtils.getParquetEncoding (com.facebook.presto.parquet.ParquetTypeUtils.getParquetEncoding)1 Slice (io.airlift.slice.Slice)1 ByteArrayInputStream (java.io.ByteArrayInputStream)1 IOException (java.io.IOException)1 DictionaryPageHeader (org.apache.parquet.format.DictionaryPageHeader)1 PageHeader (org.apache.parquet.format.PageHeader)1