Search in sources :

Example 11 with DictionaryPage

use of com.facebook.presto.parquet.DictionaryPage in project presto by prestodb.

the class AbstractColumnReader method init.

@Override
public void init(PageReader pageReader, Field field, RowRanges rowRanges) {
    this.pageReader = requireNonNull(pageReader, "pageReader is null");
    this.field = requireNonNull(field, "field is null");
    DictionaryPage dictionaryPage = pageReader.readDictionaryPage();
    if (dictionaryPage != null) {
        try {
            dictionary = dictionaryPage.getEncoding().initDictionary(columnDescriptor, dictionaryPage);
        } catch (IOException e) {
            throw new ParquetDecodingException("could not decode the dictionary for " + columnDescriptor, e);
        }
    } else {
        dictionary = null;
    }
    checkArgument(pageReader.getTotalValueCount() > 0, "page is empty");
    totalValueCount = pageReader.getTotalValueCount();
    indexIterator = (rowRanges == null) ? null : rowRanges.iterator();
}
Also used : ParquetDecodingException(org.apache.parquet.io.ParquetDecodingException) IOException(java.io.IOException) DictionaryPage(com.facebook.presto.parquet.DictionaryPage)

Aggregations

DictionaryPage (com.facebook.presto.parquet.DictionaryPage)11 ArrayList (java.util.ArrayList)5 Random (java.util.Random)4 Test (org.testng.annotations.Test)4 IOException (java.io.IOException)3 ParquetCorruptionException (com.facebook.presto.parquet.ParquetCorruptionException)2 BinaryBatchDictionary (com.facebook.presto.parquet.batchreader.dictionary.BinaryBatchDictionary)2 TimestampDictionary (com.facebook.presto.parquet.batchreader.dictionary.TimestampDictionary)2 IntegerDictionary (com.facebook.presto.parquet.dictionary.IntegerDictionary)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 DictionaryPageHeader (org.apache.parquet.format.DictionaryPageHeader)2 PageHeader (org.apache.parquet.format.PageHeader)2 DataPage (com.facebook.presto.parquet.DataPage)1 ParquetEncoding (com.facebook.presto.parquet.ParquetEncoding)1 PLAIN_DICTIONARY (com.facebook.presto.parquet.ParquetEncoding.PLAIN_DICTIONARY)1 ParquetTypeUtils.getParquetEncoding (com.facebook.presto.parquet.ParquetTypeUtils.getParquetEncoding)1 RichColumnDescriptor (com.facebook.presto.parquet.RichColumnDescriptor)1 TestParquetUtils.generateDictionaryIdPage2048 (com.facebook.presto.parquet.batchreader.decoders.TestParquetUtils.generateDictionaryIdPage2048)1 TestParquetUtils.generatePlainValuesPage (com.facebook.presto.parquet.batchreader.decoders.TestParquetUtils.generatePlainValuesPage)1 BinaryValuesDecoder (com.facebook.presto.parquet.batchreader.decoders.ValuesDecoder.BinaryValuesDecoder)1