Search in sources :

Example 1 with Dictionary

use of com.facebook.presto.parquet.dictionary.Dictionary in project presto by prestodb.

the class TupleDomainParquetPredicate method getDomain.

@VisibleForTesting
public static Domain getDomain(Type type, DictionaryDescriptor dictionaryDescriptor) {
    if (dictionaryDescriptor == null) {
        return Domain.all(type);
    }
    ColumnDescriptor columnDescriptor = dictionaryDescriptor.getColumnDescriptor();
    Optional<DictionaryPage> dictionaryPage = dictionaryDescriptor.getDictionaryPage();
    if (!dictionaryPage.isPresent()) {
        return Domain.all(type);
    }
    Dictionary dictionary;
    try {
        dictionary = dictionaryPage.get().getEncoding().initDictionary(columnDescriptor, dictionaryPage.get());
    } catch (Exception e) {
        // OK to ignore exception when reading dictionaries
        return Domain.all(type);
    }
    int dictionarySize = dictionaryPage.get().getDictionarySize();
    DictionaryValueConverter converter = new DictionaryValueConverter(dictionary);
    Function<Integer, Object> convertFunction = converter.getConverter(columnDescriptor.getPrimitiveType());
    List<Object> values = new ArrayList<>();
    for (int i = 0; i < dictionarySize; i++) {
        values.add(convertFunction.apply(i));
    }
    // TODO: when min == max (i.e., singleton ranges, the construction of Domains can be done more efficiently
    return getDomain(columnDescriptor, type, values, values, true);
}
Also used : Dictionary(com.facebook.presto.parquet.dictionary.Dictionary) RichColumnDescriptor(com.facebook.presto.parquet.RichColumnDescriptor) ColumnDescriptor(org.apache.parquet.column.ColumnDescriptor) ArrayList(java.util.ArrayList) DictionaryPage(com.facebook.presto.parquet.DictionaryPage) ParquetCorruptionException(com.facebook.presto.parquet.ParquetCorruptionException) VerifyException(com.google.common.base.VerifyException) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Aggregations

DictionaryPage (com.facebook.presto.parquet.DictionaryPage)1 ParquetCorruptionException (com.facebook.presto.parquet.ParquetCorruptionException)1 RichColumnDescriptor (com.facebook.presto.parquet.RichColumnDescriptor)1 Dictionary (com.facebook.presto.parquet.dictionary.Dictionary)1 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 VerifyException (com.google.common.base.VerifyException)1 ArrayList (java.util.ArrayList)1 ColumnDescriptor (org.apache.parquet.column.ColumnDescriptor)1