Search in sources :

Example 1 with Dictionary

use of io.trino.parquet.dictionary.Dictionary in project trino by trinodb.

the class TupleDomainParquetPredicate method getDomain.

private static Domain getDomain(Type type, DictionaryDescriptor dictionaryDescriptor, DateTimeZone timeZone) {
    if (dictionaryDescriptor == null) {
        return Domain.all(type);
    }
    ColumnDescriptor columnDescriptor = dictionaryDescriptor.getColumnDescriptor();
    Optional<DictionaryPage> dictionaryPage = dictionaryDescriptor.getDictionaryPage();
    if (dictionaryPage.isEmpty()) {
        return Domain.all(type);
    }
    Dictionary dictionary;
    try {
        dictionary = dictionaryPage.get().getEncoding().initDictionary(columnDescriptor, dictionaryPage.get());
    } catch (Exception e) {
        // OK to ignore exception when reading dictionaries
        return Domain.all(type);
    }
    int dictionarySize = dictionaryPage.get().getDictionarySize();
    DictionaryValueConverter converter = new DictionaryValueConverter(dictionary);
    Function<Integer, Object> convertFunction = converter.getConverter(columnDescriptor.getPrimitiveType());
    List<Object> values = new ArrayList<>();
    for (int i = 0; i < dictionarySize; i++) {
        values.add(convertFunction.apply(i));
    }
    // TODO: when min == max (i.e., singleton ranges, the construction of Domains can be done more efficiently
    return getDomain(columnDescriptor, type, values, values, true, timeZone);
}
Also used : Dictionary(io.trino.parquet.dictionary.Dictionary) ColumnDescriptor(org.apache.parquet.column.ColumnDescriptor) RichColumnDescriptor(io.trino.parquet.RichColumnDescriptor) ArrayList(java.util.ArrayList) DictionaryPage(io.trino.parquet.DictionaryPage) ParquetCorruptionException(io.trino.parquet.ParquetCorruptionException) VerifyException(com.google.common.base.VerifyException)

Aggregations

VerifyException (com.google.common.base.VerifyException)1 DictionaryPage (io.trino.parquet.DictionaryPage)1 ParquetCorruptionException (io.trino.parquet.ParquetCorruptionException)1 RichColumnDescriptor (io.trino.parquet.RichColumnDescriptor)1 Dictionary (io.trino.parquet.dictionary.Dictionary)1 ArrayList (java.util.ArrayList)1 ColumnDescriptor (org.apache.parquet.column.ColumnDescriptor)1