Search in sources :

Example 1 with Descriptor

use of io.trino.sql.DynamicFilters.Descriptor in project trino by trinodb.

the class LocalDynamicFiltersCollector method createDynamicFilter.

// Called during TableScan planning (no need to be synchronized as local planning is single threaded)
public DynamicFilter createDynamicFilter(List<Descriptor> descriptors, Map<Symbol, ColumnHandle> columnsMap, TypeProvider typeProvider, PlannerContext plannerContext) {
    Multimap<DynamicFilterId, Descriptor> descriptorMap = extractSourceSymbols(descriptors);
    // Iterate over dynamic filters that are collected (correspond to one of the futures), and required for filtering (correspond to one of the descriptors).
    // It is possible that some dynamic filters are collected in a different stage - and will not available here.
    // It is also possible that not all local dynamic filters are needed for this specific table scan.
    List<ListenableFuture<TupleDomain<ColumnHandle>>> predicateFutures = descriptorMap.keySet().stream().filter(futures.keySet()::contains).map(filterId -> {
        // Probe-side columns that can be filtered with this dynamic filter resulting domain.
        return Futures.transform(requireNonNull(futures.get(filterId), () -> format("Missing dynamic filter %s", filterId)), // Construct a probe-side predicate by duplicating the resulting domain over the corresponding columns.
        domain -> TupleDomain.withColumnDomains(descriptorMap.get(filterId).stream().collect(toImmutableMap(descriptor -> {
            Symbol probeSymbol = Symbol.from(descriptor.getInput());
            return requireNonNull(columnsMap.get(probeSymbol), () -> format("Missing probe column for %s", probeSymbol));
        }, descriptor -> {
            Type targetType = typeProvider.get(Symbol.from(descriptor.getInput()));
            Domain updatedDomain = descriptor.applyComparison(domain);
            if (!updatedDomain.getType().equals(targetType)) {
                return applySaturatedCasts(plannerContext.getMetadata(), plannerContext.getFunctionManager(), plannerContext.getTypeOperators(), session, updatedDomain, targetType);
            }
            return updatedDomain;
        }))), directExecutor());
    }).collect(toImmutableList());
    Set<ColumnHandle> columnsCovered = descriptorMap.values().stream().map(Descriptor::getInput).map(Symbol::from).map(probeSymbol -> requireNonNull(columnsMap.get(probeSymbol), () -> "Missing probe column for " + probeSymbol)).collect(toImmutableSet());
    return new TableSpecificDynamicFilter(columnsCovered, predicateFutures);
}
Also used : ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Type(io.trino.spi.type.Type) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) Multimap(com.google.common.collect.Multimap) SettableFuture(com.google.common.util.concurrent.SettableFuture) DynamicFilterId(io.trino.sql.planner.plan.DynamicFilterId) DynamicFilters.extractSourceSymbols(io.trino.sql.DynamicFilters.extractSourceSymbols) Verify.verify(com.google.common.base.Verify.verify) Descriptor(io.trino.sql.DynamicFilters.Descriptor) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) ColumnHandle(io.trino.spi.connector.ColumnHandle) ImmutableSet.toImmutableSet(com.google.common.collect.ImmutableSet.toImmutableSet) ImmutableSet(com.google.common.collect.ImmutableSet) Domain(io.trino.spi.predicate.Domain) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) Set(java.util.Set) ThreadSafe(javax.annotation.concurrent.ThreadSafe) TupleDomain(io.trino.spi.predicate.TupleDomain) GuardedBy(javax.annotation.concurrent.GuardedBy) String.format(java.lang.String.format) MoreExecutors.directExecutor(com.google.common.util.concurrent.MoreExecutors.directExecutor) Futures(com.google.common.util.concurrent.Futures) List(java.util.List) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) DynamicFilter(io.trino.spi.connector.DynamicFilter) DomainCoercer.applySaturatedCasts(io.trino.sql.planner.DomainCoercer.applySaturatedCasts) MoreFutures.addSuccessCallback(io.airlift.concurrent.MoreFutures.addSuccessCallback) MoreFutures.unmodifiableFuture(io.airlift.concurrent.MoreFutures.unmodifiableFuture) Session(io.trino.Session) PlannerContext(io.trino.sql.PlannerContext) ColumnHandle(io.trino.spi.connector.ColumnHandle) Type(io.trino.spi.type.Type) Descriptor(io.trino.sql.DynamicFilters.Descriptor) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) Domain(io.trino.spi.predicate.Domain) TupleDomain(io.trino.spi.predicate.TupleDomain) DynamicFilterId(io.trino.sql.planner.plan.DynamicFilterId)

Aggregations

Verify.verify (com.google.common.base.Verify.verify)1 ImmutableList.toImmutableList (com.google.common.collect.ImmutableList.toImmutableList)1 ImmutableMap.toImmutableMap (com.google.common.collect.ImmutableMap.toImmutableMap)1 ImmutableSet (com.google.common.collect.ImmutableSet)1 ImmutableSet.toImmutableSet (com.google.common.collect.ImmutableSet.toImmutableSet)1 Multimap (com.google.common.collect.Multimap)1 Futures (com.google.common.util.concurrent.Futures)1 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)1 MoreExecutors.directExecutor (com.google.common.util.concurrent.MoreExecutors.directExecutor)1 SettableFuture (com.google.common.util.concurrent.SettableFuture)1 MoreFutures.addSuccessCallback (io.airlift.concurrent.MoreFutures.addSuccessCallback)1 MoreFutures.unmodifiableFuture (io.airlift.concurrent.MoreFutures.unmodifiableFuture)1 Session (io.trino.Session)1 ColumnHandle (io.trino.spi.connector.ColumnHandle)1 DynamicFilter (io.trino.spi.connector.DynamicFilter)1 Domain (io.trino.spi.predicate.Domain)1 TupleDomain (io.trino.spi.predicate.TupleDomain)1 Type (io.trino.spi.type.Type)1 Descriptor (io.trino.sql.DynamicFilters.Descriptor)1 DynamicFilters.extractSourceSymbols (io.trino.sql.DynamicFilters.extractSourceSymbols)1