Search in sources :

Example 11 with FunctionResolution

use of com.facebook.presto.sql.relational.FunctionResolution in project presto by prestodb.

the class TestValuesNodeStats method testStatsForValuesNode.

@Test
public void testStatsForValuesNode() {
    FunctionResolution resolution = new FunctionResolution(tester().getMetadata().getFunctionAndTypeManager());
    tester().assertStatsFor(pb -> pb.values(ImmutableList.of(pb.variable("a", BIGINT), pb.variable("b", DOUBLE)), ImmutableList.of(ImmutableList.of(call(ADD.name(), resolution.arithmeticFunction(ADD, BIGINT, BIGINT), BIGINT, constantExpressions(BIGINT, 3L, 3L)), constant(13.5, DOUBLE)), ImmutableList.of(constant(55L, BIGINT), constantNull(DOUBLE)), ImmutableList.of(constant(6L, BIGINT), constant(13.5, DOUBLE))))).check(outputStats -> outputStats.equalTo(PlanNodeStatsEstimate.builder().setOutputRowCount(3).setConfident(true).addVariableStatistics(new VariableReferenceExpression(Optional.empty(), "a", BIGINT), VariableStatsEstimate.builder().setNullsFraction(0).setLowValue(6).setHighValue(55).setDistinctValuesCount(2).build()).addVariableStatistics(new VariableReferenceExpression(Optional.empty(), "b", DOUBLE), VariableStatsEstimate.builder().setNullsFraction(0.33333333333333333).setLowValue(13.5).setHighValue(13.5).setDistinctValuesCount(1).build()).build()));
    tester().assertStatsFor(pb -> pb.values(ImmutableList.of(pb.variable("v", createVarcharType(30))), ImmutableList.of(constantExpressions(VARCHAR, utf8Slice("Alice")), constantExpressions(VARCHAR, utf8Slice("has")), constantExpressions(VARCHAR, utf8Slice("a cat")), ImmutableList.of(constantNull(VARCHAR))))).check(outputStats -> outputStats.equalTo(PlanNodeStatsEstimate.builder().setOutputRowCount(4).setConfident(true).addVariableStatistics(new VariableReferenceExpression(Optional.empty(), "v", createVarcharType(30)), VariableStatsEstimate.builder().setNullsFraction(0.25).setDistinctValuesCount(3).build()).build()));
}
Also used : BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) DOUBLE(com.facebook.presto.common.type.DoubleType.DOUBLE) VARCHAR(com.facebook.presto.common.type.VarcharType.VARCHAR) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) VarcharType.createVarcharType(com.facebook.presto.common.type.VarcharType.createVarcharType) Expressions.call(com.facebook.presto.sql.relational.Expressions.call) Expressions.constantNull(com.facebook.presto.sql.relational.Expressions.constantNull) Expressions.constant(com.facebook.presto.sql.relational.Expressions.constant) PlanBuilder.constantExpressions(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.constantExpressions) UNKNOWN(com.facebook.presto.common.type.UnknownType.UNKNOWN) ImmutableList(com.google.common.collect.ImmutableList) ADD(com.facebook.presto.sql.tree.ArithmeticBinaryExpression.Operator.ADD) Optional(java.util.Optional) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) Test(org.testng.annotations.Test)

Example 12 with FunctionResolution

use of com.facebook.presto.sql.relational.FunctionResolution in project presto by prestodb.

the class TestFunctionAndTypeManager method testOperatorTypes.

@Test
public void testOperatorTypes() {
    FunctionAndTypeManager functionAndTypeManager = createTestFunctionAndTypeManager();
    FunctionResolution functionResolution = new FunctionResolution(functionAndTypeManager);
    assertTrue(functionAndTypeManager.getFunctionMetadata(functionResolution.arithmeticFunction(ADD, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isArithmeticOperator).orElse(false));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.arithmeticFunction(ADD, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isComparisonOperator).orElse(true));
    assertTrue(functionAndTypeManager.getFunctionMetadata(functionResolution.comparisonFunction(GREATER_THAN, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isComparisonOperator).orElse(false));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.comparisonFunction(GREATER_THAN, BIGINT, BIGINT)).getOperatorType().map(OperatorType::isArithmeticOperator).orElse(true));
    assertFalse(functionAndTypeManager.getFunctionMetadata(functionResolution.notFunction()).getOperatorType().isPresent());
}
Also used : FunctionAndTypeManager.createTestFunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager.createTestFunctionAndTypeManager) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) OperatorType.tryGetOperatorType(com.facebook.presto.common.function.OperatorType.tryGetOperatorType) OperatorType(com.facebook.presto.common.function.OperatorType) Test(org.testng.annotations.Test)

Example 13 with FunctionResolution

use of com.facebook.presto.sql.relational.FunctionResolution in project presto by prestodb.

the class DynamicFilterMatcher method match.

private MatchResult match(FilterNode filterNode, Session session, Metadata metadata, SymbolAliases symbolAliases) {
    checkState(this.filterNode == null, "filterNode must be null at this point");
    this.filterNode = filterNode;
    this.symbolAliases = symbolAliases;
    LogicalRowExpressions logicalRowExpressions = new LogicalRowExpressions(new RowExpressionDeterminismEvaluator(metadata.getFunctionAndTypeManager()), new FunctionResolution(metadata.getFunctionAndTypeManager()), metadata.getFunctionAndTypeManager());
    boolean staticFilterMatches = expectedStaticFilter.map(filter -> {
        RowExpressionVerifier verifier = new RowExpressionVerifier(symbolAliases, metadata, session);
        RowExpression staticFilter = logicalRowExpressions.combineConjuncts(extractDynamicFilters(filterNode.getPredicate()).getStaticConjuncts());
        return verifier.process(filter, staticFilter);
    }).orElse(true);
    return new MatchResult(match() && staticFilterMatches);
}
Also used : RowExpression(com.facebook.presto.spi.relation.RowExpression) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) Session(com.facebook.presto.Session) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) StatsProvider(com.facebook.presto.cost.StatsProvider) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) HashMap(java.util.HashMap) RowExpressionDeterminismEvaluator(com.facebook.presto.sql.relational.RowExpressionDeterminismEvaluator) Preconditions.checkState(com.google.common.base.Preconditions.checkState) FilterNode(com.facebook.presto.spi.plan.FilterNode) PlanNode(com.facebook.presto.spi.plan.PlanNode) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) Expression(com.facebook.presto.sql.tree.Expression) LogicalRowExpressions(com.facebook.presto.expressions.LogicalRowExpressions) Symbol(com.facebook.presto.sql.planner.Symbol) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) Optional(java.util.Optional) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) DynamicFilters(com.facebook.presto.expressions.DynamicFilters) Metadata(com.facebook.presto.metadata.Metadata) Joiner(com.google.common.base.Joiner) DynamicFilters.extractDynamicFilters(com.facebook.presto.expressions.DynamicFilters.extractDynamicFilters) MoreObjects.toStringHelper(com.google.common.base.MoreObjects.toStringHelper) RowExpressionDeterminismEvaluator(com.facebook.presto.sql.relational.RowExpressionDeterminismEvaluator) LogicalRowExpressions(com.facebook.presto.expressions.LogicalRowExpressions) RowExpression(com.facebook.presto.spi.relation.RowExpression) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution)

Example 14 with FunctionResolution

use of com.facebook.presto.sql.relational.FunctionResolution in project presto by prestodb.

the class StatisticsAggregationPlanner method createStatisticsAggregation.

public TableStatisticAggregation createStatisticsAggregation(TableStatisticsMetadata statisticsMetadata, Map<String, VariableReferenceExpression> columnToVariableMap, boolean useOriginalExpression) {
    StatisticAggregationsDescriptor.Builder<VariableReferenceExpression> descriptor = StatisticAggregationsDescriptor.builder();
    List<String> groupingColumns = statisticsMetadata.getGroupingColumns();
    List<VariableReferenceExpression> groupingVariables = groupingColumns.stream().map(columnToVariableMap::get).collect(toImmutableList());
    for (int i = 0; i < groupingVariables.size(); i++) {
        descriptor.addGrouping(groupingColumns.get(i), groupingVariables.get(i));
    }
    ImmutableMap.Builder<VariableReferenceExpression, AggregationNode.Aggregation> aggregations = ImmutableMap.builder();
    StandardFunctionResolution functionResolution = new FunctionResolution(metadata.getFunctionAndTypeManager());
    for (TableStatisticType type : statisticsMetadata.getTableStatistics()) {
        if (type != ROW_COUNT) {
            throw new PrestoException(NOT_SUPPORTED, "Table-wide statistic type not supported: " + type);
        }
        AggregationNode.Aggregation aggregation = new AggregationNode.Aggregation(new CallExpression("count", functionResolution.countFunction(), BIGINT, ImmutableList.of()), Optional.empty(), Optional.empty(), false, Optional.empty());
        VariableReferenceExpression variable = variableAllocator.newVariable("rowCount", BIGINT);
        aggregations.put(variable, aggregation);
        descriptor.addTableStatistic(ROW_COUNT, variable);
    }
    for (ColumnStatisticMetadata columnStatisticMetadata : statisticsMetadata.getColumnStatistics()) {
        String columnName = columnStatisticMetadata.getColumnName();
        ColumnStatisticType statisticType = columnStatisticMetadata.getStatisticType();
        VariableReferenceExpression inputVariable = columnToVariableMap.get(columnName);
        verify(inputVariable != null, "inputVariable is null");
        ColumnStatisticsAggregation aggregation = createColumnAggregation(statisticType, inputVariable, useOriginalExpression);
        VariableReferenceExpression variable = variableAllocator.newVariable(statisticType + ":" + columnName, aggregation.getOutputType());
        aggregations.put(variable, aggregation.getAggregation());
        descriptor.addColumnStatistic(columnStatisticMetadata, variable);
    }
    StatisticAggregations aggregation = new StatisticAggregations(aggregations.build(), groupingVariables);
    return new TableStatisticAggregation(aggregation, descriptor.build());
}
Also used : ColumnStatisticMetadata(com.facebook.presto.spi.statistics.ColumnStatisticMetadata) PrestoException(com.facebook.presto.spi.PrestoException) AggregationNode(com.facebook.presto.spi.plan.AggregationNode) StandardFunctionResolution(com.facebook.presto.spi.function.StandardFunctionResolution) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) ImmutableMap(com.google.common.collect.ImmutableMap) StatisticAggregations(com.facebook.presto.sql.planner.plan.StatisticAggregations) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) ColumnStatisticType(com.facebook.presto.spi.statistics.ColumnStatisticType) StatisticAggregationsDescriptor(com.facebook.presto.sql.planner.plan.StatisticAggregationsDescriptor) StandardFunctionResolution(com.facebook.presto.spi.function.StandardFunctionResolution) TableStatisticType(com.facebook.presto.spi.statistics.TableStatisticType) CallExpression(com.facebook.presto.spi.relation.CallExpression)

Example 15 with FunctionResolution

use of com.facebook.presto.sql.relational.FunctionResolution in project presto by prestodb.

the class TestReorderJoins method setUp.

@BeforeClass
public void setUp() {
    tester = new RuleTester(ImmutableList.of(), ImmutableMap.of(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name(), JOIN_REORDERING_STRATEGY, JoinReorderingStrategy.AUTOMATIC.name()), Optional.of(4));
    this.functionResolution = new FunctionResolution(tester.getMetadata().getFunctionAndTypeManager());
}
Also used : RuleTester(com.facebook.presto.sql.planner.iterative.rule.test.RuleTester) FunctionResolution(com.facebook.presto.sql.relational.FunctionResolution) BeforeClass(org.testng.annotations.BeforeClass)

Aggregations

FunctionResolution (com.facebook.presto.sql.relational.FunctionResolution)16 VariableReferenceExpression (com.facebook.presto.spi.relation.VariableReferenceExpression)8 RowExpressionDeterminismEvaluator (com.facebook.presto.sql.relational.RowExpressionDeterminismEvaluator)7 LogicalRowExpressions (com.facebook.presto.expressions.LogicalRowExpressions)5 ImmutableList (com.google.common.collect.ImmutableList)5 Optional (java.util.Optional)5 BeforeClass (org.testng.annotations.BeforeClass)5 Test (org.testng.annotations.Test)5 Session (com.facebook.presto.Session)4 TableHandle (com.facebook.presto.spi.TableHandle)4 RowExpression (com.facebook.presto.spi.relation.RowExpression)4 Expressions.call (com.facebook.presto.sql.relational.Expressions.call)4 ImmutableMap (com.google.common.collect.ImmutableMap)4 BIGINT (com.facebook.presto.common.type.BigintType.BIGINT)3 VarcharType.createVarcharType (com.facebook.presto.common.type.VarcharType.createVarcharType)3 Metadata (com.facebook.presto.metadata.Metadata)3 Expressions.constant (com.facebook.presto.sql.relational.Expressions.constant)3 RowExpressionDomainTranslator (com.facebook.presto.sql.relational.RowExpressionDomainTranslator)3 Objects.requireNonNull (java.util.Objects.requireNonNull)3 TupleDomain (com.facebook.presto.common.predicate.TupleDomain)2