Search in sources :

Example 11 with EquiJoinClause

use of io.prestosql.spi.plan.JoinNode.EquiJoinClause in project hetu-core by openlookeng.

the class TestReorderJoins method testReplicatedScalarJoinEvenWhereSessionRequiresRepartitioned.

@Test
public void testReplicatedScalarJoinEvenWhereSessionRequiresRepartitioned() {
    PlanMatchPattern expectedPlan = join(INNER, ImmutableList.of(equiJoinClause("A1", "B1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0)));
    PlanNodeStatsEstimate valuesA = PlanNodeStatsEstimate.builder().setOutputRowCount(10000).addSymbolStatistics(ImmutableMap.of(new Symbol("A1"), new SymbolStatsEstimate(0, 100, 0, 640000, 100))).build();
    PlanNodeStatsEstimate valuesB = PlanNodeStatsEstimate.builder().setOutputRowCount(10000).addSymbolStatistics(ImmutableMap.of(new Symbol("B1"), new SymbolStatsEstimate(0, 100, 0, 640000, 100))).build();
    assertReorderJoins().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.PARTITIONED.name()).on(p -> p.join(INNER, // matches isAtMostScalar
    p.values(new PlanNodeId("valuesA"), p.symbol("A1")), p.values(new PlanNodeId("valuesB"), ImmutableList.of(p.symbol("B1")), TWO_ROWS), ImmutableList.of(new EquiJoinClause(p.symbol("A1"), p.symbol("B1"))), ImmutableList.of(p.symbol("A1"), p.symbol("B1")), Optional.empty())).overrideStats("valuesA", valuesA).overrideStats("valuesB", valuesB).matches(expectedPlan);
    assertReorderJoins().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.PARTITIONED.name()).on(p -> p.join(INNER, p.values(new PlanNodeId("valuesB"), ImmutableList.of(p.symbol("B1")), TWO_ROWS), // matches isAtMostScalar
    p.values(new PlanNodeId("valuesA"), p.symbol("A1")), ImmutableList.of(new EquiJoinClause(p.symbol("A1"), p.symbol("B1"))), ImmutableList.of(p.symbol("A1"), p.symbol("B1")), Optional.empty())).overrideStats("valuesA", valuesA).overrideStats("valuesB", valuesB).matches(expectedPlan);
}
Also used : SymbolStatsEstimate(io.prestosql.cost.SymbolStatsEstimate) REPLICATED(io.prestosql.spi.plan.JoinNode.DistributionType.REPLICATED) EquiJoinClause(io.prestosql.spi.plan.JoinNode.EquiJoinClause) PlanMatchPattern.equiJoinClause(io.prestosql.sql.planner.assertions.PlanMatchPattern.equiJoinClause) QualifiedName(io.prestosql.sql.tree.QualifiedName) BROADCAST(io.prestosql.sql.analyzer.FeaturesConfig.JoinDistributionType.BROADCAST) JoinDistributionType(io.prestosql.sql.analyzer.FeaturesConfig.JoinDistributionType) Test(org.testng.annotations.Test) PARTITIONED(io.prestosql.spi.plan.JoinNode.DistributionType.PARTITIONED) LESS_THAN(io.prestosql.spi.function.OperatorType.LESS_THAN) JOIN_REORDERING_STRATEGY(io.prestosql.SystemSessionProperties.JOIN_REORDERING_STRATEGY) PlanMatchPattern.values(io.prestosql.sql.planner.assertions.PlanMatchPattern.values) ImmutableList(com.google.common.collect.ImmutableList) Expressions.call(io.prestosql.sql.relational.Expressions.call) OperatorType(io.prestosql.spi.function.OperatorType) JoinReorderingStrategy(io.prestosql.sql.analyzer.FeaturesConfig.JoinReorderingStrategy) BOOLEAN(io.prestosql.spi.type.BooleanType.BOOLEAN) Type(io.prestosql.spi.type.Type) RuleAssert(io.prestosql.sql.planner.iterative.rule.test.RuleAssert) BIGINT(io.prestosql.spi.type.BigintType.BIGINT) Symbol(io.prestosql.spi.plan.Symbol) PlanNodeId(io.prestosql.spi.plan.PlanNodeId) AfterClass(org.testng.annotations.AfterClass) EQUAL(io.prestosql.spi.function.OperatorType.EQUAL) PlanMatchPattern.join(io.prestosql.sql.planner.assertions.PlanMatchPattern.join) PlanNodeStatsEstimate(io.prestosql.cost.PlanNodeStatsEstimate) ImmutableMap(com.google.common.collect.ImmutableMap) PlanMatchPattern(io.prestosql.sql.planner.assertions.PlanMatchPattern) BeforeClass(org.testng.annotations.BeforeClass) FunctionAndTypeManager.qualifyObjectName(io.prestosql.metadata.FunctionAndTypeManager.qualifyObjectName) CostComparator(io.prestosql.cost.CostComparator) AUTOMATIC(io.prestosql.sql.analyzer.FeaturesConfig.JoinDistributionType.AUTOMATIC) Expressions.variable(io.prestosql.sql.relational.Expressions.variable) List(java.util.List) Closeables.closeAllRuntimeException(io.airlift.testing.Closeables.closeAllRuntimeException) FunctionResolution(io.prestosql.sql.relational.FunctionResolution) VarcharType.createUnboundedVarcharType(io.prestosql.spi.type.VarcharType.createUnboundedVarcharType) RowExpression(io.prestosql.spi.relation.RowExpression) INNER(io.prestosql.spi.plan.JoinNode.Type.INNER) Optional(java.util.Optional) JOIN_MAX_BROADCAST_TABLE_SIZE(io.prestosql.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) RuleTester(io.prestosql.sql.planner.iterative.rule.test.RuleTester) JOIN_DISTRIBUTION_TYPE(io.prestosql.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) PlanNodeId(io.prestosql.spi.plan.PlanNodeId) PlanNodeStatsEstimate(io.prestosql.cost.PlanNodeStatsEstimate) Symbol(io.prestosql.spi.plan.Symbol) EquiJoinClause(io.prestosql.spi.plan.JoinNode.EquiJoinClause) PlanMatchPattern(io.prestosql.sql.planner.assertions.PlanMatchPattern) SymbolStatsEstimate(io.prestosql.cost.SymbolStatsEstimate) Test(org.testng.annotations.Test)

Example 12 with EquiJoinClause

use of io.prestosql.spi.plan.JoinNode.EquiJoinClause in project hetu-core by openlookeng.

the class JoinStatsRule method computeInnerJoinStats.

private PlanNodeStatsEstimate computeInnerJoinStats(JoinNode node, PlanNodeStatsEstimate crossJoinStats, Session session, TypeProvider types) {
    List<EquiJoinClause> equiJoinCriteria = node.getCriteria();
    Map<Integer, Symbol> layout = new HashMap<>();
    int channel = 0;
    for (Symbol symbol : node.getOutputSymbols()) {
        layout.put(channel++, symbol);
    }
    if (equiJoinCriteria.isEmpty()) {
        if (!node.getFilter().isPresent()) {
            return crossJoinStats;
        }
        // TODO: this might explode stats
        if (isExpression(node.getFilter().get())) {
            return filterStatsCalculator.filterStats(crossJoinStats, castToExpression(node.getFilter().get()), session, types);
        } else {
            return filterStatsCalculator.filterStats(crossJoinStats, node.getFilter().get(), session, types, layout);
        }
    }
    PlanNodeStatsEstimate equiJoinEstimate = filterByEquiJoinClauses(crossJoinStats, node.getCriteria(), session, types);
    if (equiJoinEstimate.isOutputRowCountUnknown()) {
        return PlanNodeStatsEstimate.unknown();
    }
    if (!node.getFilter().isPresent()) {
        return equiJoinEstimate;
    }
    PlanNodeStatsEstimate filteredEquiJoinEstimate;
    if (isExpression(node.getFilter().get())) {
        filteredEquiJoinEstimate = filterStatsCalculator.filterStats(equiJoinEstimate, castToExpression(node.getFilter().get()), session, types);
    } else {
        filteredEquiJoinEstimate = filterStatsCalculator.filterStats(equiJoinEstimate, node.getFilter().get(), session, types, layout);
    }
    if (filteredEquiJoinEstimate.isOutputRowCountUnknown()) {
        return normalizer.normalize(equiJoinEstimate.mapOutputRowCount(rowCount -> rowCount * UNKNOWN_FILTER_COEFFICIENT), types);
    }
    return filteredEquiJoinEstimate;
}
Also used : EquiJoinClause(io.prestosql.spi.plan.JoinNode.EquiJoinClause) Lookup(io.prestosql.sql.planner.iterative.Lookup) Patterns.join(io.prestosql.sql.planner.plan.Patterns.join) LogicalRowExpressions(io.prestosql.expressions.LogicalRowExpressions) TypeProvider(io.prestosql.sql.planner.TypeProvider) HashMap(java.util.HashMap) Pattern(io.prestosql.matching.Pattern) SymbolStatsEstimate.buildFrom(io.prestosql.cost.SymbolStatsEstimate.buildFrom) OriginalExpressionUtils.isExpression(io.prestosql.sql.relational.OriginalExpressionUtils.isExpression) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) Sets.difference(com.google.common.collect.Sets.difference) NaN(java.lang.Double.NaN) Map(java.util.Map) Objects.requireNonNull(java.util.Objects.requireNonNull) Session(io.prestosql.Session) EQUAL(io.prestosql.sql.tree.ComparisonExpression.Operator.EQUAL) OriginalExpressionUtils.castToExpression(io.prestosql.sql.relational.OriginalExpressionUtils.castToExpression) LinkedList(java.util.LinkedList) JoinNode(io.prestosql.spi.plan.JoinNode) Symbol(io.prestosql.spi.plan.Symbol) UNKNOWN_FILTER_COEFFICIENT(io.prestosql.cost.FilterStatsCalculator.UNKNOWN_FILTER_COEFFICIENT) Collection(java.util.Collection) ImmutableList.toImmutableList(com.google.common.collect.ImmutableList.toImmutableList) ComparisonExpression(io.prestosql.sql.tree.ComparisonExpression) Math.min(java.lang.Math.min) Comparator.comparingDouble(java.util.Comparator.comparingDouble) MoreMath(io.prestosql.util.MoreMath) SymbolUtils.toSymbolReference(io.prestosql.sql.planner.SymbolUtils.toSymbolReference) List(java.util.List) Double.isNaN(java.lang.Double.isNaN) RowExpression(io.prestosql.spi.relation.RowExpression) Optional(java.util.Optional) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Queue(java.util.Queue) ExpressionUtils.extractConjuncts(io.prestosql.sql.ExpressionUtils.extractConjuncts) HashMap(java.util.HashMap) EquiJoinClause(io.prestosql.spi.plan.JoinNode.EquiJoinClause) Symbol(io.prestosql.spi.plan.Symbol)

Aggregations

EquiJoinClause (io.prestosql.spi.plan.JoinNode.EquiJoinClause)12 Symbol (io.prestosql.spi.plan.Symbol)10 Test (org.testng.annotations.Test)8 Optional (java.util.Optional)6 ImmutableList (com.google.common.collect.ImmutableList)5 ImmutableMap (com.google.common.collect.ImmutableMap)5 INNER (io.prestosql.spi.plan.JoinNode.Type.INNER)5 RowExpression (io.prestosql.spi.relation.RowExpression)5 Type (io.prestosql.spi.type.Type)5 SymbolStatsEstimate (io.prestosql.cost.SymbolStatsEstimate)4 OperatorType (io.prestosql.spi.function.OperatorType)4 PlanNodeId (io.prestosql.spi.plan.PlanNodeId)4 VarcharType.createUnboundedVarcharType (io.prestosql.spi.type.VarcharType.createUnboundedVarcharType)4 JoinDistributionType (io.prestosql.sql.analyzer.FeaturesConfig.JoinDistributionType)4 FunctionResolution (io.prestosql.sql.relational.FunctionResolution)4 Closeables.closeAllRuntimeException (io.airlift.testing.Closeables.closeAllRuntimeException)3 JOIN_DISTRIBUTION_TYPE (io.prestosql.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE)3 JOIN_MAX_BROADCAST_TABLE_SIZE (io.prestosql.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE)3 JOIN_REORDERING_STRATEGY (io.prestosql.SystemSessionProperties.JOIN_REORDERING_STRATEGY)3 CostComparator (io.prestosql.cost.CostComparator)3