Search in sources :

Example 81 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestDetermineJoinDistributionType method testFlipAndReplicateWhenOneTableMuchSmallerAndJoinCardinalityUnknown.

@Test
public void testFlipAndReplicateWhenOneTableMuchSmallerAndJoinCardinalityUnknown() {
    int aRows = 100;
    int bRows = 10_000;
    assertDetermineJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "A1", BIGINT), VariableStatsEstimate.unknown())).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "B1", BIGINT), VariableStatsEstimate.unknown())).build()).on(p -> p.join(INNER, p.values(new PlanNodeId("valuesA"), aRows, p.variable("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.variable("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.variable("A1", BIGINT), p.variable("B1", BIGINT))), ImmutableList.of(p.variable("A1", BIGINT), p.variable("B1", BIGINT)), Optional.empty())).matches(join(INNER, ImmutableList.of(equiJoinClause("B1", "A1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("B1", 0)), values(ImmutableMap.of("A1", 0))));
}
Also used : Type(com.facebook.presto.sql.planner.plan.JoinNode.Type) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PlanBuilder.castToRowExpression(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.castToRowExpression) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) FULL(com.facebook.presto.sql.planner.plan.JoinNode.Type.FULL) PlanMatchPattern.join(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.join) JoinDistributionType(com.facebook.presto.sql.analyzer.FeaturesConfig.JoinDistributionType) ImmutableList(com.google.common.collect.ImmutableList) JOIN_MAX_BROADCAST_TABLE_SIZE(com.facebook.presto.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) PlanMatchPattern.equiJoinClause(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.equiJoinClause) PlanMatchPattern.enforceSingleRow(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.enforceSingleRow) RuleAssert(com.facebook.presto.sql.planner.iterative.rule.test.RuleAssert) TaskCountEstimator(com.facebook.presto.cost.TaskCountEstimator) DistributionType(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) AfterClass(org.testng.annotations.AfterClass) BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) ImmutableMap(com.google.common.collect.ImmutableMap) BeforeClass(org.testng.annotations.BeforeClass) JOIN_DISTRIBUTION_TYPE(com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) RIGHT(com.facebook.presto.sql.planner.plan.JoinNode.Type.RIGHT) PlanBuilder.constantExpressions(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.constantExpressions) REPLICATED(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType.REPLICATED) LEFT(com.facebook.presto.sql.planner.plan.JoinNode.Type.LEFT) CostComparator(com.facebook.presto.cost.CostComparator) Optional(java.util.Optional) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) PARTITIONED(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType.PARTITIONED) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) RuleTester(com.facebook.presto.sql.planner.iterative.rule.test.RuleTester) INNER(com.facebook.presto.sql.planner.plan.JoinNode.Type.INNER) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test)

Example 82 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestDetermineJoinDistributionType method testReplicatesWhenRequiredBySession.

@Test
public void testReplicatesWhenRequiredBySession() {
    int aRows = 10_000;
    int bRows = 10_000;
    assertDetermineJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "A1", BIGINT), new VariableStatsEstimate(0, 100, 0, 640000, 100))).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "B1", BIGINT), new VariableStatsEstimate(0, 100, 0, 640000, 100))).build()).on(p -> p.join(INNER, p.values(new PlanNodeId("valuesA"), aRows, p.variable("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.variable("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.variable("A1", BIGINT), p.variable("B1", BIGINT))), ImmutableList.of(p.variable("A1", BIGINT), p.variable("B1", BIGINT)), Optional.empty())).setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.BROADCAST.name()).matches(join(INNER, ImmutableList.of(equiJoinClause("A1", "B1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) Test(org.testng.annotations.Test)

Example 83 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestDetermineJoinDistributionType method testPartitionFullOuterJoin.

@Test
public void testPartitionFullOuterJoin() {
    int aRows = 10_000;
    int bRows = 10;
    assertDetermineJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "A1", BIGINT), new VariableStatsEstimate(0, 100, 0, 640000, 100))).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "B1", BIGINT), new VariableStatsEstimate(0, 100, 0, 640000, 100))).build()).on(p -> p.join(FULL, p.values(new PlanNodeId("valuesA"), aRows, p.variable("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.variable("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.variable("A1", BIGINT), p.variable("B1", BIGINT))), ImmutableList.of(p.variable("A1", BIGINT), p.variable("B1", BIGINT)), Optional.empty())).matches(join(FULL, ImmutableList.of(equiJoinClause("A1", "B1")), Optional.empty(), Optional.of(PARTITIONED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : Type(com.facebook.presto.sql.planner.plan.JoinNode.Type) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PlanBuilder.castToRowExpression(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.castToRowExpression) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) FULL(com.facebook.presto.sql.planner.plan.JoinNode.Type.FULL) PlanMatchPattern.join(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.join) JoinDistributionType(com.facebook.presto.sql.analyzer.FeaturesConfig.JoinDistributionType) ImmutableList(com.google.common.collect.ImmutableList) JOIN_MAX_BROADCAST_TABLE_SIZE(com.facebook.presto.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) PlanMatchPattern.equiJoinClause(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.equiJoinClause) PlanMatchPattern.enforceSingleRow(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.enforceSingleRow) RuleAssert(com.facebook.presto.sql.planner.iterative.rule.test.RuleAssert) TaskCountEstimator(com.facebook.presto.cost.TaskCountEstimator) DistributionType(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType) JoinNode(com.facebook.presto.sql.planner.plan.JoinNode) AfterClass(org.testng.annotations.AfterClass) BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) ImmutableMap(com.google.common.collect.ImmutableMap) BeforeClass(org.testng.annotations.BeforeClass) JOIN_DISTRIBUTION_TYPE(com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) RIGHT(com.facebook.presto.sql.planner.plan.JoinNode.Type.RIGHT) PlanBuilder.constantExpressions(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.constantExpressions) REPLICATED(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType.REPLICATED) LEFT(com.facebook.presto.sql.planner.plan.JoinNode.Type.LEFT) CostComparator(com.facebook.presto.cost.CostComparator) Optional(java.util.Optional) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) PARTITIONED(com.facebook.presto.sql.planner.plan.JoinNode.DistributionType.PARTITIONED) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) RuleTester(com.facebook.presto.sql.planner.iterative.rule.test.RuleTester) INNER(com.facebook.presto.sql.planner.plan.JoinNode.Type.INNER) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) Test(org.testng.annotations.Test)

Example 84 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestDetermineSemiJoinDistributionType method testPartitionsWhenBothTablesEqual.

@Test
public void testPartitionsWhenBothTablesEqual() {
    int aRows = 10_000;
    int bRows = 10_000;
    assertDetermineSemiJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "A1", BIGINT), VariableStatsEstimate.unknown())).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "B1", BIGINT), VariableStatsEstimate.unknown())).build()).on(p -> p.semiJoin(p.values(new PlanNodeId("valuesA"), aRows, p.variable("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.variable("B1", BIGINT)), p.variable("A1"), p.variable("B1"), p.variable("output"), Optional.empty(), Optional.empty(), Optional.empty())).matches(semiJoin("A1", "B1", "output", Optional.of(PARTITIONED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) AfterClass(org.testng.annotations.AfterClass) BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) ImmutableMap(com.google.common.collect.ImmutableMap) PlanMatchPattern.semiJoin(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.semiJoin) BeforeClass(org.testng.annotations.BeforeClass) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) JOIN_DISTRIBUTION_TYPE(com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) PARTITIONED(com.facebook.presto.sql.planner.plan.SemiJoinNode.DistributionType.PARTITIONED) PlanBuilder.constantExpressions(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.constantExpressions) REPLICATED(com.facebook.presto.sql.planner.plan.SemiJoinNode.DistributionType.REPLICATED) JoinDistributionType(com.facebook.presto.sql.analyzer.FeaturesConfig.JoinDistributionType) ImmutableList(com.google.common.collect.ImmutableList) JOIN_MAX_BROADCAST_TABLE_SIZE(com.facebook.presto.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) CostComparator(com.facebook.presto.cost.CostComparator) Optional(java.util.Optional) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) RuleAssert(com.facebook.presto.sql.planner.iterative.rule.test.RuleAssert) TaskCountEstimator(com.facebook.presto.cost.TaskCountEstimator) RuleTester(com.facebook.presto.sql.planner.iterative.rule.test.RuleTester) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test)

Example 85 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestDetermineSemiJoinDistributionType method testPartitionWhenRequiredBySession.

@Test
public void testPartitionWhenRequiredBySession() {
    int aRows = 10_000;
    int bRows = 100;
    assertDetermineSemiJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.PARTITIONED.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "A1", BIGINT), new VariableStatsEstimate(0, 100, 0, 6400, 100))).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addVariableStatistics(ImmutableMap.of(new VariableReferenceExpression(Optional.empty(), "B1", BIGINT), new VariableStatsEstimate(0, 100, 0, 640000, 100))).build()).on(p -> p.semiJoin(p.values(new PlanNodeId("valuesA"), aRows, p.variable("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.variable("B1", BIGINT)), p.variable("A1"), p.variable("B1"), p.variable("output"), Optional.empty(), Optional.empty(), Optional.empty())).matches(semiJoin("A1", "B1", "output", Optional.of(PARTITIONED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) AfterClass(org.testng.annotations.AfterClass) BIGINT(com.facebook.presto.common.type.BigintType.BIGINT) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) ImmutableMap(com.google.common.collect.ImmutableMap) PlanMatchPattern.semiJoin(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.semiJoin) BeforeClass(org.testng.annotations.BeforeClass) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) JOIN_DISTRIBUTION_TYPE(com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) PARTITIONED(com.facebook.presto.sql.planner.plan.SemiJoinNode.DistributionType.PARTITIONED) PlanBuilder.constantExpressions(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.constantExpressions) REPLICATED(com.facebook.presto.sql.planner.plan.SemiJoinNode.DistributionType.REPLICATED) JoinDistributionType(com.facebook.presto.sql.analyzer.FeaturesConfig.JoinDistributionType) ImmutableList(com.google.common.collect.ImmutableList) JOIN_MAX_BROADCAST_TABLE_SIZE(com.facebook.presto.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) CostComparator(com.facebook.presto.cost.CostComparator) Optional(java.util.Optional) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) RuleAssert(com.facebook.presto.sql.planner.iterative.rule.test.RuleAssert) TaskCountEstimator(com.facebook.presto.cost.TaskCountEstimator) RuleTester(com.facebook.presto.sql.planner.iterative.rule.test.RuleTester) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) Test(org.testng.annotations.Test)

Aggregations

PlanNodeId (com.facebook.presto.spi.plan.PlanNodeId)204 Test (org.testng.annotations.Test)123 Page (com.facebook.presto.common.Page)83 MaterializedResult (com.facebook.presto.testing.MaterializedResult)52 Type (com.facebook.presto.common.type.Type)47 VariableReferenceExpression (com.facebook.presto.spi.relation.VariableReferenceExpression)43 ImmutableList (com.google.common.collect.ImmutableList)43 RowPagesBuilder (com.facebook.presto.RowPagesBuilder)39 DataSize (io.airlift.units.DataSize)39 Optional (java.util.Optional)35 ImmutableMap (com.google.common.collect.ImmutableMap)34 JoinNode (com.facebook.presto.sql.planner.plan.JoinNode)25 BIGINT (com.facebook.presto.common.type.BigintType.BIGINT)23 VariableStatsEstimate (com.facebook.presto.cost.VariableStatsEstimate)23 Split (com.facebook.presto.metadata.Split)23 OperatorFactory (com.facebook.presto.operator.OperatorFactory)23 PlanNodeStatsEstimate (com.facebook.presto.cost.PlanNodeStatsEstimate)22 RowExpression (com.facebook.presto.spi.relation.RowExpression)21 PlanMatchPattern.values (com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values)21 JOIN_DISTRIBUTION_TYPE (com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE)20