Search in sources :

Example 36 with BIGINT

use of io.trino.spi.type.BigintType.BIGINT in project trino by trinodb.

the class TestDetermineJoinDistributionType method testFlipAndReplicateWhenOneTableMuchSmallerAndJoinCardinalityUnknown.

@Test
public void testFlipAndReplicateWhenOneTableMuchSmallerAndJoinCardinalityUnknown() {
    int aRows = 100;
    int bRows = 10_000;
    assertDetermineJoinDistributionType().setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addSymbolStatistics(ImmutableMap.of(new Symbol("A1"), SymbolStatsEstimate.unknown())).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addSymbolStatistics(ImmutableMap.of(new Symbol("B1"), SymbolStatsEstimate.unknown())).build()).on(p -> p.join(INNER, p.values(new PlanNodeId("valuesA"), aRows, p.symbol("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.symbol("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.symbol("A1", BIGINT), p.symbol("B1", BIGINT))), ImmutableList.of(p.symbol("A1", BIGINT)), ImmutableList.of(p.symbol("B1", BIGINT)), Optional.empty())).matches(join(INNER, ImmutableList.of(equiJoinClause("B1", "A1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("B1", 0)), values(ImmutableMap.of("A1", 0))));
}
Also used : PARTITIONED(io.trino.sql.planner.plan.JoinNode.DistributionType.PARTITIONED) INNER(io.trino.sql.planner.plan.JoinNode.Type.INNER) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) Assert.assertEquals(org.testng.Assert.assertEquals) Test(org.testng.annotations.Test) PlanMatchPattern.filter(io.trino.sql.planner.assertions.PlanMatchPattern.filter) REPLICATED(io.trino.sql.planner.plan.JoinNode.DistributionType.REPLICATED) Lookup.noLookup(io.trino.sql.planner.iterative.Lookup.noLookup) VarcharType(io.trino.spi.type.VarcharType) LEFT(io.trino.sql.planner.plan.JoinNode.Type.LEFT) RuleAssert(io.trino.sql.planner.iterative.rule.test.RuleAssert) Type(io.trino.sql.planner.plan.JoinNode.Type) PlanBuilder.expressions(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expressions) ImmutableList(com.google.common.collect.ImmutableList) NaN(java.lang.Double.NaN) DistributionType(io.trino.sql.planner.plan.JoinNode.DistributionType) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) PlanBuilder(io.trino.sql.planner.iterative.rule.test.PlanBuilder) PlanMatchPattern.equiJoinClause(io.trino.sql.planner.assertions.PlanMatchPattern.equiJoinClause) JoinNode(io.trino.sql.planner.plan.JoinNode) TableScanNode(io.trino.sql.planner.plan.TableScanNode) PlanMatchPattern.join(io.trino.sql.planner.assertions.PlanMatchPattern.join) PlanNodeStatsEstimate(io.trino.cost.PlanNodeStatsEstimate) JOIN_MAX_BROADCAST_TABLE_SIZE(io.trino.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) TaskCountEstimator(io.trino.cost.TaskCountEstimator) Symbol(io.trino.sql.planner.Symbol) AfterClass(org.testng.annotations.AfterClass) SymbolStatsEstimate(io.trino.cost.SymbolStatsEstimate) RuleTester.defaultRuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester.defaultRuleTester) ImmutableMap(com.google.common.collect.ImmutableMap) BeforeClass(org.testng.annotations.BeforeClass) FULL(io.trino.sql.planner.plan.JoinNode.Type.FULL) RuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester) PlanMatchPattern.values(io.trino.sql.planner.assertions.PlanMatchPattern.values) TRUE_LITERAL(io.trino.sql.tree.BooleanLiteral.TRUE_LITERAL) JoinDistributionType(io.trino.sql.planner.OptimizerConfig.JoinDistributionType) DetermineJoinDistributionType.getSourceTablesSizeInBytes(io.trino.sql.planner.iterative.rule.DetermineJoinDistributionType.getSourceTablesSizeInBytes) CostComparator(io.trino.cost.CostComparator) PlanMatchPattern.enforceSingleRow(io.trino.sql.planner.assertions.PlanMatchPattern.enforceSingleRow) BIGINT(io.trino.spi.type.BigintType.BIGINT) RIGHT(io.trino.sql.planner.plan.JoinNode.Type.RIGHT) JOIN_DISTRIBUTION_TYPE(io.trino.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) ImmutableListMultimap(com.google.common.collect.ImmutableListMultimap) Optional(java.util.Optional) ValuesNode(io.trino.sql.planner.plan.ValuesNode) TestingColumnHandle(io.trino.testing.TestingMetadata.TestingColumnHandle) PlanBuilder.expression(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expression) PlanNodeIdAllocator(io.trino.sql.planner.PlanNodeIdAllocator) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) Symbol(io.trino.sql.planner.Symbol) Test(org.testng.annotations.Test)

Example 37 with BIGINT

use of io.trino.spi.type.BigintType.BIGINT in project trino by trinodb.

the class TestDetermineJoinDistributionType method testFlipAndReplicateRightOuterJoin.

@Test
public void testFlipAndReplicateRightOuterJoin() {
    int aRows = 10;
    int bRows = 1_000_000;
    assertDetermineJoinDistributionType(new CostComparator(75, 10, 15)).setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addSymbolStatistics(ImmutableMap.of(new Symbol("A1"), new SymbolStatsEstimate(0, 100, 0, 640000, 100))).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addSymbolStatistics(ImmutableMap.of(new Symbol("B1"), new SymbolStatsEstimate(0, 100, 0, 640000, 100))).build()).on(p -> p.join(RIGHT, p.values(new PlanNodeId("valuesA"), aRows, p.symbol("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.symbol("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.symbol("A1", BIGINT), p.symbol("B1", BIGINT))), ImmutableList.of(p.symbol("A1", BIGINT)), ImmutableList.of(p.symbol("B1", BIGINT)), Optional.empty())).matches(join(LEFT, ImmutableList.of(equiJoinClause("A1", "B1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : CostComparator(io.trino.cost.CostComparator) PARTITIONED(io.trino.sql.planner.plan.JoinNode.DistributionType.PARTITIONED) INNER(io.trino.sql.planner.plan.JoinNode.Type.INNER) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) Assert.assertEquals(org.testng.Assert.assertEquals) Test(org.testng.annotations.Test) PlanMatchPattern.filter(io.trino.sql.planner.assertions.PlanMatchPattern.filter) REPLICATED(io.trino.sql.planner.plan.JoinNode.DistributionType.REPLICATED) Lookup.noLookup(io.trino.sql.planner.iterative.Lookup.noLookup) VarcharType(io.trino.spi.type.VarcharType) LEFT(io.trino.sql.planner.plan.JoinNode.Type.LEFT) RuleAssert(io.trino.sql.planner.iterative.rule.test.RuleAssert) Type(io.trino.sql.planner.plan.JoinNode.Type) PlanBuilder.expressions(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expressions) ImmutableList(com.google.common.collect.ImmutableList) NaN(java.lang.Double.NaN) DistributionType(io.trino.sql.planner.plan.JoinNode.DistributionType) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) PlanBuilder(io.trino.sql.planner.iterative.rule.test.PlanBuilder) PlanMatchPattern.equiJoinClause(io.trino.sql.planner.assertions.PlanMatchPattern.equiJoinClause) JoinNode(io.trino.sql.planner.plan.JoinNode) TableScanNode(io.trino.sql.planner.plan.TableScanNode) PlanMatchPattern.join(io.trino.sql.planner.assertions.PlanMatchPattern.join) PlanNodeStatsEstimate(io.trino.cost.PlanNodeStatsEstimate) JOIN_MAX_BROADCAST_TABLE_SIZE(io.trino.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) TaskCountEstimator(io.trino.cost.TaskCountEstimator) Symbol(io.trino.sql.planner.Symbol) AfterClass(org.testng.annotations.AfterClass) SymbolStatsEstimate(io.trino.cost.SymbolStatsEstimate) RuleTester.defaultRuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester.defaultRuleTester) ImmutableMap(com.google.common.collect.ImmutableMap) BeforeClass(org.testng.annotations.BeforeClass) FULL(io.trino.sql.planner.plan.JoinNode.Type.FULL) RuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester) PlanMatchPattern.values(io.trino.sql.planner.assertions.PlanMatchPattern.values) TRUE_LITERAL(io.trino.sql.tree.BooleanLiteral.TRUE_LITERAL) JoinDistributionType(io.trino.sql.planner.OptimizerConfig.JoinDistributionType) DetermineJoinDistributionType.getSourceTablesSizeInBytes(io.trino.sql.planner.iterative.rule.DetermineJoinDistributionType.getSourceTablesSizeInBytes) CostComparator(io.trino.cost.CostComparator) PlanMatchPattern.enforceSingleRow(io.trino.sql.planner.assertions.PlanMatchPattern.enforceSingleRow) BIGINT(io.trino.spi.type.BigintType.BIGINT) RIGHT(io.trino.sql.planner.plan.JoinNode.Type.RIGHT) JOIN_DISTRIBUTION_TYPE(io.trino.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) ImmutableListMultimap(com.google.common.collect.ImmutableListMultimap) Optional(java.util.Optional) ValuesNode(io.trino.sql.planner.plan.ValuesNode) TestingColumnHandle(io.trino.testing.TestingMetadata.TestingColumnHandle) PlanBuilder.expression(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expression) PlanNodeIdAllocator(io.trino.sql.planner.PlanNodeIdAllocator) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) Symbol(io.trino.sql.planner.Symbol) SymbolStatsEstimate(io.trino.cost.SymbolStatsEstimate) Test(org.testng.annotations.Test)

Example 38 with BIGINT

use of io.trino.spi.type.BigintType.BIGINT in project trino by trinodb.

the class TestDetermineJoinDistributionType method testFlipAndReplicateRightOuterJoinWhenJoinCardinalityUnknown.

@Test
public void testFlipAndReplicateRightOuterJoinWhenJoinCardinalityUnknown() {
    int aRows = 10;
    int bRows = 1_000_000;
    assertDetermineJoinDistributionType(new CostComparator(75, 10, 15)).setSystemProperty(JOIN_DISTRIBUTION_TYPE, JoinDistributionType.AUTOMATIC.name()).overrideStats("valuesA", PlanNodeStatsEstimate.builder().setOutputRowCount(aRows).addSymbolStatistics(ImmutableMap.of(new Symbol("A1"), SymbolStatsEstimate.unknown())).build()).overrideStats("valuesB", PlanNodeStatsEstimate.builder().setOutputRowCount(bRows).addSymbolStatistics(ImmutableMap.of(new Symbol("B1"), SymbolStatsEstimate.unknown())).build()).on(p -> p.join(RIGHT, p.values(new PlanNodeId("valuesA"), aRows, p.symbol("A1", BIGINT)), p.values(new PlanNodeId("valuesB"), bRows, p.symbol("B1", BIGINT)), ImmutableList.of(new JoinNode.EquiJoinClause(p.symbol("A1", BIGINT), p.symbol("B1", BIGINT))), ImmutableList.of(p.symbol("A1", BIGINT)), ImmutableList.of(p.symbol("B1", BIGINT)), Optional.empty())).matches(join(LEFT, ImmutableList.of(equiJoinClause("A1", "B1")), Optional.empty(), Optional.of(REPLICATED), values(ImmutableMap.of("A1", 0)), values(ImmutableMap.of("B1", 0))));
}
Also used : CostComparator(io.trino.cost.CostComparator) PARTITIONED(io.trino.sql.planner.plan.JoinNode.DistributionType.PARTITIONED) INNER(io.trino.sql.planner.plan.JoinNode.Type.INNER) VarcharType.createUnboundedVarcharType(io.trino.spi.type.VarcharType.createUnboundedVarcharType) Assert.assertEquals(org.testng.Assert.assertEquals) Test(org.testng.annotations.Test) PlanMatchPattern.filter(io.trino.sql.planner.assertions.PlanMatchPattern.filter) REPLICATED(io.trino.sql.planner.plan.JoinNode.DistributionType.REPLICATED) Lookup.noLookup(io.trino.sql.planner.iterative.Lookup.noLookup) VarcharType(io.trino.spi.type.VarcharType) LEFT(io.trino.sql.planner.plan.JoinNode.Type.LEFT) RuleAssert(io.trino.sql.planner.iterative.rule.test.RuleAssert) Type(io.trino.sql.planner.plan.JoinNode.Type) PlanBuilder.expressions(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expressions) ImmutableList(com.google.common.collect.ImmutableList) NaN(java.lang.Double.NaN) DistributionType(io.trino.sql.planner.plan.JoinNode.DistributionType) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) PlanBuilder(io.trino.sql.planner.iterative.rule.test.PlanBuilder) PlanMatchPattern.equiJoinClause(io.trino.sql.planner.assertions.PlanMatchPattern.equiJoinClause) JoinNode(io.trino.sql.planner.plan.JoinNode) TableScanNode(io.trino.sql.planner.plan.TableScanNode) PlanMatchPattern.join(io.trino.sql.planner.assertions.PlanMatchPattern.join) PlanNodeStatsEstimate(io.trino.cost.PlanNodeStatsEstimate) JOIN_MAX_BROADCAST_TABLE_SIZE(io.trino.SystemSessionProperties.JOIN_MAX_BROADCAST_TABLE_SIZE) TaskCountEstimator(io.trino.cost.TaskCountEstimator) Symbol(io.trino.sql.planner.Symbol) AfterClass(org.testng.annotations.AfterClass) SymbolStatsEstimate(io.trino.cost.SymbolStatsEstimate) RuleTester.defaultRuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester.defaultRuleTester) ImmutableMap(com.google.common.collect.ImmutableMap) BeforeClass(org.testng.annotations.BeforeClass) FULL(io.trino.sql.planner.plan.JoinNode.Type.FULL) RuleTester(io.trino.sql.planner.iterative.rule.test.RuleTester) PlanMatchPattern.values(io.trino.sql.planner.assertions.PlanMatchPattern.values) TRUE_LITERAL(io.trino.sql.tree.BooleanLiteral.TRUE_LITERAL) JoinDistributionType(io.trino.sql.planner.OptimizerConfig.JoinDistributionType) DetermineJoinDistributionType.getSourceTablesSizeInBytes(io.trino.sql.planner.iterative.rule.DetermineJoinDistributionType.getSourceTablesSizeInBytes) CostComparator(io.trino.cost.CostComparator) PlanMatchPattern.enforceSingleRow(io.trino.sql.planner.assertions.PlanMatchPattern.enforceSingleRow) BIGINT(io.trino.spi.type.BigintType.BIGINT) RIGHT(io.trino.sql.planner.plan.JoinNode.Type.RIGHT) JOIN_DISTRIBUTION_TYPE(io.trino.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE) ImmutableListMultimap(com.google.common.collect.ImmutableListMultimap) Optional(java.util.Optional) ValuesNode(io.trino.sql.planner.plan.ValuesNode) TestingColumnHandle(io.trino.testing.TestingMetadata.TestingColumnHandle) PlanBuilder.expression(io.trino.sql.planner.iterative.rule.test.PlanBuilder.expression) PlanNodeIdAllocator(io.trino.sql.planner.PlanNodeIdAllocator) PlanNodeId(io.trino.sql.planner.plan.PlanNodeId) Symbol(io.trino.sql.planner.Symbol) Test(org.testng.annotations.Test)

Example 39 with BIGINT

use of io.trino.spi.type.BigintType.BIGINT in project trino by trinodb.

the class PatternRecognitionExpressionRewriter method rewrite.

public static ExpressionAndValuePointers rewrite(Expression definition, Map<IrLabel, Set<IrLabel>> subsets) {
    Expression expression = rewriteIdentifiers(definition);
    Map<Symbol, Type> types = extractExpressions(ImmutableList.of(expression), SymbolReference.class).stream().collect(toImmutableMap(Symbol::from, reference -> BIGINT));
    return LogicalIndexExtractor.rewrite(expression, subsets, new SymbolAllocator(types), createTestMetadataManager());
}
Also used : ExpressionTreeUtils.extractExpressions(io.trino.sql.analyzer.ExpressionTreeUtils.extractExpressions) ParsingOptions(io.trino.sql.parser.ParsingOptions) SymbolAllocator(io.trino.sql.planner.SymbolAllocator) Type(io.trino.spi.type.Type) LambdaExpression(io.trino.sql.tree.LambdaExpression) Preconditions.checkArgument(com.google.common.base.Preconditions.checkArgument) RowDataType(io.trino.sql.tree.RowDataType) ImmutableList(com.google.common.collect.ImmutableList) Map(java.util.Map) GenericDataType(io.trino.sql.tree.GenericDataType) SqlParser(io.trino.sql.parser.SqlParser) ExpressionRewriter(io.trino.sql.tree.ExpressionRewriter) ExpressionAndValuePointers(io.trino.sql.planner.rowpattern.LogicalIndexExtractor.ExpressionAndValuePointers) Identifier(io.trino.sql.tree.Identifier) Symbol(io.trino.sql.planner.Symbol) IrLabel(io.trino.sql.planner.rowpattern.ir.IrLabel) LabelDereference(io.trino.sql.tree.LabelDereference) ExpressionTreeRewriter(io.trino.sql.tree.ExpressionTreeRewriter) LogicalIndexExtractor(io.trino.sql.planner.rowpattern.LogicalIndexExtractor) Set(java.util.Set) DereferenceExpression(io.trino.sql.tree.DereferenceExpression) ImmutableMap.toImmutableMap(com.google.common.collect.ImmutableMap.toImmutableMap) BIGINT(io.trino.spi.type.BigintType.BIGINT) SymbolReference(io.trino.sql.tree.SymbolReference) MetadataManager.createTestMetadataManager(io.trino.metadata.MetadataManager.createTestMetadataManager) Expression(io.trino.sql.tree.Expression) SymbolAllocator(io.trino.sql.planner.SymbolAllocator) Type(io.trino.spi.type.Type) RowDataType(io.trino.sql.tree.RowDataType) GenericDataType(io.trino.sql.tree.GenericDataType) LambdaExpression(io.trino.sql.tree.LambdaExpression) DereferenceExpression(io.trino.sql.tree.DereferenceExpression) Expression(io.trino.sql.tree.Expression) Symbol(io.trino.sql.planner.Symbol)

Example 40 with BIGINT

use of io.trino.spi.type.BigintType.BIGINT in project trino by trinodb.

the class TestShowQueries method init.

@BeforeAll
public void init() {
    LocalQueryRunner queryRunner = LocalQueryRunner.create(testSessionBuilder().setCatalog("local").setSchema("default").build());
    queryRunner.createCatalog("mock", MockConnectorFactory.builder().withGetColumns(schemaTableName -> ImmutableList.of(ColumnMetadata.builder().setName("colaa").setType(BIGINT).build(), ColumnMetadata.builder().setName("cola_").setType(BIGINT).build(), ColumnMetadata.builder().setName("colabc").setType(BIGINT).build())).withListSchemaNames(session -> ImmutableList.of("mockschema")).withListTables((session, schemaName) -> ImmutableList.of(new SchemaTableName("mockSchema", "mockTable"))).build(), ImmutableMap.of());
    queryRunner.createCatalog("testing_catalog", "mock", ImmutableMap.of());
    assertions = new QueryAssertions(queryRunner);
}
Also used : ImmutableMap(com.google.common.collect.ImmutableMap) ColumnMetadata(io.trino.spi.connector.ColumnMetadata) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) PER_CLASS(org.junit.jupiter.api.TestInstance.Lifecycle.PER_CLASS) SchemaTableName(io.trino.spi.connector.SchemaTableName) AfterAll(org.junit.jupiter.api.AfterAll) Test(org.junit.jupiter.api.Test) TestingSession.testSessionBuilder(io.trino.testing.TestingSession.testSessionBuilder) TestInstance(org.junit.jupiter.api.TestInstance) ImmutableList(com.google.common.collect.ImmutableList) MockConnectorFactory(io.trino.connector.MockConnectorFactory) BIGINT(io.trino.spi.type.BigintType.BIGINT) Assertions.assertThatThrownBy(org.assertj.core.api.Assertions.assertThatThrownBy) BeforeAll(org.junit.jupiter.api.BeforeAll) LocalQueryRunner(io.trino.testing.LocalQueryRunner) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) SchemaTableName(io.trino.spi.connector.SchemaTableName) LocalQueryRunner(io.trino.testing.LocalQueryRunner) BeforeAll(org.junit.jupiter.api.BeforeAll)

Aggregations

BIGINT (io.trino.spi.type.BigintType.BIGINT)106 ImmutableList (com.google.common.collect.ImmutableList)99 Optional (java.util.Optional)87 Test (org.testng.annotations.Test)86 ImmutableMap (com.google.common.collect.ImmutableMap)84 VARCHAR (io.trino.spi.type.VarcharType.VARCHAR)44 List (java.util.List)44 Map (java.util.Map)44 ColumnHandle (io.trino.spi.connector.ColumnHandle)38 Type (io.trino.spi.type.Type)38 Symbol (io.trino.sql.planner.Symbol)38 QualifiedName (io.trino.sql.tree.QualifiedName)38 ImmutableSet (com.google.common.collect.ImmutableSet)37 TupleDomain (io.trino.spi.predicate.TupleDomain)36 PlanMatchPattern.values (io.trino.sql.planner.assertions.PlanMatchPattern.values)36 TableHandle (io.trino.metadata.TableHandle)35 BaseRuleTest (io.trino.sql.planner.iterative.rule.test.BaseRuleTest)35 Session (io.trino.Session)34 PlanMatchPattern.filter (io.trino.sql.planner.assertions.PlanMatchPattern.filter)33 PlanBuilder.expression (io.trino.sql.planner.iterative.rule.test.PlanBuilder.expression)32