Search in sources :

Example 96 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class FunctionAssertions method compileScanFilterProject.

private static SourceOperatorFactory compileScanFilterProject(SqlFunctionProperties sqlFunctionProperties, Optional<RowExpression> filter, RowExpression projection, ExpressionCompiler compiler) {
    try {
        Supplier<CursorProcessor> cursorProcessor = compiler.compileCursorProcessor(sqlFunctionProperties, filter, ImmutableList.of(projection), SOURCE_ID);
        Supplier<PageProcessor> pageProcessor = compiler.compilePageProcessor(sqlFunctionProperties, filter, ImmutableList.of(projection));
        return new ScanFilterAndProjectOperator.ScanFilterAndProjectOperatorFactory(0, new PlanNodeId("test"), SOURCE_ID, PAGE_SOURCE_PROVIDER, cursorProcessor, pageProcessor, new TableHandle(new ConnectorId("test"), new ConnectorTableHandle() {
        }, new ConnectorTransactionHandle() {
        }, Optional.empty()), ImmutableList.of(), ImmutableList.of(projection.getType()), Optional.empty(), new DataSize(0, BYTE), 0);
    } catch (Throwable e) {
        if (e instanceof UncheckedExecutionException) {
            e = e.getCause();
        }
        throw new RuntimeException("Error compiling filter " + filter + ": " + e.getMessage(), e);
    }
}
Also used : CursorProcessor(com.facebook.presto.operator.project.CursorProcessor) UncheckedExecutionException(com.google.common.util.concurrent.UncheckedExecutionException) ConnectorTransactionHandle(com.facebook.presto.spi.connector.ConnectorTransactionHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PageProcessor(com.facebook.presto.operator.project.PageProcessor) DataSize(io.airlift.units.DataSize) TableHandle(com.facebook.presto.spi.TableHandle) ConnectorTableHandle(com.facebook.presto.spi.ConnectorTableHandle) ConnectorId(com.facebook.presto.spi.ConnectorId)

Example 97 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestPushPartialAggregationThroughExchange method testPartialAggregationEnabledWhenNotConfident.

@Test
public void testPartialAggregationEnabledWhenNotConfident() {
    tester().assertThat(new PushPartialAggregationThroughExchange(getFunctionManager())).setSystemProperty(PARTIAL_AGGREGATION_STRATEGY, "AUTOMATIC").on(p -> {
        VariableReferenceExpression a = p.variable("a", DOUBLE);
        VariableReferenceExpression b = p.variable("b", DOUBLE);
        return p.aggregation(ab -> ab.source(p.exchange(e -> e.addSource(p.values(new PlanNodeId("values"), a, b)).addInputsSet(a, b).singleDistributionPartitioningScheme(a, b))).addAggregation(p.variable("SUM", DOUBLE), expression("SUM(a)"), ImmutableList.of(DOUBLE)).singleGroupingSet(b).step(PARTIAL));
    }).overrideStats("values", PlanNodeStatsEstimate.builder().setOutputRowCount(1000).addVariableStatistics(variable("b", DOUBLE), new VariableStatsEstimate(0, 100, 0, 8, 800)).setConfident(false).build()).matches(exchange(project(aggregation(ImmutableMap.of("SUM", functionCall("sum", ImmutableList.of("a"))), PARTIAL, values("a", "b")))));
}
Also used : PARTIAL(com.facebook.presto.spi.plan.AggregationNode.Step.PARTIAL) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) PlanMatchPattern.project(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.project) ImmutableMap(com.google.common.collect.ImmutableMap) DOUBLE(com.facebook.presto.common.type.DoubleType.DOUBLE) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) PlanMatchPattern.exchange(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.exchange) SINGLE(com.facebook.presto.spi.plan.AggregationNode.Step.SINGLE) BaseRuleTest(com.facebook.presto.sql.planner.iterative.rule.test.BaseRuleTest) ImmutableList(com.google.common.collect.ImmutableList) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) Expressions.variable(com.facebook.presto.sql.relational.Expressions.variable) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) PlanBuilder.expression(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.expression) PlanMatchPattern.functionCall(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.functionCall) PARTIAL_AGGREGATION_STRATEGY(com.facebook.presto.SystemSessionProperties.PARTIAL_AGGREGATION_STRATEGY) PlanMatchPattern.aggregation(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.aggregation) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) Test(org.testng.annotations.Test) BaseRuleTest(com.facebook.presto.sql.planner.iterative.rule.test.BaseRuleTest)

Example 98 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class TestPushPartialAggregationThroughExchange method testNoPartialAggregationWhenReductionBelowThreshold.

@Test
public void testNoPartialAggregationWhenReductionBelowThreshold() {
    tester().assertThat(new PushPartialAggregationThroughExchange(getFunctionManager())).setSystemProperty(PARTIAL_AGGREGATION_STRATEGY, "AUTOMATIC").on(p -> {
        VariableReferenceExpression a = p.variable("a", DOUBLE);
        VariableReferenceExpression b = p.variable("b", DOUBLE);
        return p.aggregation(ab -> ab.source(p.exchange(e -> e.addSource(p.values(new PlanNodeId("values"), a, b)).addInputsSet(a, b).singleDistributionPartitioningScheme(a, b))).addAggregation(p.variable("SUM", DOUBLE), expression("SUM(a)"), ImmutableList.of(DOUBLE)).singleGroupingSet(b).step(SINGLE));
    }).overrideStats("values", PlanNodeStatsEstimate.builder().setOutputRowCount(1000).addVariableStatistics(variable("b", DOUBLE), new VariableStatsEstimate(0, 100, 0, 8, 800)).setConfident(true).build()).doesNotFire();
}
Also used : PARTIAL(com.facebook.presto.spi.plan.AggregationNode.Step.PARTIAL) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) PlanNodeStatsEstimate(com.facebook.presto.cost.PlanNodeStatsEstimate) PlanMatchPattern.project(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.project) ImmutableMap(com.google.common.collect.ImmutableMap) DOUBLE(com.facebook.presto.common.type.DoubleType.DOUBLE) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) Test(org.testng.annotations.Test) PlanMatchPattern.exchange(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.exchange) SINGLE(com.facebook.presto.spi.plan.AggregationNode.Step.SINGLE) BaseRuleTest(com.facebook.presto.sql.planner.iterative.rule.test.BaseRuleTest) ImmutableList(com.google.common.collect.ImmutableList) PlanMatchPattern.values(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values) Expressions.variable(com.facebook.presto.sql.relational.Expressions.variable) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) PlanBuilder.expression(com.facebook.presto.sql.planner.iterative.rule.test.PlanBuilder.expression) PlanMatchPattern.functionCall(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.functionCall) PARTIAL_AGGREGATION_STRATEGY(com.facebook.presto.SystemSessionProperties.PARTIAL_AGGREGATION_STRATEGY) PlanMatchPattern.aggregation(com.facebook.presto.sql.planner.assertions.PlanMatchPattern.aggregation) PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) VariableReferenceExpression(com.facebook.presto.spi.relation.VariableReferenceExpression) VariableStatsEstimate(com.facebook.presto.cost.VariableStatsEstimate) Test(org.testng.annotations.Test) BaseRuleTest(com.facebook.presto.sql.planner.iterative.rule.test.BaseRuleTest)

Example 99 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class CountAggregationBenchmark method createOperatorFactories.

@Override
protected List<? extends OperatorFactory> createOperatorFactories() {
    OperatorFactory tableScanOperator = createTableScanOperator(0, new PlanNodeId("test"), "orders", "orderkey");
    FunctionAndTypeManager functionAndTypeManager = localQueryRunner.getMetadata().getFunctionAndTypeManager();
    InternalAggregationFunction countFunction = functionAndTypeManager.getAggregateFunctionImplementation(functionAndTypeManager.lookupFunction("count", fromTypes(BIGINT)));
    AggregationOperatorFactory aggregationOperator = new AggregationOperatorFactory(1, new PlanNodeId("test"), Step.SINGLE, ImmutableList.of(countFunction.bind(ImmutableList.of(0), Optional.empty())), false);
    return ImmutableList.of(tableScanOperator, aggregationOperator);
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) FunctionAndTypeManager(com.facebook.presto.metadata.FunctionAndTypeManager) OperatorFactory(com.facebook.presto.operator.OperatorFactory) AggregationOperatorFactory(com.facebook.presto.operator.AggregationOperator.AggregationOperatorFactory) AggregationOperatorFactory(com.facebook.presto.operator.AggregationOperator.AggregationOperatorFactory) InternalAggregationFunction(com.facebook.presto.operator.aggregation.InternalAggregationFunction)

Example 100 with PlanNodeId

use of com.facebook.presto.spi.plan.PlanNodeId in project presto by prestodb.

the class SqlTask method createTaskInfo.

private TaskInfo createTaskInfo(TaskHolder taskHolder) {
    TaskStats taskStats = getTaskStats(taskHolder);
    Set<PlanNodeId> noMoreSplits = getNoMoreSplits(taskHolder);
    MetadataUpdates metadataRequests = getMetadataUpdateRequests(taskHolder);
    TaskStatus taskStatus = createTaskStatus(taskHolder);
    return new TaskInfo(taskStateMachine.getTaskId(), taskStatus, lastHeartbeat.get(), outputBuffer.getInfo(), noMoreSplits, taskStats, needsPlan.get(), metadataRequests, nodeId);
}
Also used : PlanNodeId(com.facebook.presto.spi.plan.PlanNodeId) MetadataUpdates(com.facebook.presto.metadata.MetadataUpdates) TaskStats(com.facebook.presto.operator.TaskStats)

Aggregations

PlanNodeId (com.facebook.presto.spi.plan.PlanNodeId)204 Test (org.testng.annotations.Test)123 Page (com.facebook.presto.common.Page)83 MaterializedResult (com.facebook.presto.testing.MaterializedResult)52 Type (com.facebook.presto.common.type.Type)47 VariableReferenceExpression (com.facebook.presto.spi.relation.VariableReferenceExpression)43 ImmutableList (com.google.common.collect.ImmutableList)43 RowPagesBuilder (com.facebook.presto.RowPagesBuilder)39 DataSize (io.airlift.units.DataSize)39 Optional (java.util.Optional)35 ImmutableMap (com.google.common.collect.ImmutableMap)34 JoinNode (com.facebook.presto.sql.planner.plan.JoinNode)25 BIGINT (com.facebook.presto.common.type.BigintType.BIGINT)23 VariableStatsEstimate (com.facebook.presto.cost.VariableStatsEstimate)23 Split (com.facebook.presto.metadata.Split)23 OperatorFactory (com.facebook.presto.operator.OperatorFactory)23 PlanNodeStatsEstimate (com.facebook.presto.cost.PlanNodeStatsEstimate)22 RowExpression (com.facebook.presto.spi.relation.RowExpression)21 PlanMatchPattern.values (com.facebook.presto.sql.planner.assertions.PlanMatchPattern.values)21 JOIN_DISTRIBUTION_TYPE (com.facebook.presto.SystemSessionProperties.JOIN_DISTRIBUTION_TYPE)20