Search in sources :

Example 1 with VacuumTableNode

use of io.prestosql.sql.planner.plan.VacuumTableNode in project hetu-core by openlookeng.

the class TestExternalFunctionPushDownChecker method testVacuumTableNode.

@Test
public void testVacuumTableNode() {
    TableWriterNode.DeleteTarget deleteTarget = new TableWriterNode.DeleteTarget(tableHandle, new SchemaTableName("sch", "tab"));
    PlanNode node = new VacuumTableNode(idAllocator.getNextId(), tableHandle, deleteTarget, columnA, columnB, "p1", false, ImmutableList.of(), Optional.of(new StatisticAggregations(ImmutableMap.of(columnA, new AggregationNode.Aggregation(sumCall, sumCall.getArguments(), false, Optional.empty(), Optional.empty(), Optional.empty())), ImmutableList.of(columnA))), Optional.of(new StatisticAggregationsDescriptor<>(ImmutableMap.of(), ImmutableMap.of(), ImmutableMap.of())));
    validatePlan(node);
}
Also used : PlanNode(io.prestosql.spi.plan.PlanNode) VacuumTableNode(io.prestosql.sql.planner.plan.VacuumTableNode) StatisticAggregationsDescriptor(io.prestosql.sql.planner.plan.StatisticAggregationsDescriptor) TableWriterNode(io.prestosql.sql.planner.plan.TableWriterNode) SchemaTableName(io.prestosql.spi.connector.SchemaTableName) StatisticAggregations(io.prestosql.sql.planner.plan.StatisticAggregations) Test(org.testng.annotations.Test) BasePlanTest(io.prestosql.sql.planner.assertions.BasePlanTest)

Example 2 with VacuumTableNode

use of io.prestosql.sql.planner.plan.VacuumTableNode in project hetu-core by openlookeng.

the class LogicalPlanner method createVacuumWriterPlan.

private RelationPlan createVacuumWriterPlan(Analysis analysis, TableHandle handle, VacuumTable node, WriterTarget target, List<Symbol> symbols, List<String> columnNames, TableStatisticsMetadata statisticsMetadata) {
    Optional<StatisticAggregations> statisticsAggregation = Optional.empty();
    Optional<StatisticAggregationsDescriptor<Symbol>> statisticsAggregationDescriptor = Optional.empty();
    Optional<StatisticAggregations> finalStatisticsAggregation = Optional.empty();
    Optional<StatisticAggregationsDescriptor<Symbol>> finalStatisticsAggregationDescriptor = Optional.empty();
    if (!statisticsMetadata.isEmpty()) {
        verify(columnNames.size() == symbols.size(), "columnNames.size() != symbols.size(): %s and %s", columnNames, symbols);
        Map<String, Symbol> columnToSymbolMap = zip(columnNames.stream(), symbols.stream(), SimpleImmutableEntry::new).collect(toImmutableMap(Entry::getKey, Entry::getValue));
        TableStatisticAggregation result = statisticsAggregationPlanner.createStatisticsAggregation(statisticsMetadata, columnToSymbolMap);
        StatisticAggregations.Parts aggregations = result.getAggregations().createPartialAggregations(planSymbolAllocator, metadata);
        // partial aggregation is run within the VacuumTableOperator to calculate the statistics for
        // the data consumed by the VacuumTableOperator
        // final aggregation is run within the TableFinishOperator to summarize collected statistics
        // by the partial aggregation from all of the writer nodes
        statisticsAggregation = Optional.of(aggregations.getPartialAggregation());
        statisticsAggregationDescriptor = Optional.of(result.getDescriptor().map(aggregations.getMappings()::get));
        finalStatisticsAggregation = Optional.of(aggregations.getFinalAggregation());
        finalStatisticsAggregationDescriptor = Optional.of(result.getDescriptor());
    }
    VacuumTableNode vacuumTableNode = new VacuumTableNode(idAllocator.getNextId(), handle, target, planSymbolAllocator.newSymbol("partialrows", BIGINT), planSymbolAllocator.newSymbol("fragment", VARBINARY), node.getPartition().orElse(""), node.isFull(), symbols, statisticsAggregation, statisticsAggregationDescriptor);
    TableFinishNode commitNode = new TableFinishNode(idAllocator.getNextId(), vacuumTableNode, target, planSymbolAllocator.newSymbol("rows", BIGINT), finalStatisticsAggregation, finalStatisticsAggregationDescriptor);
    return new RelationPlan(commitNode, analysis.getRootScope(), commitNode.getOutputSymbols());
}
Also used : TableStatisticAggregation(io.prestosql.sql.planner.StatisticsAggregationPlanner.TableStatisticAggregation) VacuumTableNode(io.prestosql.sql.planner.plan.VacuumTableNode) Symbol(io.prestosql.spi.plan.Symbol) StatisticAggregationsDescriptor(io.prestosql.sql.planner.plan.StatisticAggregationsDescriptor) TableFinishNode(io.prestosql.sql.planner.plan.TableFinishNode) StatisticAggregations(io.prestosql.sql.planner.plan.StatisticAggregations)

Example 3 with VacuumTableNode

use of io.prestosql.sql.planner.plan.VacuumTableNode in project hetu-core by openlookeng.

the class TestExternalFunctionPushDownChecker method testVacuumTableNodeWithExternalCall.

@Test(expectedExceptions = ExternalFunctionPushDownChecker.IllegalExternalFunctionUsageException.class, expectedExceptionsMessageRegExp = "The external function jdbc.v1.foo does not support to push down to data source for this query.")
public void testVacuumTableNodeWithExternalCall() {
    TableWriterNode.DeleteTarget deleteTarget = new TableWriterNode.DeleteTarget(tableHandle, new SchemaTableName("sch", "tab"));
    PlanNode node = new VacuumTableNode(idAllocator.getNextId(), tableHandle, deleteTarget, columnA, columnB, "p1", false, ImmutableList.of(), Optional.of(new StatisticAggregations(ImmutableMap.of(columnA, new AggregationNode.Aggregation(externalFooCall1, externalFooCall1.getArguments(), false, Optional.empty(), Optional.empty(), Optional.empty())), ImmutableList.of(columnA))), Optional.of(new StatisticAggregationsDescriptor<>(ImmutableMap.of(), ImmutableMap.of(), ImmutableMap.of())));
    validatePlan(node);
}
Also used : PlanNode(io.prestosql.spi.plan.PlanNode) VacuumTableNode(io.prestosql.sql.planner.plan.VacuumTableNode) StatisticAggregationsDescriptor(io.prestosql.sql.planner.plan.StatisticAggregationsDescriptor) TableWriterNode(io.prestosql.sql.planner.plan.TableWriterNode) SchemaTableName(io.prestosql.spi.connector.SchemaTableName) StatisticAggregations(io.prestosql.sql.planner.plan.StatisticAggregations) Test(org.testng.annotations.Test) BasePlanTest(io.prestosql.sql.planner.assertions.BasePlanTest)

Aggregations

StatisticAggregations (io.prestosql.sql.planner.plan.StatisticAggregations)3 StatisticAggregationsDescriptor (io.prestosql.sql.planner.plan.StatisticAggregationsDescriptor)3 VacuumTableNode (io.prestosql.sql.planner.plan.VacuumTableNode)3 SchemaTableName (io.prestosql.spi.connector.SchemaTableName)2 PlanNode (io.prestosql.spi.plan.PlanNode)2 BasePlanTest (io.prestosql.sql.planner.assertions.BasePlanTest)2 TableWriterNode (io.prestosql.sql.planner.plan.TableWriterNode)2 Test (org.testng.annotations.Test)2 Symbol (io.prestosql.spi.plan.Symbol)1 TableStatisticAggregation (io.prestosql.sql.planner.StatisticsAggregationPlanner.TableStatisticAggregation)1 TableFinishNode (io.prestosql.sql.planner.plan.TableFinishNode)1