Search in sources :

Example 1 with InMemoryHashAggregationBuilder

use of com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder in project presto by prestodb.

the class HashAggregationOperator method addInput.

@Override
public void addInput(Page page) {
    checkState(unfinishedWork == null, "Operator has unfinished work");
    checkState(!finishing, "Operator is already finishing");
    requireNonNull(page, "page is null");
    inputProcessed = true;
    if (aggregationBuilder == null) {
        if (step.isOutputPartial() || !spillEnabled) {
            aggregationBuilder = new InMemoryHashAggregationBuilder(accumulatorFactories, step, expectedGroups, groupByTypes, groupByChannels, hashChannel, operatorContext, maxPartialMemory, joinCompiler, true, useSystemMemory);
        } else {
            verify(!useSystemMemory, "using system memory in spillable aggregations is not supported");
            aggregationBuilder = new SpillableHashAggregationBuilder(accumulatorFactories, step, expectedGroups, groupByTypes, groupByChannels, hashChannel, operatorContext, memoryLimitForMerge, memoryLimitForMergeWithMemory, spillerFactory, joinCompiler);
        }
    // assume initial aggregationBuilder is not full
    } else {
        checkState(!aggregationBuilder.isFull(), "Aggregation buffer is full");
    }
    // process the current page; save the unfinished work if we are waiting for memory
    unfinishedWork = aggregationBuilder.processPage(page);
    if (unfinishedWork.process()) {
        unfinishedWork = null;
    }
    aggregationBuilder.updateMemory();
}
Also used : InMemoryHashAggregationBuilder(com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder) SpillableHashAggregationBuilder(com.facebook.presto.operator.aggregation.builder.SpillableHashAggregationBuilder)

Example 2 with InMemoryHashAggregationBuilder

use of com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder in project presto by prestodb.

the class TestHashAggregationOperator method getHashCapacity.

private int getHashCapacity(Operator operator) {
    assertTrue(operator instanceof HashAggregationOperator);
    HashAggregationBuilder aggregationBuilder = ((HashAggregationOperator) operator).getAggregationBuilder();
    if (aggregationBuilder == null) {
        return 0;
    }
    assertTrue(aggregationBuilder instanceof InMemoryHashAggregationBuilder);
    return ((InMemoryHashAggregationBuilder) aggregationBuilder).getCapacity();
}
Also used : HashAggregationBuilder(com.facebook.presto.operator.aggregation.builder.HashAggregationBuilder) InMemoryHashAggregationBuilder(com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder) InMemoryHashAggregationBuilder(com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder)

Aggregations

InMemoryHashAggregationBuilder (com.facebook.presto.operator.aggregation.builder.InMemoryHashAggregationBuilder)2 HashAggregationBuilder (com.facebook.presto.operator.aggregation.builder.HashAggregationBuilder)1 SpillableHashAggregationBuilder (com.facebook.presto.operator.aggregation.builder.SpillableHashAggregationBuilder)1