Search in sources :

Example 11 with HyperLogLog

use of io.airlift.stats.cardinality.HyperLogLog in project presto by prestodb.

the class ApproximateCountDistinctAggregations method inputBinary.

@InputFunction
public static void inputBinary(@AggregationState HyperLogLogState state, @SqlType(StandardTypes.VARBINARY) Slice value, @SqlType(StandardTypes.DOUBLE) double maxStandardError) {
    HyperLogLog hll = getOrCreateHyperLogLog(state, maxStandardError);
    state.addMemoryUsage(-hll.estimatedInMemorySize());
    hll.add(value);
    state.addMemoryUsage(hll.estimatedInMemorySize());
}
Also used : HyperLogLog(io.airlift.stats.cardinality.HyperLogLog) InputFunction(com.facebook.presto.spi.function.InputFunction)

Example 12 with HyperLogLog

use of io.airlift.stats.cardinality.HyperLogLog in project presto by prestodb.

the class ApproximateCountDistinctAggregations method getOrCreateHyperLogLog.

private static HyperLogLog getOrCreateHyperLogLog(HyperLogLogState state, double maxStandardError) {
    HyperLogLog hll = state.getHyperLogLog();
    if (hll == null) {
        hll = HyperLogLog.newInstance(standardErrorToBuckets(maxStandardError));
        state.setHyperLogLog(hll);
        state.addMemoryUsage(hll.estimatedInMemorySize());
    }
    return hll;
}
Also used : HyperLogLog(io.airlift.stats.cardinality.HyperLogLog)

Example 13 with HyperLogLog

use of io.airlift.stats.cardinality.HyperLogLog in project presto by prestodb.

the class ApproximateCountDistinctAggregations method combineState.

@CombineFunction
public static void combineState(@AggregationState HyperLogLogState state, @AggregationState HyperLogLogState otherState) {
    HyperLogLog input = otherState.getHyperLogLog();
    HyperLogLog previous = state.getHyperLogLog();
    if (previous == null) {
        state.setHyperLogLog(input);
        state.addMemoryUsage(input.estimatedInMemorySize());
    } else {
        state.addMemoryUsage(-previous.estimatedInMemorySize());
        previous.mergeWith(input);
        state.addMemoryUsage(previous.estimatedInMemorySize());
    }
}
Also used : HyperLogLog(io.airlift.stats.cardinality.HyperLogLog) CombineFunction(com.facebook.presto.spi.function.CombineFunction)

Example 14 with HyperLogLog

use of io.airlift.stats.cardinality.HyperLogLog in project presto by prestodb.

the class ApproximateCountDistinctAggregations method input.

@InputFunction
public static void input(@AggregationState HyperLogLogState state, @SqlType(StandardTypes.BIGINT) long value, @SqlType(StandardTypes.DOUBLE) double maxStandardError) {
    HyperLogLog hll = getOrCreateHyperLogLog(state, maxStandardError);
    state.addMemoryUsage(-hll.estimatedInMemorySize());
    hll.add(value);
    state.addMemoryUsage(hll.estimatedInMemorySize());
}
Also used : HyperLogLog(io.airlift.stats.cardinality.HyperLogLog) InputFunction(com.facebook.presto.spi.function.InputFunction)

Aggregations

HyperLogLog (io.airlift.stats.cardinality.HyperLogLog)14 InputFunction (com.facebook.presto.spi.function.InputFunction)6 CombineFunction (com.facebook.presto.spi.function.CombineFunction)2 SqlType (com.facebook.presto.spi.function.SqlType)2 Block (com.facebook.presto.spi.block.Block)1 BlockBuilder (com.facebook.presto.spi.block.BlockBuilder)1 BlockBuilderStatus (com.facebook.presto.spi.block.BlockBuilderStatus)1 LiteralParameters (com.facebook.presto.spi.function.LiteralParameters)1 ScalarFunction (com.facebook.presto.spi.function.ScalarFunction)1 ScalarOperator (com.facebook.presto.spi.function.ScalarOperator)1