Examples with QuantileDigest io.airlift.stats.QuantileDigest used on opensource projects

Example 6 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileAggregations method weightedInput.

@InputFunction
public static void weightedInput(@AggregationState DigestAndPercentileState state, @SqlType(StandardTypes.BIGINT) long value, @SqlType(StandardTypes.BIGINT) long weight, @SqlType(StandardTypes.DOUBLE) double percentile, @SqlType(StandardTypes.DOUBLE) double accuracy) {
    checkWeight(weight);
    QuantileDigest digest = state.getDigest();
    if (digest == null) {
        if (accuracy > 0 && accuracy < 1) {
            digest = new QuantileDigest(accuracy);
        } else {
            throw new IllegalArgumentException("Percentile accuracy must be strictly between 0 and 1");
        }
        state.setDigest(digest);
        state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
    }
    state.addMemoryUsage(-digest.estimatedInMemorySizeInBytes());
    digest.add(value, weight);
    state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
    // use last percentile
    state.setPercentile(percentile);
}

Example 7 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class DigestAndPercentileStateSerializer method deserialize.

@Override
public void deserialize(Block block, int index, DigestAndPercentileState state) {
    SliceInput input = VARBINARY.getSlice(block, index).getInput();
    // read percentile
    state.setPercentile(input.readDouble());
    // read digest
    int length = input.readInt();
    QuantileDigest digest = new QuantileDigest(input.readSlice(length));
    state.setDigest(digest);
    state.addMemoryUsage(state.getDigest().estimatedInMemorySizeInBytes());
}

Example 8 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileArrayAggregations method input.

@InputFunction
public static void input(@AggregationState DigestAndPercentileArrayState state, @SqlType(StandardTypes.BIGINT) long value, @SqlType("array(double)") Block percentilesArrayBlock) {
    initializePercentilesArray(state, percentilesArrayBlock);
    initializeDigest(state);
    QuantileDigest digest = state.getDigest();
    state.addMemoryUsage(-digest.estimatedInMemorySizeInBytes());
    digest.add(value);
    state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
}

Example 9 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileArrayAggregations method combine.

@CombineFunction
public static void combine(@AggregationState DigestAndPercentileArrayState state, DigestAndPercentileArrayState otherState) {
    QuantileDigest otherDigest = otherState.getDigest();
    QuantileDigest digest = state.getDigest();
    if (digest == null) {
        state.setDigest(otherDigest);
        state.addMemoryUsage(otherDigest.estimatedInMemorySizeInBytes());
    } else {
        state.addMemoryUsage(-digest.estimatedInMemorySizeInBytes());
        digest.merge(otherDigest);
        state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
    }
    state.setPercentiles(otherState.getPercentiles());
}

Example 10 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileArrayAggregations method output.

@OutputFunction("array(bigint)")
public static void output(@AggregationState DigestAndPercentileArrayState state, BlockBuilder out) {
    QuantileDigest digest = state.getDigest();
    List<Double> percentiles = state.getPercentiles();
    if (percentiles == null || digest == null) {
        out.appendNull();
        return;
    }
    BlockBuilder blockBuilder = out.beginBlockEntry();
    for (int i = 0; i < percentiles.size(); i++) {
        Double percentile = percentiles.get(i);
        BIGINT.writeLong(blockBuilder, digest.getQuantile(percentile));
    }
    out.closeEntry();
}