Examples with QuantileDigest io.airlift.stats.QuantileDigest used on opensource projects

Example 1 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileArrayAggregations method weightedInput.

@InputFunction
public static void weightedInput(@AggregationState DigestAndPercentileArrayState state, @SqlType(StandardTypes.BIGINT) long value, @SqlType(StandardTypes.BIGINT) long weight, @SqlType("array(double)") Block percentilesArrayBlock) {
    initializePercentilesArray(state, percentilesArrayBlock);
    initializeDigest(state);
    QuantileDigest digest = state.getDigest();
    state.addMemoryUsage(-digest.estimatedInMemorySizeInBytes());
    digest.add(value, weight);
    state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
}

Example 2 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateLongPercentileArrayAggregations method initializeDigest.

private static void initializeDigest(@AggregationState DigestAndPercentileArrayState state) {
    QuantileDigest digest = state.getDigest();
    if (digest == null) {
        digest = new QuantileDigest(0.01);
        state.setDigest(digest);
        state.addMemoryUsage(digest.estimatedInMemorySizeInBytes());
    }
}

Example 3 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class DigestAndPercentileArrayStateSerializer method deserialize.

@Override
public void deserialize(Block block, int index, DigestAndPercentileArrayState state) {
    SliceInput input = VARBINARY.getSlice(block, index).getInput();
    // read number of percentiles
    int numPercentiles = input.readInt();
    ImmutableList.Builder<Double> percentilesListBuilder = ImmutableList.builder();
    for (int i = 0; i < numPercentiles; i++) {
        percentilesListBuilder.add(input.readDouble());
    }
    state.setPercentiles(percentilesListBuilder.build());
    // read digest
    int length = input.readInt();
    state.setDigest(new QuantileDigest(input.readSlice(length)));
    state.addMemoryUsage(state.getDigest().estimatedInMemorySizeInBytes());
}

Example 4 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateDoublePercentileAggregations method output.

@OutputFunction(StandardTypes.DOUBLE)
public static void output(@AggregationState DigestAndPercentileState state, BlockBuilder out) {
    QuantileDigest digest = state.getDigest();
    double percentile = state.getPercentile();
    if (digest == null || digest.getCount() == 0.0) {
        out.appendNull();
    } else {
        checkState(percentile != -1.0, "Percentile is missing");
        checkCondition(0 <= percentile && percentile <= 1, INVALID_FUNCTION_ARGUMENT, "Percentile must be between 0 and 1");
        DOUBLE.writeDouble(out, sortableLongToDouble(digest.getQuantile(percentile)));
    }
}

Example 5 with QuantileDigest

use of io.airlift.stats.QuantileDigest in project presto by prestodb.

the class ApproximateDoublePercentileArrayAggregations method output.

@OutputFunction("array(double)")
public static void output(@AggregationState DigestAndPercentileArrayState state, BlockBuilder out) {
    QuantileDigest digest = state.getDigest();
    List<Double> percentiles = state.getPercentiles();
    if (percentiles == null || digest == null) {
        out.appendNull();
        return;
    }
    BlockBuilder blockBuilder = out.beginBlockEntry();
    for (int i = 0; i < percentiles.size(); i++) {
        Double percentile = percentiles.get(i);
        DOUBLE.writeDouble(blockBuilder, sortableLongToDouble(digest.getQuantile(percentile)));
    }
    out.closeEntry();
}