Examples with DoublesSketch org.apache.datasketches.quantiles.DoublesSketch used on opensource projects

Example 1 with DoublesSketch

use of org.apache.datasketches.quantiles.DoublesSketch in project druid by druid-io.

the class DoublesSketchAggregatorFactory method makeAggregateCombiner.

@Override
public AggregateCombiner makeAggregateCombiner() {
    return new ObjectAggregateCombiner<DoublesSketch>() {

        private final DoublesUnion union = DoublesUnion.builder().setMaxK(k).build();

        @Override
        public void reset(final ColumnValueSelector selector) {
            union.reset();
            fold(selector);
        }

        @Override
        public void fold(final ColumnValueSelector selector) {
            final DoublesSketch sketch = (DoublesSketch) selector.getObject();
            union.update(sketch);
        }

        @Nullable
        @Override
        public DoublesSketch getObject() {
            return union.getResult();
        }

        @Override
        public Class<DoublesSketch> classOfObject() {
            return DoublesSketch.class;
        }
    };
}

Example 2 with DoublesSketch

use of org.apache.datasketches.quantiles.DoublesSketch in project druid by druid-io.

the class DoublesSketchComplexMetricSerde method deserializeColumn.

@Override
public void deserializeColumn(final ByteBuffer buffer, final ColumnBuilder builder) {
    final GenericIndexed<DoublesSketch> column = GenericIndexed.read(buffer, STRATEGY, builder.getFileMapper());
    builder.setComplexColumnSupplier(new ComplexColumnPartSupplier(getTypeName(), column));
}

Example 3 with DoublesSketch

use of org.apache.datasketches.quantiles.DoublesSketch in project druid by druid-io.

the class DoublesSketchMergeVectorAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int numRows, final int[] positions, @Nullable final int[] rows, final int positionOffset) {
    final Object[] vector = selector.getObjectVector();
    DoublesSketches.handleMaxStreamLengthLimit(() -> {
        for (int i = 0; i < numRows; i++) {
            final DoublesSketch sketch = (DoublesSketch) vector[rows != null ? rows[i] : i];
            if (sketch != null) {
                final int position = positions[i] + positionOffset;
                final DoublesUnion union = helper.getSketchAtPosition(buf, position);
                union.update(sketch);
            }
        }
    });
}

Example 4 with DoublesSketch

use of org.apache.datasketches.quantiles.DoublesSketch in project druid by druid-io.

the class DoublesSketchToHistogramPostAggregator method compute.

@Override
public Object compute(final Map<String, Object> combinedAggregators) {
    final DoublesSketch sketch = (DoublesSketch) field.compute(combinedAggregators);
    final int numBins = splitPoints != null ? splitPoints.length + 1 : (this.numBins != null ? this.numBins.intValue() : DEFAULT_NUM_BINS);
    if (numBins < 2) {
        throw new IAE("at least 2 bins expected");
    }
    if (sketch.isEmpty()) {
        final double[] histogram = new double[numBins];
        Arrays.fill(histogram, Double.NaN);
        return histogram;
    }
    final double[] histogram = sketch.getPMF(splitPoints != null ? splitPoints : equallySpacedPoints(numBins, sketch.getMinValue(), sketch.getMaxValue()));
    for (int i = 0; i < histogram.length; i++) {
        // scale fractions to counts
        histogram[i] *= sketch.getN();
    }
    return histogram;
}

Example 5 with DoublesSketch

use of org.apache.datasketches.quantiles.DoublesSketch in project druid by druid-io.

the class DoublesSketchToQuantilesPostAggregator method compute.

@Override
public Object compute(final Map<String, Object> combinedAggregators) {
    final DoublesSketch sketch = (DoublesSketch) field.compute(combinedAggregators);
    if (sketch.isEmpty()) {
        final double[] quantiles = new double[fractions.length];
        Arrays.fill(quantiles, Double.NaN);
        return quantiles;
    }
    return sketch.getQuantiles(fractions);
}