Examples with UpdateDoublesSketch org.apache.datasketches.quantiles.UpdateDoublesSketch used on opensource projects

Example 1 with UpdateDoublesSketch

use of org.apache.datasketches.quantiles.UpdateDoublesSketch in project druid by druid-io.

the class DoublesSketchBuildBufferAggregatorHelper method init.

public void init(final ByteBuffer buffer, final int position) {
    final WritableMemory mem = getMemory(buffer);
    final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
    final UpdateDoublesSketch sketch = DoublesSketch.builder().setK(size).build(region);
    putSketch(buffer, position, sketch);
}

Example 2 with UpdateDoublesSketch

use of org.apache.datasketches.quantiles.UpdateDoublesSketch in project druid by druid-io.

the class DoublesSketchBuildBufferAggregatorHelper method relocate.

// A small number of sketches may run out of the given memory, request more memory on heap and move there.
// In that case we need to reuse the object from the cache as opposed to wrapping the new buffer.
public void relocate(int oldPosition, int newPosition, ByteBuffer oldBuffer, ByteBuffer newBuffer) {
    UpdateDoublesSketch sketch = sketches.get(oldBuffer).get(oldPosition);
    final WritableMemory oldRegion = getMemory(oldBuffer).writableRegion(oldPosition, maxIntermediateSize);
    if (sketch.isSameResource(oldRegion)) {
        // sketch was not relocated on heap
        final WritableMemory newRegion = getMemory(newBuffer).writableRegion(newPosition, maxIntermediateSize);
        sketch = UpdateDoublesSketch.wrap(newRegion);
    }
    putSketch(newBuffer, newPosition, sketch);
    final Int2ObjectMap<UpdateDoublesSketch> map = sketches.get(oldBuffer);
    map.remove(oldPosition);
    if (map.isEmpty()) {
        sketches.remove(oldBuffer);
        memCache.remove(oldBuffer);
    }
}

Example 3 with UpdateDoublesSketch

use of org.apache.datasketches.quantiles.UpdateDoublesSketch in project druid by druid-io.

the class DoublesSketchBuildBufferAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buffer, final int position) {
    if (selector.isNull()) {
        return;
    }
    final UpdateDoublesSketch sketch = helper.getSketchAtPosition(buffer, position);
    DoublesSketches.handleMaxStreamLengthLimit(() -> sketch.update(selector.getDouble()));
}

Example 4 with UpdateDoublesSketch

use of org.apache.datasketches.quantiles.UpdateDoublesSketch in project druid by druid-io.

the class DoublesSketchBuildVectorAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int position, final int startRow, final int endRow) {
    final double[] doubles = selector.getDoubleVector();
    final boolean[] nulls = selector.getNullVector();
    final UpdateDoublesSketch sketch = helper.getSketchAtPosition(buf, position);
    DoublesSketches.handleMaxStreamLengthLimit(() -> {
        for (int i = startRow; i < endRow; i++) {
            if (nulls == null || !nulls[i]) {
                sketch.update(doubles[i]);
            }
        }
    });
}

Example 5 with UpdateDoublesSketch

use of org.apache.datasketches.quantiles.UpdateDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchToQuantilesSketchPostAggregator method compute.

@Override
public DoublesSketch compute(final Map<String, Object> combinedAggregators) {
    final ArrayOfDoublesSketch sketch = (ArrayOfDoublesSketch) getField().compute(combinedAggregators);
    final UpdateDoublesSketch qs = DoublesSketch.builder().setK(k).build();
    final ArrayOfDoublesSketchIterator it = sketch.iterator();
    while (it.next()) {
        // convert 1-based column number to zero-based index
        qs.update(it.getValues()[column - 1]);
    }
    return qs;
}