Examples with DoublesUnion org.apache.datasketches.quantiles.DoublesUnion used on opensource projects

Example 1 with DoublesUnion

use of org.apache.datasketches.quantiles.DoublesUnion in project druid by druid-io.

the class DoublesSketchAggregatorFactory method makeAggregateCombiner.

@Override
public AggregateCombiner makeAggregateCombiner() {
    return new ObjectAggregateCombiner<DoublesSketch>() {

        private final DoublesUnion union = DoublesUnion.builder().setMaxK(k).build();

        @Override
        public void reset(final ColumnValueSelector selector) {
            union.reset();
            fold(selector);
        }

        @Override
        public void fold(final ColumnValueSelector selector) {
            final DoublesSketch sketch = (DoublesSketch) selector.getObject();
            union.update(sketch);
        }

        @Nullable
        @Override
        public DoublesSketch getObject() {
            return union.getResult();
        }

        @Override
        public Class<DoublesSketch> classOfObject() {
            return DoublesSketch.class;
        }
    };
}

Example 2 with DoublesUnion

use of org.apache.datasketches.quantiles.DoublesUnion in project druid by druid-io.

the class DoublesSketchAggregatorFactory method combine.

@Override
public Object combine(final Object lhs, final Object rhs) {
    final DoublesUnion union = DoublesUnion.builder().setMaxK(k).build();
    union.update((DoublesSketch) lhs);
    union.update((DoublesSketch) rhs);
    return union.getResultAndReset();
}

Example 3 with DoublesUnion

use of org.apache.datasketches.quantiles.DoublesUnion in project druid by druid-io.

the class DoublesSketchMergeBufferAggregatorHelper method init.

public void init(final ByteBuffer buffer, final int position) {
    final WritableMemory mem = getMemory(buffer);
    final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
    final DoublesUnion union = DoublesUnion.builder().setMaxK(k).build(region);
    putUnion(buffer, position, union);
}

Example 4 with DoublesUnion

use of org.apache.datasketches.quantiles.DoublesUnion in project druid by druid-io.

the class DoublesSketchMergeVectorAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int numRows, final int[] positions, @Nullable final int[] rows, final int positionOffset) {
    final Object[] vector = selector.getObjectVector();
    DoublesSketches.handleMaxStreamLengthLimit(() -> {
        for (int i = 0; i < numRows; i++) {
            final DoublesSketch sketch = (DoublesSketch) vector[rows != null ? rows[i] : i];
            if (sketch != null) {
                final int position = positions[i] + positionOffset;
                final DoublesUnion union = helper.getSketchAtPosition(buf, position);
                union.update(sketch);
            }
        }
    });
}

Example 5 with DoublesUnion

use of org.apache.datasketches.quantiles.DoublesUnion in project druid by druid-io.

the class DoublesSketchMergeBufferAggregatorHelper method relocate.

// A small number of sketches may run out of the given memory, request more memory on heap and move there.
// In that case we need to reuse the object from the cache as opposed to wrapping the new buffer.
public void relocate(int oldPosition, int newPosition, ByteBuffer oldBuffer, ByteBuffer newBuffer) {
    DoublesUnion union = unions.get(oldBuffer).get(oldPosition);
    final WritableMemory oldMem = getMemory(oldBuffer).writableRegion(oldPosition, maxIntermediateSize);
    if (union.isSameResource(oldMem)) {
        // union was not relocated on heap
        final WritableMemory newMem = getMemory(newBuffer).writableRegion(newPosition, maxIntermediateSize);
        union = DoublesUnion.wrap(newMem);
    }
    putUnion(newBuffer, newPosition, union);
    Int2ObjectMap<DoublesUnion> map = unions.get(oldBuffer);
    map.remove(oldPosition);
    if (map.isEmpty()) {
        unions.remove(oldBuffer);
        memCache.remove(oldBuffer);
    }
}