Search in sources :

Example 1 with Union

use of org.apache.datasketches.hll.Union in project druid by druid-io.

the class HllSketchAggregatorFactory method makeAggregateCombiner.

@Override
public AggregateCombiner makeAggregateCombiner() {
    return new ObjectAggregateCombiner<HllSketch>() {

        private final Union union = new Union(lgK);

        @Override
        public void reset(final ColumnValueSelector selector) {
            union.reset();
            fold(selector);
        }

        @Override
        public void fold(final ColumnValueSelector selector) {
            final HllSketch sketch = (HllSketch) selector.getObject();
            union.update(sketch);
        }

        @Nullable
        @Override
        public HllSketch getObject() {
            return union.getResult(tgtHllType);
        }

        @Override
        public Class<HllSketch> classOfObject() {
            return HllSketch.class;
        }
    };
}
Also used : HllSketch(org.apache.datasketches.hll.HllSketch) ObjectAggregateCombiner(org.apache.druid.query.aggregation.ObjectAggregateCombiner) Union(org.apache.datasketches.hll.Union) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector)

Example 2 with Union

use of org.apache.datasketches.hll.Union in project druid by druid-io.

the class HllSketchAggregatorFactory method combine.

@Override
public HllSketch combine(final Object objectA, final Object objectB) {
    final Union union = new Union(lgK);
    union.update((HllSketch) objectA);
    union.update((HllSketch) objectB);
    return union.getResult(tgtHllType);
}
Also used : Union(org.apache.datasketches.hll.Union)

Example 3 with Union

use of org.apache.datasketches.hll.Union in project druid by druid-io.

the class HllSketchMergeBufferAggregatorHelper method get.

/**
 * Helper for implementing {@link org.apache.druid.query.aggregation.BufferAggregator#get} and
 * {@link org.apache.druid.query.aggregation.VectorAggregator#get}.
 */
public Object get(ByteBuffer buf, int position) {
    final WritableMemory mem = WritableMemory.writableWrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, size);
    final Union union = Union.writableWrap(mem);
    return union.getResult(tgtHllType);
}
Also used : WritableMemory(org.apache.datasketches.memory.WritableMemory) Union(org.apache.datasketches.hll.Union)

Example 4 with Union

use of org.apache.datasketches.hll.Union in project druid by druid-io.

the class HllSketchMergeBufferAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int position) {
    final HllSketch sketch = selector.getObject();
    if (sketch == null) {
        return;
    }
    final WritableMemory mem = WritableMemory.writableWrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, helper.getSize());
    final Union union = Union.writableWrap(mem);
    union.update(sketch);
}
Also used : HllSketch(org.apache.datasketches.hll.HllSketch) WritableMemory(org.apache.datasketches.memory.WritableMemory) Union(org.apache.datasketches.hll.Union)

Example 5 with Union

use of org.apache.datasketches.hll.Union in project druid by druid-io.

the class HllSketchMergeVectorAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int numRows, final int[] positions, @Nullable final int[] rows, final int positionOffset) {
    final Object[] vector = objectSupplier.get();
    for (int i = 0; i < numRows; i++) {
        final HllSketch o = (HllSketch) vector[rows != null ? rows[i] : i];
        if (o != null) {
            final int position = positions[i] + positionOffset;
            final WritableMemory mem = WritableMemory.writableWrap(buf, ByteOrder.LITTLE_ENDIAN).writableRegion(position, helper.getSize());
            final Union union = Union.writableWrap(mem);
            union.update(o);
        }
    }
}
Also used : HllSketch(org.apache.datasketches.hll.HllSketch) WritableMemory(org.apache.datasketches.memory.WritableMemory) Union(org.apache.datasketches.hll.Union)

Aggregations

Union (org.apache.datasketches.hll.Union)9 HllSketch (org.apache.datasketches.hll.HllSketch)5 WritableMemory (org.apache.datasketches.memory.WritableMemory)4 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 ISE (org.apache.druid.java.util.common.ISE)2 Interval (org.joda.time.Interval)2 JsonCreator (com.fasterxml.jackson.annotation.JsonCreator)1 JsonProperty (com.fasterxml.jackson.annotation.JsonProperty)1 SmileMediaTypes (com.fasterxml.jackson.jaxrs.smile.SmileMediaTypes)1 Preconditions (com.google.common.base.Preconditions)1 Throwables (com.google.common.base.Throwables)1 ArrayListMultimap (com.google.common.collect.ArrayListMultimap)1 ImmutableMap (com.google.common.collect.ImmutableMap)1 Multimap (com.google.common.collect.Multimap)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1 Comparator (java.util.Comparator)1 HashMap (java.util.HashMap)1