Examples with ArrayOfDoublesSketch org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch used on opensource projects

Example 6 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchMergeComplexMetricSerde method deserializeColumn.

@Override
public void deserializeColumn(final ByteBuffer buffer, final ColumnBuilder builder) {
    final GenericIndexed<ArrayOfDoublesSketch> ge = GenericIndexed.read(buffer, ArrayOfDoublesSketchObjectStrategy.STRATEGY);
    builder.setComplexColumnSupplier(new ComplexColumnPartSupplier(getTypeName(), ge));
}

Example 7 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchToMeansPostAggregator method compute.

@Override
public double[] compute(final Map<String, Object> combinedAggregators) {
    final ArrayOfDoublesSketch sketch = (ArrayOfDoublesSketch) getField().compute(combinedAggregators);
    final SummaryStatistics[] stats = new SummaryStatistics[sketch.getNumValues()];
    Arrays.setAll(stats, i -> new SummaryStatistics());
    final ArrayOfDoublesSketchIterator it = sketch.iterator();
    while (it.next()) {
        final double[] values = it.getValues();
        for (int i = 0; i < values.length; i++) {
            stats[i].addValue(values[i]);
        }
    }
    final double[] means = new double[sketch.getNumValues()];
    Arrays.setAll(means, i -> stats[i].getMean());
    return means;
}

Example 8 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchAggregatorFactoryTest method makeAggregateCombiner.

@Test
public void makeAggregateCombiner() {
    AggregatorFactory aggregatorFactory = new ArrayOfDoublesSketchAggregatorFactory("", "", null, null, null);
    AggregatorFactory combiningFactory = aggregatorFactory.getCombiningFactory();
    AggregateCombiner<ArrayOfDoublesSketch> combiner = combiningFactory.makeAggregateCombiner();
    ArrayOfDoublesUpdatableSketch sketch1 = new ArrayOfDoublesUpdatableSketchBuilder().build();
    sketch1.update("a", new double[] { 1 });
    ArrayOfDoublesUpdatableSketch sketch2 = new ArrayOfDoublesUpdatableSketchBuilder().build();
    sketch2.update("b", new double[] { 1 });
    sketch2.update("c", new double[] { 1 });
    TestObjectColumnSelector<ArrayOfDoublesSketch> selector = new TestObjectColumnSelector<ArrayOfDoublesSketch>(new ArrayOfDoublesSketch[] { sketch1, sketch2 });
    combiner.reset(selector);
    Assert.assertEquals(1, combiner.getObject().getEstimate(), 0);
    selector.increment();
    combiner.fold(selector);
    Assert.assertEquals(3, combiner.getObject().getEstimate(), 0);
}

Example 9 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project sketches-core by DataSketches.

the class ReadOnlyMemoryTest method heapifyAndUpdateUnion.

@Test
public void heapifyAndUpdateUnion() {
    final int numUniques = 10000;
    int key = 0;
    final ArrayOfDoublesUpdatableSketch sketch1 = new ArrayOfDoublesUpdatableSketchBuilder().build();
    for (int i = 0; i < numUniques; i++) {
        sketch1.update(key++, new double[] { 1 });
    }
    final ArrayOfDoublesUnion union1 = new ArrayOfDoublesSetOperationBuilder().buildUnion();
    union1.union(sketch1);
    final ArrayOfDoublesUnion union2 = ArrayOfDoublesSketches.heapifyUnion(Memory.wrap(union1.toByteArray()));
    final ArrayOfDoublesSketch resultSketch = union2.getResult();
    Assert.assertTrue(resultSketch.isEstimationMode());
    Assert.assertEquals(resultSketch.getEstimate(), numUniques, numUniques * 0.04);
    // make sure union update actually needs to modify the union
    final ArrayOfDoublesUpdatableSketch sketch2 = new ArrayOfDoublesUpdatableSketchBuilder().build();
    for (int i = 0; i < numUniques; i++) {
        sketch2.update(key++, new double[] { 1 });
    }
    union2.union(sketch2);
}

Example 10 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchAggregatorFactory method makeAggregateCombiner.

@Override
public AggregateCombiner makeAggregateCombiner() {
    return new ObjectAggregateCombiner<ArrayOfDoublesSketch>() {

        private final ArrayOfDoublesUnion union = new ArrayOfDoublesSetOperationBuilder().setNominalEntries(nominalEntries).setNumberOfValues(numberOfValues).buildUnion();

        @Override
        public void reset(final ColumnValueSelector selector) {
            union.reset();
            fold(selector);
        }

        @Override
        public void fold(final ColumnValueSelector selector) {
            final ArrayOfDoublesSketch sketch = (ArrayOfDoublesSketch) selector.getObject();
            union.union(sketch);
        }

        @Override
        public ArrayOfDoublesSketch getObject() {
            return union.getResult();
        }

        @Override
        public Class<ArrayOfDoublesSketch> classOfObject() {
            return ArrayOfDoublesSketch.class;
        }
    };
}