Examples with ArrayOfDoublesSketch org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch used on opensource projects

Example 11 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchToQuantilesSketchPostAggregator method compute.

@Override
public DoublesSketch compute(final Map<String, Object> combinedAggregators) {
    final ArrayOfDoublesSketch sketch = (ArrayOfDoublesSketch) getField().compute(combinedAggregators);
    final UpdateDoublesSketch qs = DoublesSketch.builder().setK(k).build();
    final ArrayOfDoublesSketchIterator it = sketch.iterator();
    while (it.next()) {
        // convert 1-based column number to zero-based index
        qs.update(it.getValues()[column - 1]);
    }
    return qs;
}

Example 12 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchSetOpPostAggregatorTest method testComparator.

@Test
public void testComparator() {
    ArrayOfDoublesUpdatableSketch s1 = new ArrayOfDoublesUpdatableSketchBuilder().setNominalEntries(16).setNumberOfValues(2).build();
    s1.update("foo", new double[] { 1.0, 2.0 });
    ArrayOfDoublesUpdatableSketch s2 = new ArrayOfDoublesUpdatableSketchBuilder().setNominalEntries(16).setNumberOfValues(2).build();
    s2.update("foo", new double[] { 2.0, 2.0 });
    s2.update("bar", new double[] { 3.0, 4.0 });
    // duplicate
    ArrayOfDoublesUpdatableSketch s3 = new ArrayOfDoublesUpdatableSketchBuilder().setNominalEntries(16).setNumberOfValues(2).build();
    s3.update("foo", new double[] { 1.0, 2.0 });
    ArrayOfDoublesUpdatableSketch s4 = new ArrayOfDoublesUpdatableSketchBuilder().setNominalEntries(16).setNumberOfValues(2).build();
    s4.update("foo", new double[] { 2.0, 2.0 });
    s4.update("bar", new double[] { 3.0, 4.0 });
    PostAggregator field1 = EasyMock.createMock(PostAggregator.class);
    EasyMock.expect(field1.compute(EasyMock.anyObject(Map.class))).andReturn(s1).anyTimes();
    PostAggregator field2 = EasyMock.createMock(PostAggregator.class);
    EasyMock.expect(field2.compute(EasyMock.anyObject(Map.class))).andReturn(s2).anyTimes();
    PostAggregator field3 = EasyMock.createMock(PostAggregator.class);
    EasyMock.expect(field3.compute(EasyMock.anyObject(Map.class))).andReturn(s3).anyTimes();
    PostAggregator field4 = EasyMock.createMock(PostAggregator.class);
    EasyMock.expect(field4.compute(EasyMock.anyObject(Map.class))).andReturn(s4).anyTimes();
    EasyMock.replay(field1, field2, field3, field4);
    final ArrayOfDoublesSketchSetOpPostAggregator postAgg1 = new ArrayOfDoublesSketchSetOpPostAggregator("a", "UNION", 16, 2, ImmutableList.of(field1, field2));
    final ArrayOfDoublesSketchSetOpPostAggregator postAgg2 = new ArrayOfDoublesSketchSetOpPostAggregator("a", "UNION", 16, 2, ImmutableList.of(field3, field4));
    Comparator comparator = postAgg1.getComparator();
    ArrayOfDoublesSketch sketch1 = postAgg1.compute(ImmutableMap.of());
    ArrayOfDoublesSketch sketch2 = postAgg2.compute(ImmutableMap.of());
    // comparator compares value of each sketches estimate so should be identical
    Assert.assertEquals(0, comparator.compare(sketch1, sketch2));
    Assert.assertEquals(0, Double.compare(sketch1.getEstimate(), sketch2.getEstimate()));
}

Example 13 with ArrayOfDoublesSketch

use of org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch in project druid by druid-io.

the class ArrayOfDoublesSketchMergeBufferAggregator method aggregate.

@Override
public void aggregate(final ByteBuffer buf, final int position) {
    final ArrayOfDoublesSketch update = selector.getObject();
    if (update == null) {
        return;
    }
    // Wrapping memory and ArrayOfDoublesUnion is inexpensive compared to union operations.
    // Maintaining a cache of wrapped objects per buffer position like in Theta sketch aggregator
    // might be considered, but it would increase complexity including relocate() support.
    final WritableMemory mem = WritableMemory.writableWrap(buf, ByteOrder.LITTLE_ENDIAN);
    final WritableMemory region = mem.writableRegion(position, maxIntermediateSize);
    final ArrayOfDoublesUnion union = ArrayOfDoublesSketches.wrapUnion(region);
    union.union(update);
}