Search in sources :

Example 46 with UpdateSketch

use of org.apache.datasketches.theta.UpdateSketch in project druid by druid-io.

the class BufferHashGrouperUsingSketchMergeAggregatorFactoryTest method testGrowingBufferGrouper.

@Test
public void testGrowingBufferGrouper() {
    final TestColumnSelectorFactory columnSelectorFactory = GrouperTestUtil.newColumnSelectorFactory();
    final Grouper<Integer> grouper = makeGrouper(columnSelectorFactory, 100000, 2);
    try {
        final int expectedMaxSize = 5;
        SketchHolder sketchHolder = SketchHolder.of(Sketches.updateSketchBuilder().setNominalEntries(16).build());
        UpdateSketch updateSketch = (UpdateSketch) sketchHolder.getSketch();
        updateSketch.update(1);
        columnSelectorFactory.setRow(new MapBasedRow(0, ImmutableMap.of("sketch", sketchHolder)));
        for (int i = 0; i < expectedMaxSize; i++) {
            Assert.assertTrue(String.valueOf(i), grouper.aggregate(i).isOk());
        }
        updateSketch.update(3);
        columnSelectorFactory.setRow(new MapBasedRow(0, ImmutableMap.of("sketch", sketchHolder)));
        for (int i = 0; i < expectedMaxSize; i++) {
            Assert.assertTrue(String.valueOf(i), grouper.aggregate(i).isOk());
        }
        Object[] holders = Lists.newArrayList(grouper.iterator(true)).get(0).getValues();
        Assert.assertEquals(2.0d, ((SketchHolder) holders[0]).getEstimate(), 0);
    } finally {
        grouper.close();
    }
}
Also used : MapBasedRow(org.apache.druid.data.input.MapBasedRow) TestColumnSelectorFactory(org.apache.druid.query.groupby.epinephelinae.TestColumnSelectorFactory) UpdateSketch(org.apache.datasketches.theta.UpdateSketch) Test(org.junit.Test)

Aggregations

UpdateSketch (org.apache.datasketches.theta.UpdateSketch)46 Test (org.testng.annotations.Test)42 DoubleSummary (org.apache.datasketches.tuple.adouble.DoubleSummary)12 AnotB (org.apache.datasketches.tuple.AnotB)6 JaccardSimilarity.dissimilarityTest (org.apache.datasketches.tuple.JaccardSimilarity.dissimilarityTest)6 JaccardSimilarity.similarityTest (org.apache.datasketches.tuple.JaccardSimilarity.similarityTest)6 UpdateSketchBuilder (org.apache.datasketches.theta.UpdateSketchBuilder)5 Intersection (org.apache.datasketches.tuple.Intersection)4 MapBasedRow (org.apache.druid.data.input.MapBasedRow)3 TestColumnSelectorFactory (org.apache.druid.query.groupby.epinephelinae.TestColumnSelectorFactory)3 Test (org.junit.Test)3 SketchesArgumentException (org.apache.datasketches.SketchesArgumentException)2 IntegerSummary (org.apache.datasketches.tuple.aninteger.IntegerSummary)2 GroupByQueryRunnerTest (org.apache.druid.query.groupby.GroupByQueryRunnerTest)2 SketchesStateException (org.apache.datasketches.SketchesStateException)1 CompactSketch (org.apache.datasketches.theta.CompactSketch)1 Intersection (org.apache.datasketches.theta.Intersection)1 Union (org.apache.datasketches.tuple.Union)1 SketchHolder (org.apache.druid.query.aggregation.datasketches.theta.SketchHolder)1 InitializedNullHandlingTest (org.apache.druid.testing.InitializedNullHandlingTest)1