Search in sources :

Example 16 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ExtractionDimFilterTest method testNormal.

@Test
public void testNormal() {
    Filter extractionFilter = new SelectorDimFilter("foo", "extractDimVal", DIM_EXTRACTION_FN).toFilter();
    ImmutableBitmap immutableBitmap = extractionFilter.getBitmapIndex(BITMAP_INDEX_SELECTOR);
    Assert.assertEquals(1, immutableBitmap.size());
}
Also used : SelectorDimFilter(io.druid.query.filter.SelectorDimFilter) ExtractionDimFilter(io.druid.query.filter.ExtractionDimFilter) Filter(io.druid.query.filter.Filter) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) SelectorDimFilter(io.druid.query.filter.SelectorDimFilter) Test(org.junit.Test)

Example 17 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ImmutableRTreeTest method testSearchWithSplit3.

@Test
public void testSearchWithSplit3() {
    BitmapFactory bf = new ConciseBitmapFactory();
    RTree tree = new RTree(2, new LinearGutmanSplitStrategy(0, 50, bf), bf);
    tree.insert(new float[] { 0.0f, 0.0f }, 0);
    tree.insert(new float[] { 1.0f, 3.0f }, 1);
    tree.insert(new float[] { 4.0f, 2.0f }, 2);
    tree.insert(new float[] { 7.0f, 3.0f }, 3);
    tree.insert(new float[] { 8.0f, 6.0f }, 4);
    Random rand = new Random();
    for (int i = 5; i < 5000; i++) {
        tree.insert(new float[] { (float) (rand.nextFloat() * 10 + 10.0), (float) (rand.nextFloat() * 10 + 10.0) }, i);
    }
    ImmutableRTree searchTree = ImmutableRTree.newImmutableFromMutable(tree);
    Iterable<ImmutableBitmap> points = searchTree.search(new RadiusBound(new float[] { 0.0f, 0.0f }, 5));
    ImmutableBitmap finalSet = bf.union(points);
    Assert.assertTrue(finalSet.size() >= 3);
    Set<Integer> expected = Sets.newHashSet(0, 1, 2);
    IntIterator iter = finalSet.iterator();
    while (iter.hasNext()) {
        Assert.assertTrue(expected.contains(iter.next()));
    }
}
Also used : ConciseBitmapFactory(io.druid.collections.bitmap.ConciseBitmapFactory) IntIterator(org.roaringbitmap.IntIterator) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) LinearGutmanSplitStrategy(io.druid.collections.spatial.split.LinearGutmanSplitStrategy) Random(java.util.Random) RadiusBound(io.druid.collections.spatial.search.RadiusBound) ConciseBitmapFactory(io.druid.collections.bitmap.ConciseBitmapFactory) BitmapFactory(io.druid.collections.bitmap.BitmapFactory) RoaringBitmapFactory(io.druid.collections.bitmap.RoaringBitmapFactory) Test(org.junit.Test)

Example 18 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ImmutableRTreeTest method testToAndFromByteBuffer.

@Test
public void testToAndFromByteBuffer() {
    BitmapFactory bf = new ConciseBitmapFactory();
    RTree tree = new RTree(2, new LinearGutmanSplitStrategy(0, 50, bf), bf);
    tree.insert(new float[] { 0, 0 }, 1);
    tree.insert(new float[] { 1, 1 }, 2);
    tree.insert(new float[] { 2, 2 }, 3);
    tree.insert(new float[] { 3, 3 }, 4);
    tree.insert(new float[] { 4, 4 }, 5);
    ImmutableRTree firstTree = ImmutableRTree.newImmutableFromMutable(tree);
    ByteBuffer buffer = ByteBuffer.wrap(firstTree.toBytes());
    ImmutableRTree secondTree = new ImmutableRTree(buffer, bf);
    Iterable<ImmutableBitmap> points = secondTree.search(new RadiusBound(new float[] { 0, 0 }, 10));
    ImmutableBitmap finalSet = bf.union(points);
    Assert.assertTrue(finalSet.size() >= 5);
    Set<Integer> expected = Sets.newHashSet(1, 2, 3, 4, 5);
    IntIterator iter = finalSet.iterator();
    while (iter.hasNext()) {
        Assert.assertTrue(expected.contains(iter.next()));
    }
}
Also used : ConciseBitmapFactory(io.druid.collections.bitmap.ConciseBitmapFactory) IntIterator(org.roaringbitmap.IntIterator) RadiusBound(io.druid.collections.spatial.search.RadiusBound) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) ConciseBitmapFactory(io.druid.collections.bitmap.ConciseBitmapFactory) BitmapFactory(io.druid.collections.bitmap.BitmapFactory) RoaringBitmapFactory(io.druid.collections.bitmap.RoaringBitmapFactory) LinearGutmanSplitStrategy(io.druid.collections.spatial.split.LinearGutmanSplitStrategy) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.Test)

Example 19 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ImmutableRTreeTest method testSearchWithSplit3Roaring.

@Test
public void testSearchWithSplit3Roaring() {
    BitmapFactory bf = new RoaringBitmapFactory();
    RTree tree = new RTree(2, new LinearGutmanSplitStrategy(0, 50, bf), bf);
    tree.insert(new float[] { 0.0f, 0.0f }, 0);
    tree.insert(new float[] { 1.0f, 3.0f }, 1);
    tree.insert(new float[] { 4.0f, 2.0f }, 2);
    tree.insert(new float[] { 7.0f, 3.0f }, 3);
    tree.insert(new float[] { 8.0f, 6.0f }, 4);
    Random rand = new Random();
    for (int i = 5; i < 5000; i++) {
        tree.insert(new float[] { (float) (rand.nextFloat() * 10 + 10.0), (float) (rand.nextFloat() * 10 + 10.0) }, i);
    }
    ImmutableRTree searchTree = ImmutableRTree.newImmutableFromMutable(tree);
    Iterable<ImmutableBitmap> points = searchTree.search(new RadiusBound(new float[] { 0.0f, 0.0f }, 5));
    ImmutableBitmap finalSet = bf.union(points);
    Assert.assertTrue(finalSet.size() >= 3);
    Set<Integer> expected = Sets.newHashSet(0, 1, 2);
    IntIterator iter = finalSet.iterator();
    while (iter.hasNext()) {
        Assert.assertTrue(expected.contains(iter.next()));
    }
}
Also used : IntIterator(org.roaringbitmap.IntIterator) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) LinearGutmanSplitStrategy(io.druid.collections.spatial.split.LinearGutmanSplitStrategy) Random(java.util.Random) RadiusBound(io.druid.collections.spatial.search.RadiusBound) ConciseBitmapFactory(io.druid.collections.bitmap.ConciseBitmapFactory) BitmapFactory(io.druid.collections.bitmap.BitmapFactory) RoaringBitmapFactory(io.druid.collections.bitmap.RoaringBitmapFactory) RoaringBitmapFactory(io.druid.collections.bitmap.RoaringBitmapFactory) Test(org.junit.Test)

Example 20 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class LikeFilterBenchmark method matchBoundPrefix.

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void matchBoundPrefix(Blackhole blackhole) {
    final ImmutableBitmap bitmapIndex = BOUND_PREFIX.getBitmapIndex(selector);
    blackhole.consume(bitmapIndex);
}
Also used : ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Benchmark(org.openjdk.jmh.annotations.Benchmark) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit)

Aggregations

ImmutableBitmap (io.druid.collections.bitmap.ImmutableBitmap)51 BitmapFactory (io.druid.collections.bitmap.BitmapFactory)23 Test (org.junit.Test)23 RoaringBitmapFactory (io.druid.collections.bitmap.RoaringBitmapFactory)22 ConciseBitmapFactory (io.druid.collections.bitmap.ConciseBitmapFactory)19 LinearGutmanSplitStrategy (io.druid.collections.spatial.split.LinearGutmanSplitStrategy)18 IntIterator (org.roaringbitmap.IntIterator)16 RadiusBound (io.druid.collections.spatial.search.RadiusBound)13 Benchmark (org.openjdk.jmh.annotations.Benchmark)13 Random (java.util.Random)12 BenchmarkMode (org.openjdk.jmh.annotations.BenchmarkMode)12 OutputTimeUnit (org.openjdk.jmh.annotations.OutputTimeUnit)12 MutableBitmap (io.druid.collections.bitmap.MutableBitmap)7 Filter (io.druid.query.filter.Filter)6 BitmapIndexSelector (io.druid.query.filter.BitmapIndexSelector)5 BitmapIndex (io.druid.segment.column.BitmapIndex)5 ImmutableRTree (io.druid.collections.spatial.ImmutableRTree)4 BitmapSerdeFactory (io.druid.segment.data.BitmapSerdeFactory)4 RoaringBitmapSerdeFactory (io.druid.segment.data.RoaringBitmapSerdeFactory)4 Function (com.google.common.base.Function)3