Search in sources :

Example 1 with MutableRoaringBitmap

use of org.roaringbitmap.buffer.MutableRoaringBitmap in project druid by druid-io.

the class RangeBitmapBenchmarkTest method prepareRandomRanges.

@BeforeClass
public static void prepareRandomRanges() throws Exception {
    System.setProperty("jub.customkey", String.format("%06.5f", DENSITY));
    reset();
    final BitSet expectedUnion = new BitSet();
    for (int i = 0; i < SIZE; ++i) {
        ConciseSet c = new ConciseSet();
        MutableRoaringBitmap r = new MutableRoaringBitmap();
        {
            int k = 0;
            boolean fill = true;
            while (k < LENGTH) {
                int runLength = (int) (LENGTH * DENSITY) + rand.nextInt((int) (LENGTH * DENSITY));
                for (int j = k; fill && j < LENGTH && j < k + runLength; ++j) {
                    c.add(j);
                    r.add(j);
                    expectedUnion.set(j);
                }
                k += runLength;
                fill = !fill;
            }
        }
        minIntersection = MIN_INTERSECT;
        for (int k = LENGTH / 2; k < LENGTH / 2 + minIntersection; ++k) {
            c.add(k);
            r.add(k);
            expectedUnion.set(k);
        }
        concise[i] = ImmutableConciseSet.newImmutableFromMutable(c);
        offheapConcise[i] = makeOffheapConcise(concise[i]);
        roaring[i] = r;
        immutableRoaring[i] = makeImmutableRoaring(r);
        offheapRoaring[i] = makeOffheapRoaring(r);
        genericConcise[i] = new WrappedImmutableConciseBitmap(offheapConcise[i]);
        genericRoaring[i] = new WrappedImmutableRoaringBitmap(offheapRoaring[i]);
    }
    unionCount = expectedUnion.cardinality();
    printSizeStats(DENSITY, "Random Alternating Bitmap");
}
Also used : MutableRoaringBitmap(org.roaringbitmap.buffer.MutableRoaringBitmap) ConciseSet(io.druid.extendedset.intset.ConciseSet) ImmutableConciseSet(io.druid.extendedset.intset.ImmutableConciseSet) BitSet(java.util.BitSet) BeforeClass(org.junit.BeforeClass)

Example 2 with MutableRoaringBitmap

use of org.roaringbitmap.buffer.MutableRoaringBitmap in project druid by druid-io.

the class UniformBitmapBenchmarkTest method prepareMostlyUniform.

@BeforeClass
public static void prepareMostlyUniform() throws Exception {
    System.setProperty("jub.customkey", String.format("%05.4f", DENSITY));
    reset();
    final BitSet expectedUnion = new BitSet();
    final int[] knownTrue = new int[MIN_INTERSECT];
    for (int i = 0; i < knownTrue.length; ++i) {
        knownTrue[i] = rand.nextInt(LENGTH);
    }
    for (int i = 0; i < SIZE; ++i) {
        ConciseSet c = new ConciseSet();
        MutableRoaringBitmap r = new MutableRoaringBitmap();
        for (int k = 0; k < LENGTH; ++k) {
            if (rand.nextDouble() < DENSITY) {
                c.add(k);
                r.add(k);
                expectedUnion.set(k);
            }
        }
        for (int k : knownTrue) {
            c.add(k);
            r.add(k);
            expectedUnion.set(k);
        }
        concise[i] = ImmutableConciseSet.newImmutableFromMutable(c);
        offheapConcise[i] = makeOffheapConcise(concise[i]);
        roaring[i] = r;
        immutableRoaring[i] = makeImmutableRoaring(r);
        offheapRoaring[i] = makeOffheapRoaring(r);
        genericConcise[i] = new WrappedImmutableConciseBitmap(offheapConcise[i]);
        genericRoaring[i] = new WrappedImmutableRoaringBitmap(offheapRoaring[i]);
    }
    unionCount = expectedUnion.cardinality();
    minIntersection = knownTrue.length;
    printSizeStats(DENSITY, "Uniform Bitmap");
}
Also used : MutableRoaringBitmap(org.roaringbitmap.buffer.MutableRoaringBitmap) ConciseSet(io.druid.extendedset.intset.ConciseSet) ImmutableConciseSet(io.druid.extendedset.intset.ImmutableConciseSet) BitSet(java.util.BitSet) BeforeClass(org.junit.BeforeClass)

Example 3 with MutableRoaringBitmap

use of org.roaringbitmap.buffer.MutableRoaringBitmap in project druid by druid-io.

the class WrappedRoaringBitmap method or.

@Override
public void or(MutableBitmap mutableBitmap) {
    WrappedRoaringBitmap other = (WrappedRoaringBitmap) mutableBitmap;
    MutableRoaringBitmap unwrappedOtherBitmap = other.bitmap;
    bitmap.or(unwrappedOtherBitmap);
}
Also used : MutableRoaringBitmap(org.roaringbitmap.buffer.MutableRoaringBitmap)

Example 4 with MutableRoaringBitmap

use of org.roaringbitmap.buffer.MutableRoaringBitmap in project druid by druid-io.

the class WrappedRoaringBitmap method intersection.

@Override
public ImmutableBitmap intersection(ImmutableBitmap otherBitmap) {
    WrappedRoaringBitmap other = (WrappedRoaringBitmap) otherBitmap;
    MutableRoaringBitmap unwrappedOtherBitmap = other.bitmap;
    return new WrappedImmutableRoaringBitmap(MutableRoaringBitmap.and(bitmap, unwrappedOtherBitmap));
}
Also used : MutableRoaringBitmap(org.roaringbitmap.buffer.MutableRoaringBitmap)

Example 5 with MutableRoaringBitmap

use of org.roaringbitmap.buffer.MutableRoaringBitmap in project druid by druid-io.

the class WrappedRoaringBitmap method xor.

@Override
public void xor(MutableBitmap mutableBitmap) {
    WrappedRoaringBitmap other = (WrappedRoaringBitmap) mutableBitmap;
    MutableRoaringBitmap unwrappedOtherBitmap = other.bitmap;
    bitmap.xor(unwrappedOtherBitmap);
}
Also used : MutableRoaringBitmap(org.roaringbitmap.buffer.MutableRoaringBitmap)

Aggregations

MutableRoaringBitmap (org.roaringbitmap.buffer.MutableRoaringBitmap)24 BlockDocIdIterator (com.linkedin.pinot.core.common.BlockDocIdIterator)5 ArrayList (java.util.ArrayList)5 ImmutableRoaringBitmap (org.roaringbitmap.buffer.ImmutableRoaringBitmap)5 DataOutputStream (java.io.DataOutputStream)4 BlockDocIdSet (com.linkedin.pinot.core.common.BlockDocIdSet)3 BitmapDocIdIterator (com.linkedin.pinot.core.operator.dociditerators.BitmapDocIdIterator)3 IntIterator (org.roaringbitmap.IntIterator)3 IntPair (com.linkedin.pinot.common.utils.Pairs.IntPair)2 AndDocIdIterator (com.linkedin.pinot.core.operator.dociditerators.AndDocIdIterator)2 RangelessBitmapDocIdIterator (com.linkedin.pinot.core.operator.dociditerators.RangelessBitmapDocIdIterator)2 ConciseSet (io.druid.extendedset.intset.ConciseSet)2 ImmutableConciseSet (io.druid.extendedset.intset.ImmutableConciseSet)2 BufferedOutputStream (java.io.BufferedOutputStream)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 File (java.io.File)2 FileOutputStream (java.io.FileOutputStream)2 BitSet (java.util.BitSet)2 BeforeClass (org.junit.BeforeClass)2 Test (org.testng.annotations.Test)2