Search in sources :

Example 6 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class BitmapOffset method getReverseBitmapOffsetIterator.

public static IntIterator getReverseBitmapOffsetIterator(ImmutableBitmap bitmapIndex) {
    ImmutableBitmap roaringBitmap = bitmapIndex;
    if (!(bitmapIndex instanceof WrappedImmutableRoaringBitmap)) {
        final MutableBitmap bitmap = ROARING_BITMAP_FACTORY.makeEmptyMutableBitmap();
        final IntIterator iterator = bitmapIndex.iterator();
        while (iterator.hasNext()) {
            bitmap.add(iterator.next());
        }
        roaringBitmap = ROARING_BITMAP_FACTORY.makeImmutableBitmap(bitmap);
    }
    return ((WrappedImmutableRoaringBitmap) roaringBitmap).getBitmap().getReverseIntIterator();
}
Also used : EmptyIntIterator(io.druid.extendedset.intset.EmptyIntIterator) IntIterator(org.roaringbitmap.IntIterator) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) MutableBitmap(io.druid.collections.bitmap.MutableBitmap) WrappedImmutableRoaringBitmap(io.druid.collections.bitmap.WrappedImmutableRoaringBitmap)

Example 7 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class OrFilter method makeMatcher.

@Override
public ValueMatcher makeMatcher(BitmapIndexSelector selector, ColumnSelectorFactory columnSelectorFactory, RowOffsetMatcherFactory rowOffsetMatcherFactory) {
    final List<ValueMatcher> matchers = new ArrayList<>();
    final List<ImmutableBitmap> bitmaps = new ArrayList<>();
    for (Filter filter : filters) {
        if (filter.supportsBitmapIndex(selector)) {
            bitmaps.add(filter.getBitmapIndex(selector));
        } else {
            ValueMatcher matcher = filter.makeMatcher(columnSelectorFactory);
            matchers.add(matcher);
        }
    }
    if (bitmaps.size() > 0) {
        ImmutableBitmap combinedBitmap = selector.getBitmapFactory().union(bitmaps);
        ValueMatcher offsetMatcher = rowOffsetMatcherFactory.makeRowOffsetMatcher(combinedBitmap);
        matchers.add(0, offsetMatcher);
    }
    return new ValueMatcher() {

        @Override
        public boolean matches() {
            for (ValueMatcher valueMatcher : matchers) {
                if (valueMatcher.matches()) {
                    return true;
                }
            }
            return false;
        }
    };
}
Also used : ValueMatcher(io.druid.query.filter.ValueMatcher) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) BooleanFilter(io.druid.query.filter.BooleanFilter) Filter(io.druid.query.filter.Filter) ArrayList(java.util.ArrayList)

Example 8 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class AndFilter method getBitmapIndex.

public static ImmutableBitmap getBitmapIndex(BitmapIndexSelector selector, List<Filter> filters) {
    if (filters.size() == 1) {
        return filters.get(0).getBitmapIndex(selector);
    }
    final List<ImmutableBitmap> bitmaps = Lists.newArrayListWithCapacity(filters.size());
    for (final Filter filter : filters) {
        Preconditions.checkArgument(filter.supportsBitmapIndex(selector), "Filter[%s] does not support bitmap index", filter);
        final ImmutableBitmap bitmapIndex = filter.getBitmapIndex(selector);
        if (bitmapIndex.isEmpty()) {
            // Short-circuit.
            return Filters.allFalse(selector);
        }
        bitmaps.add(bitmapIndex);
    }
    return selector.getBitmapFactory().intersection(bitmaps);
}
Also used : ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) BooleanFilter(io.druid.query.filter.BooleanFilter) Filter(io.druid.query.filter.Filter)

Example 9 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class AndFilter method makeMatcher.

@Override
public ValueMatcher makeMatcher(BitmapIndexSelector selector, ColumnSelectorFactory columnSelectorFactory, RowOffsetMatcherFactory rowOffsetMatcherFactory) {
    final List<ValueMatcher> matchers = new ArrayList<>();
    final List<ImmutableBitmap> bitmaps = new ArrayList<>();
    for (Filter filter : filters) {
        if (filter.supportsBitmapIndex(selector)) {
            bitmaps.add(filter.getBitmapIndex(selector));
        } else {
            ValueMatcher matcher = filter.makeMatcher(columnSelectorFactory);
            matchers.add(matcher);
        }
    }
    if (bitmaps.size() > 0) {
        ImmutableBitmap combinedBitmap = selector.getBitmapFactory().intersection(bitmaps);
        ValueMatcher offsetMatcher = rowOffsetMatcherFactory.makeRowOffsetMatcher(combinedBitmap);
        matchers.add(0, offsetMatcher);
    }
    return new ValueMatcher() {

        @Override
        public boolean matches() {
            for (ValueMatcher valueMatcher : matchers) {
                if (!valueMatcher.matches()) {
                    return false;
                }
            }
            return true;
        }
    };
}
Also used : ValueMatcher(io.druid.query.filter.ValueMatcher) ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) BooleanFilter(io.druid.query.filter.BooleanFilter) Filter(io.druid.query.filter.Filter) ArrayList(java.util.ArrayList)

Example 10 with ImmutableBitmap

use of io.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class LikeFilterBenchmark method matchRegexPrefix.

@Benchmark
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
public void matchRegexPrefix(Blackhole blackhole) {
    final ImmutableBitmap bitmapIndex = REGEX_PREFIX.getBitmapIndex(selector);
    blackhole.consume(bitmapIndex);
}
Also used : ImmutableBitmap(io.druid.collections.bitmap.ImmutableBitmap) BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Benchmark(org.openjdk.jmh.annotations.Benchmark) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit)

Aggregations

ImmutableBitmap (io.druid.collections.bitmap.ImmutableBitmap)51 BitmapFactory (io.druid.collections.bitmap.BitmapFactory)23 Test (org.junit.Test)23 RoaringBitmapFactory (io.druid.collections.bitmap.RoaringBitmapFactory)22 ConciseBitmapFactory (io.druid.collections.bitmap.ConciseBitmapFactory)19 LinearGutmanSplitStrategy (io.druid.collections.spatial.split.LinearGutmanSplitStrategy)18 IntIterator (org.roaringbitmap.IntIterator)16 RadiusBound (io.druid.collections.spatial.search.RadiusBound)13 Benchmark (org.openjdk.jmh.annotations.Benchmark)13 Random (java.util.Random)12 BenchmarkMode (org.openjdk.jmh.annotations.BenchmarkMode)12 OutputTimeUnit (org.openjdk.jmh.annotations.OutputTimeUnit)12 MutableBitmap (io.druid.collections.bitmap.MutableBitmap)7 Filter (io.druid.query.filter.Filter)6 BitmapIndexSelector (io.druid.query.filter.BitmapIndexSelector)5 BitmapIndex (io.druid.segment.column.BitmapIndex)5 ImmutableRTree (io.druid.collections.spatial.ImmutableRTree)4 BitmapSerdeFactory (io.druid.segment.data.BitmapSerdeFactory)4 RoaringBitmapSerdeFactory (io.druid.segment.data.RoaringBitmapSerdeFactory)4 Function (com.google.common.base.Function)3