Search in sources :

Example 1 with ImmutableBitmap

use of org.apache.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class UseIndexesStrategy method getExecutionPlan.

@Override
public List<SearchQueryExecutor> getExecutionPlan(SearchQuery query, Segment segment) {
    final ImmutableList.Builder<SearchQueryExecutor> builder = ImmutableList.builder();
    final QueryableIndex index = segment.asQueryableIndex();
    final StorageAdapter adapter = segment.asStorageAdapter();
    final List<DimensionSpec> searchDims = getDimsToSearch(adapter.getAvailableDimensions(), query.getDimensions());
    if (index != null) {
        // pair of bitmap dims and non-bitmap dims
        final Pair<List<DimensionSpec>, List<DimensionSpec>> pair = partitionDimensionList(adapter, searchDims);
        final List<DimensionSpec> bitmapSuppDims = pair.lhs;
        final List<DimensionSpec> nonBitmapSuppDims = pair.rhs;
        if (bitmapSuppDims.size() > 0) {
            final BitmapIndexSelector selector = new ColumnSelectorBitmapIndexSelector(index.getBitmapFactoryForDimensions(), VirtualColumns.EMPTY, index);
            // from the non-bitmap-support filter, and then use it to compute the filtered result by intersecting bitmaps.
            if (filter == null || filter.supportsBitmapIndex(selector)) {
                final ImmutableBitmap timeFilteredBitmap = makeTimeFilteredBitmap(index, segment, filter, interval);
                builder.add(new IndexOnlyExecutor(query, segment, timeFilteredBitmap, bitmapSuppDims));
            } else {
                // Fall back to cursor-based execution strategy
                nonBitmapSuppDims.addAll(bitmapSuppDims);
            }
        }
        if (nonBitmapSuppDims.size() > 0) {
            builder.add(new CursorBasedExecutor(query, segment, filter, interval, nonBitmapSuppDims));
        }
    } else {
        builder.add(new CursorBasedExecutor(query, segment, filter, interval, searchDims));
    }
    return builder.build();
}
Also used : DimensionSpec(org.apache.druid.query.dimension.DimensionSpec) ImmutableBitmap(org.apache.druid.collections.bitmap.ImmutableBitmap) ImmutableList(com.google.common.collect.ImmutableList) StorageAdapter(org.apache.druid.segment.StorageAdapter) CursorBasedExecutor(org.apache.druid.query.search.CursorOnlyStrategy.CursorBasedExecutor) ColumnSelectorBitmapIndexSelector(org.apache.druid.segment.ColumnSelectorBitmapIndexSelector) QueryableIndex(org.apache.druid.segment.QueryableIndex) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) ColumnSelectorBitmapIndexSelector(org.apache.druid.segment.ColumnSelectorBitmapIndexSelector) BitmapIndexSelector(org.apache.druid.query.filter.BitmapIndexSelector)

Example 2 with ImmutableBitmap

use of org.apache.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class NumericNullColumnSelectorTest method testFloatSelectorWithNullsCanResetOffset.

@Test
public void testFloatSelectorWithNullsCanResetOffset() {
    for (ImmutableBitmap bitmap : bitmaps) {
        ColumnarFloats floats = new ColumnarFloats() {

            @Override
            public int size() {
                return numRows;
            }

            @Override
            public float get(int index) {
                return ThreadLocalRandom.current().nextFloat();
            }

            @Override
            public void close() {
            }
        };
        FloatsColumn columnWithNulls = FloatsColumn.create(floats, bitmap);
        ColumnValueSelector<?> selector = columnWithNulls.makeColumnValueSelector(offset);
        assertOffsetCanReset(selector, bitmap, offset);
        VectorValueSelector vectorSelector = columnWithNulls.makeVectorValueSelector(vectorOffset);
        assertVectorOffsetCanReset(vectorSelector, bitmap, vectorOffset);
        VectorValueSelector anotherSelector = columnWithNulls.makeVectorValueSelector(anotherVectorOffset);
        assertVectorChillWhenOffsetsOverlap(anotherSelector, bitmap, anotherVectorOffset);
    }
}
Also used : FloatsColumn(org.apache.druid.segment.column.FloatsColumn) ImmutableBitmap(org.apache.druid.collections.bitmap.ImmutableBitmap) VectorValueSelector(org.apache.druid.segment.vector.VectorValueSelector) Test(org.junit.Test)

Example 3 with ImmutableBitmap

use of org.apache.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class NumericNullColumnSelectorTest method testDoubleSelectorWithNullsCanResetOffset.

@Test
public void testDoubleSelectorWithNullsCanResetOffset() {
    for (ImmutableBitmap bitmap : bitmaps) {
        ColumnarDoubles doubles = new ColumnarDoubles() {

            @Override
            public int size() {
                return numRows;
            }

            @Override
            public double get(int index) {
                return ThreadLocalRandom.current().nextDouble();
            }

            @Override
            public void close() {
            }
        };
        DoublesColumn columnWithNulls = DoublesColumn.create(doubles, bitmap);
        ColumnValueSelector<?> selector = columnWithNulls.makeColumnValueSelector(offset);
        assertOffsetCanReset(selector, bitmap, offset);
        VectorValueSelector vectorSelector = columnWithNulls.makeVectorValueSelector(vectorOffset);
        assertVectorOffsetCanReset(vectorSelector, bitmap, vectorOffset);
    }
}
Also used : ImmutableBitmap(org.apache.druid.collections.bitmap.ImmutableBitmap) DoublesColumn(org.apache.druid.segment.column.DoublesColumn) VectorValueSelector(org.apache.druid.segment.vector.VectorValueSelector) Test(org.junit.Test)

Example 4 with ImmutableBitmap

use of org.apache.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ExtractionDimFilterTest method testEmpty.

@Test
public void testEmpty() {
    Filter extractionFilter = new SelectorDimFilter("foo", "NFDJUKFNDSJFNS", DIM_EXTRACTION_FN).toFilter();
    ImmutableBitmap immutableBitmap = extractionFilter.getBitmapIndex(BITMAP_INDEX_SELECTOR);
    Assert.assertEquals(0, immutableBitmap.size());
}
Also used : SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) ExtractionDimFilter(org.apache.druid.query.filter.ExtractionDimFilter) Filter(org.apache.druid.query.filter.Filter) ImmutableBitmap(org.apache.druid.collections.bitmap.ImmutableBitmap) SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) Test(org.junit.Test)

Example 5 with ImmutableBitmap

use of org.apache.druid.collections.bitmap.ImmutableBitmap in project druid by druid-io.

the class ExtractionDimFilterTest method testNull.

@Test
public void testNull() {
    Filter extractionFilter = new SelectorDimFilter("FDHJSFFHDS", "extractDimVal", DIM_EXTRACTION_FN).toFilter();
    ImmutableBitmap immutableBitmap = extractionFilter.getBitmapIndex(BITMAP_INDEX_SELECTOR);
    Assert.assertEquals(0, immutableBitmap.size());
}
Also used : SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) ExtractionDimFilter(org.apache.druid.query.filter.ExtractionDimFilter) Filter(org.apache.druid.query.filter.Filter) ImmutableBitmap(org.apache.druid.collections.bitmap.ImmutableBitmap) SelectorDimFilter(org.apache.druid.query.filter.SelectorDimFilter) Test(org.junit.Test)

Aggregations

ImmutableBitmap (org.apache.druid.collections.bitmap.ImmutableBitmap)64 Test (org.junit.Test)37 BitmapFactory (org.apache.druid.collections.bitmap.BitmapFactory)24 RoaringBitmapFactory (org.apache.druid.collections.bitmap.RoaringBitmapFactory)22 ConciseBitmapFactory (org.apache.druid.collections.bitmap.ConciseBitmapFactory)19 LinearGutmanSplitStrategy (org.apache.druid.collections.spatial.split.LinearGutmanSplitStrategy)18 IntIterator (org.roaringbitmap.IntIterator)17 Benchmark (org.openjdk.jmh.annotations.Benchmark)14 RadiusBound (org.apache.druid.collections.spatial.search.RadiusBound)13 BenchmarkMode (org.openjdk.jmh.annotations.BenchmarkMode)13 OutputTimeUnit (org.openjdk.jmh.annotations.OutputTimeUnit)13 Random (java.util.Random)12 ThreadLocalRandom (java.util.concurrent.ThreadLocalRandom)12 BitmapIndex (org.apache.druid.segment.column.BitmapIndex)9 ArrayList (java.util.ArrayList)6 WrappedImmutableRoaringBitmap (org.apache.druid.collections.bitmap.WrappedImmutableRoaringBitmap)6 Filter (org.apache.druid.query.filter.Filter)6 ColumnHolder (org.apache.druid.segment.column.ColumnHolder)6 MutableBitmap (org.apache.druid.collections.bitmap.MutableBitmap)5 MutableRoaringBitmap (org.roaringbitmap.buffer.MutableRoaringBitmap)5