Search in sources :

Example 36 with DimensionSelector

use of org.apache.druid.segment.DimensionSelector in project druid by druid-io.

the class ExpressionSelectorBenchmark method strlenUsingExtractionFn.

@Benchmark
public void strlenUsingExtractionFn(Blackhole blackhole) {
    final Sequence<Cursor> cursors = new QueryableIndexStorageAdapter(index).makeCursors(null, index.getDataInterval(), VirtualColumns.EMPTY, Granularities.ALL, false, null);
    final List<?> results = cursors.map(cursor -> {
        final DimensionSelector selector = cursor.getColumnSelectorFactory().makeDimensionSelector(new ExtractionDimensionSpec("x", "v", StrlenExtractionFn.instance()));
        consumeDimension(cursor, selector, blackhole);
        return null;
    }).toList();
    blackhole.consume(results);
}
Also used : BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Measurement(org.openjdk.jmh.annotations.Measurement) Intervals(org.apache.druid.java.util.common.Intervals) SegmentGenerator(org.apache.druid.segment.generator.SegmentGenerator) Blackhole(org.openjdk.jmh.infra.Blackhole) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector) Scope(org.openjdk.jmh.annotations.Scope) Warmup(org.openjdk.jmh.annotations.Warmup) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) ColumnHolder(org.apache.druid.segment.column.ColumnHolder) ImmutableList(com.google.common.collect.ImmutableList) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) DimensionSelector(org.apache.druid.segment.DimensionSelector) TearDown(org.openjdk.jmh.annotations.TearDown) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) Sequence(org.apache.druid.java.util.common.guava.Sequence) Setup(org.openjdk.jmh.annotations.Setup) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) VirtualColumns(org.apache.druid.segment.VirtualColumns) GeneratorColumnSchema(org.apache.druid.segment.generator.GeneratorColumnSchema) Mode(org.openjdk.jmh.annotations.Mode) Closer(org.apache.druid.java.util.common.io.Closer) TimeFormatExtractionFn(org.apache.druid.query.extraction.TimeFormatExtractionFn) Param(org.openjdk.jmh.annotations.Param) QueryableIndex(org.apache.druid.segment.QueryableIndex) GeneratorSchemaInfo(org.apache.druid.segment.generator.GeneratorSchemaInfo) ValueType(org.apache.druid.segment.column.ValueType) TestExprMacroTable(org.apache.druid.query.expression.TestExprMacroTable) State(org.openjdk.jmh.annotations.State) StrlenExtractionFn(org.apache.druid.query.extraction.StrlenExtractionFn) Benchmark(org.openjdk.jmh.annotations.Benchmark) Granularities(org.apache.druid.java.util.common.granularity.Granularities) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Cursor(org.apache.druid.segment.Cursor) NullHandling(org.apache.druid.common.config.NullHandling) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) Level(org.openjdk.jmh.annotations.Level) ColumnType(org.apache.druid.segment.column.ColumnType) Fork(org.openjdk.jmh.annotations.Fork) BitSet(java.util.BitSet) DimensionSelector(org.apache.druid.segment.DimensionSelector) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) Cursor(org.apache.druid.segment.Cursor) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Example 37 with DimensionSelector

use of org.apache.druid.segment.DimensionSelector in project druid by druid-io.

the class ExpressionSelectorBenchmark method strlenUsingExpressionAsString.

@Benchmark
public void strlenUsingExpressionAsString(Blackhole blackhole) {
    final Sequence<Cursor> cursors = new QueryableIndexStorageAdapter(index).makeCursors(null, index.getDataInterval(), VirtualColumns.create(ImmutableList.of(new ExpressionVirtualColumn("v", "strlen(s)", ColumnType.STRING, TestExprMacroTable.INSTANCE))), Granularities.ALL, false, null);
    final List<?> results = cursors.map(cursor -> {
        final DimensionSelector selector = cursor.getColumnSelectorFactory().makeDimensionSelector(new DefaultDimensionSpec("v", "v", ColumnType.STRING));
        consumeDimension(cursor, selector, blackhole);
        return null;
    }).toList();
    blackhole.consume(results);
}
Also used : BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Measurement(org.openjdk.jmh.annotations.Measurement) Intervals(org.apache.druid.java.util.common.Intervals) SegmentGenerator(org.apache.druid.segment.generator.SegmentGenerator) Blackhole(org.openjdk.jmh.infra.Blackhole) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector) Scope(org.openjdk.jmh.annotations.Scope) Warmup(org.openjdk.jmh.annotations.Warmup) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) ColumnHolder(org.apache.druid.segment.column.ColumnHolder) ImmutableList(com.google.common.collect.ImmutableList) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) DimensionSelector(org.apache.druid.segment.DimensionSelector) TearDown(org.openjdk.jmh.annotations.TearDown) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) Sequence(org.apache.druid.java.util.common.guava.Sequence) Setup(org.openjdk.jmh.annotations.Setup) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) VirtualColumns(org.apache.druid.segment.VirtualColumns) GeneratorColumnSchema(org.apache.druid.segment.generator.GeneratorColumnSchema) Mode(org.openjdk.jmh.annotations.Mode) Closer(org.apache.druid.java.util.common.io.Closer) TimeFormatExtractionFn(org.apache.druid.query.extraction.TimeFormatExtractionFn) Param(org.openjdk.jmh.annotations.Param) QueryableIndex(org.apache.druid.segment.QueryableIndex) GeneratorSchemaInfo(org.apache.druid.segment.generator.GeneratorSchemaInfo) ValueType(org.apache.druid.segment.column.ValueType) TestExprMacroTable(org.apache.druid.query.expression.TestExprMacroTable) State(org.openjdk.jmh.annotations.State) StrlenExtractionFn(org.apache.druid.query.extraction.StrlenExtractionFn) Benchmark(org.openjdk.jmh.annotations.Benchmark) Granularities(org.apache.druid.java.util.common.granularity.Granularities) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Cursor(org.apache.druid.segment.Cursor) NullHandling(org.apache.druid.common.config.NullHandling) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) Level(org.openjdk.jmh.annotations.Level) ColumnType(org.apache.druid.segment.column.ColumnType) Fork(org.openjdk.jmh.annotations.Fork) BitSet(java.util.BitSet) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) DimensionSelector(org.apache.druid.segment.DimensionSelector) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) Cursor(org.apache.druid.segment.Cursor) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Example 38 with DimensionSelector

use of org.apache.druid.segment.DimensionSelector in project druid by druid-io.

the class FilterPartitionBenchmark method readCursors.

private void readCursors(Sequence<Cursor> cursors, Blackhole blackhole) {
    final Sequence<Void> voids = Sequences.map(cursors, input -> {
        List<DimensionSelector> selectors = new ArrayList<>();
        selectors.add(input.getColumnSelectorFactory().makeDimensionSelector(new DefaultDimensionSpec("dimSequential", null)));
        while (!input.isDone()) {
            for (DimensionSelector selector : selectors) {
                IndexedInts row = selector.getRow();
                blackhole.consume(selector.lookupName(row.get(0)));
            }
            input.advance();
        }
        return null;
    });
    blackhole.consume(voids.toList());
}
Also used : DimensionSelector(org.apache.druid.segment.DimensionSelector) IndexedInts(org.apache.druid.segment.data.IndexedInts) ArrayList(java.util.ArrayList) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec)

Example 39 with DimensionSelector

use of org.apache.druid.segment.DimensionSelector in project druid by druid-io.

the class ExpressionSelectorBenchmark method timeFormatUsingExpression.

@Benchmark
public void timeFormatUsingExpression(Blackhole blackhole) {
    final Sequence<Cursor> cursors = new QueryableIndexStorageAdapter(index).makeCursors(null, index.getDataInterval(), VirtualColumns.create(ImmutableList.of(new ExpressionVirtualColumn("v", "timestamp_format(__time, 'yyyy-MM-dd')", ColumnType.STRING, TestExprMacroTable.INSTANCE))), Granularities.ALL, false, null);
    final List<?> results = cursors.map(cursor -> {
        final DimensionSelector selector = cursor.getColumnSelectorFactory().makeDimensionSelector(DefaultDimensionSpec.of("v"));
        consumeDimension(cursor, selector, blackhole);
        return null;
    }).toList();
    blackhole.consume(results);
}
Also used : BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Measurement(org.openjdk.jmh.annotations.Measurement) Intervals(org.apache.druid.java.util.common.Intervals) SegmentGenerator(org.apache.druid.segment.generator.SegmentGenerator) Blackhole(org.openjdk.jmh.infra.Blackhole) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector) Scope(org.openjdk.jmh.annotations.Scope) Warmup(org.openjdk.jmh.annotations.Warmup) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) ColumnHolder(org.apache.druid.segment.column.ColumnHolder) ImmutableList(com.google.common.collect.ImmutableList) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) DimensionSelector(org.apache.druid.segment.DimensionSelector) TearDown(org.openjdk.jmh.annotations.TearDown) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) Sequence(org.apache.druid.java.util.common.guava.Sequence) Setup(org.openjdk.jmh.annotations.Setup) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) VirtualColumns(org.apache.druid.segment.VirtualColumns) GeneratorColumnSchema(org.apache.druid.segment.generator.GeneratorColumnSchema) Mode(org.openjdk.jmh.annotations.Mode) Closer(org.apache.druid.java.util.common.io.Closer) TimeFormatExtractionFn(org.apache.druid.query.extraction.TimeFormatExtractionFn) Param(org.openjdk.jmh.annotations.Param) QueryableIndex(org.apache.druid.segment.QueryableIndex) GeneratorSchemaInfo(org.apache.druid.segment.generator.GeneratorSchemaInfo) ValueType(org.apache.druid.segment.column.ValueType) TestExprMacroTable(org.apache.druid.query.expression.TestExprMacroTable) State(org.openjdk.jmh.annotations.State) StrlenExtractionFn(org.apache.druid.query.extraction.StrlenExtractionFn) Benchmark(org.openjdk.jmh.annotations.Benchmark) Granularities(org.apache.druid.java.util.common.granularity.Granularities) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Cursor(org.apache.druid.segment.Cursor) NullHandling(org.apache.druid.common.config.NullHandling) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) Level(org.openjdk.jmh.annotations.Level) ColumnType(org.apache.druid.segment.column.ColumnType) Fork(org.openjdk.jmh.annotations.Fork) BitSet(java.util.BitSet) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) DimensionSelector(org.apache.druid.segment.DimensionSelector) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) Cursor(org.apache.druid.segment.Cursor) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Example 40 with DimensionSelector

use of org.apache.druid.segment.DimensionSelector in project druid by druid-io.

the class ExpressionSelectorBenchmark method timeFormatUsingExtractionFn.

@Benchmark
public void timeFormatUsingExtractionFn(Blackhole blackhole) {
    final Sequence<Cursor> cursors = new QueryableIndexStorageAdapter(index).makeCursors(null, index.getDataInterval(), VirtualColumns.EMPTY, Granularities.ALL, false, null);
    final List<?> results = cursors.map(cursor -> {
        final DimensionSelector selector = cursor.getColumnSelectorFactory().makeDimensionSelector(new ExtractionDimensionSpec(ColumnHolder.TIME_COLUMN_NAME, "v", new TimeFormatExtractionFn("yyyy-MM-dd", null, null, null, false)));
        consumeDimension(cursor, selector, blackhole);
        return null;
    }).toList();
    blackhole.consume(results);
}
Also used : BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Measurement(org.openjdk.jmh.annotations.Measurement) Intervals(org.apache.druid.java.util.common.Intervals) SegmentGenerator(org.apache.druid.segment.generator.SegmentGenerator) Blackhole(org.openjdk.jmh.infra.Blackhole) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector) Scope(org.openjdk.jmh.annotations.Scope) Warmup(org.openjdk.jmh.annotations.Warmup) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) ColumnHolder(org.apache.druid.segment.column.ColumnHolder) ImmutableList(com.google.common.collect.ImmutableList) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) DimensionSelector(org.apache.druid.segment.DimensionSelector) TearDown(org.openjdk.jmh.annotations.TearDown) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) ExpressionVirtualColumn(org.apache.druid.segment.virtual.ExpressionVirtualColumn) Sequence(org.apache.druid.java.util.common.guava.Sequence) Setup(org.openjdk.jmh.annotations.Setup) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) VirtualColumns(org.apache.druid.segment.VirtualColumns) GeneratorColumnSchema(org.apache.druid.segment.generator.GeneratorColumnSchema) Mode(org.openjdk.jmh.annotations.Mode) Closer(org.apache.druid.java.util.common.io.Closer) TimeFormatExtractionFn(org.apache.druid.query.extraction.TimeFormatExtractionFn) Param(org.openjdk.jmh.annotations.Param) QueryableIndex(org.apache.druid.segment.QueryableIndex) GeneratorSchemaInfo(org.apache.druid.segment.generator.GeneratorSchemaInfo) ValueType(org.apache.druid.segment.column.ValueType) TestExprMacroTable(org.apache.druid.query.expression.TestExprMacroTable) State(org.openjdk.jmh.annotations.State) StrlenExtractionFn(org.apache.druid.query.extraction.StrlenExtractionFn) Benchmark(org.openjdk.jmh.annotations.Benchmark) Granularities(org.apache.druid.java.util.common.granularity.Granularities) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Cursor(org.apache.druid.segment.Cursor) NullHandling(org.apache.druid.common.config.NullHandling) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) Level(org.openjdk.jmh.annotations.Level) ColumnType(org.apache.druid.segment.column.ColumnType) Fork(org.openjdk.jmh.annotations.Fork) BitSet(java.util.BitSet) TimeFormatExtractionFn(org.apache.druid.query.extraction.TimeFormatExtractionFn) DimensionSelector(org.apache.druid.segment.DimensionSelector) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) Cursor(org.apache.druid.segment.Cursor) ExtractionDimensionSpec(org.apache.druid.query.dimension.ExtractionDimensionSpec) Benchmark(org.openjdk.jmh.annotations.Benchmark)

Aggregations

DimensionSelector (org.apache.druid.segment.DimensionSelector)66 Test (org.junit.Test)36 DefaultDimensionSpec (org.apache.druid.query.dimension.DefaultDimensionSpec)31 Cursor (org.apache.druid.segment.Cursor)24 InitializedNullHandlingTest (org.apache.druid.testing.InitializedNullHandlingTest)24 IndexedInts (org.apache.druid.segment.data.IndexedInts)22 List (java.util.List)14 VirtualColumns (org.apache.druid.segment.VirtualColumns)14 ImmutableList (com.google.common.collect.ImmutableList)12 StorageAdapter (org.apache.druid.segment.StorageAdapter)12 ArrayList (java.util.ArrayList)11 Intervals (org.apache.druid.java.util.common.Intervals)11 Granularities (org.apache.druid.java.util.common.granularity.Granularities)11 Sequence (org.apache.druid.java.util.common.guava.Sequence)11 QueryableIndexStorageAdapter (org.apache.druid.segment.QueryableIndexStorageAdapter)11 NullHandling (org.apache.druid.common.config.NullHandling)10 ColumnSelectorFactory (org.apache.druid.segment.ColumnSelectorFactory)9 DataSegment (org.apache.druid.timeline.DataSegment)9 ConstantDimensionSelector (org.apache.druid.segment.ConstantDimensionSelector)8 ValueMatcher (org.apache.druid.query.filter.ValueMatcher)7