Search in sources :

Example 1 with AggregationImplModule

use of io.crate.execution.engine.aggregation.impl.AggregationImplModule in project crate by crate.

the class AggregateCollectorBenchmark method setup.

@Setup
public void setup() {
    InputCollectExpression inExpr0 = new InputCollectExpression(0);
    Functions functions = new ModulesBuilder().add(new AggregationImplModule()).createInjector().getInstance(Functions.class);
    SumAggregation<?> sumAggregation = (SumAggregation<?>) functions.getQualified(Signature.aggregate(SumAggregation.NAME, DataTypes.INTEGER.getTypeSignature(), DataTypes.LONG.getTypeSignature()), List.of(DataTypes.INTEGER), DataTypes.INTEGER);
    var memoryManager = new OnHeapMemoryManager(bytes -> {
    });
    collector = new AggregateCollector(Collections.singletonList(inExpr0), RamAccounting.NO_ACCOUNTING, memoryManager, Version.CURRENT, AggregateMode.ITER_FINAL, new AggregationFunction[] { sumAggregation }, Version.CURRENT, new Input[][] { { inExpr0 } }, new Input[] { Literal.BOOLEAN_TRUE });
}
Also used : Input(io.crate.data.Input) InputCollectExpression(io.crate.execution.engine.collect.InputCollectExpression) OnHeapMemoryManager(io.crate.memory.OnHeapMemoryManager) SumAggregation(io.crate.execution.engine.aggregation.impl.SumAggregation) Functions(io.crate.metadata.Functions) ModulesBuilder(org.elasticsearch.common.inject.ModulesBuilder) AggregationImplModule(io.crate.execution.engine.aggregation.impl.AggregationImplModule) Setup(org.openjdk.jmh.annotations.Setup)

Example 2 with AggregationImplModule

use of io.crate.execution.engine.aggregation.impl.AggregationImplModule in project crate by crate.

the class GroupingStringCollectorBenchmark method createGroupingCollector.

@Setup
public void createGroupingCollector() {
    Functions functions = new ModulesBuilder().add(new AggregationImplModule()).createInjector().getInstance(Functions.class);
    groupByMinCollector = createGroupByMinBytesRefCollector(functions);
    memoryManager = new OnHeapMemoryManager(bytes -> {
    });
    List<String> keys = new ArrayList<>(Locale.getISOCountries().length);
    keys.addAll(Arrays.asList(Locale.getISOCountries()));
    rows = new ArrayList<>(20_000_000);
    for (int i = 0; i < 20_000_000; i++) {
        rows.add(new Row1(keys.get(i % keys.size())));
    }
}
Also used : Input(io.crate.data.Input) MinimumAggregation(io.crate.execution.engine.aggregation.impl.MinimumAggregation) BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Arrays(java.util.Arrays) AggregationImplModule(io.crate.execution.engine.aggregation.impl.AggregationImplModule) Blackhole(org.openjdk.jmh.infra.Blackhole) SENTINEL(io.crate.data.SentinelRow.SENTINEL) CollectExpression(io.crate.execution.engine.collect.CollectExpression) BatchIterator(io.crate.data.BatchIterator) Scope(org.openjdk.jmh.annotations.Scope) ArrayList(java.util.ArrayList) Functions(io.crate.metadata.Functions) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) Locale(java.util.Locale) Setup(org.openjdk.jmh.annotations.Setup) Mode(org.openjdk.jmh.annotations.Mode) InMemoryBatchIterator(io.crate.data.InMemoryBatchIterator) Signature(io.crate.metadata.functions.Signature) State(org.openjdk.jmh.annotations.State) RamAccounting(io.crate.breaker.RamAccounting) InputCollectExpression(io.crate.execution.engine.collect.InputCollectExpression) Benchmark(org.openjdk.jmh.annotations.Benchmark) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) BatchIterators(io.crate.data.BatchIterators) Version(org.elasticsearch.Version) Row(io.crate.data.Row) Literal(io.crate.expression.symbol.Literal) AggregateMode(io.crate.expression.symbol.AggregateMode) DataTypes(io.crate.types.DataTypes) OnHeapMemoryManager(io.crate.memory.OnHeapMemoryManager) ModulesBuilder(org.elasticsearch.common.inject.ModulesBuilder) Collections(java.util.Collections) Row1(io.crate.data.Row1) Row1(io.crate.data.Row1) OnHeapMemoryManager(io.crate.memory.OnHeapMemoryManager) ArrayList(java.util.ArrayList) Functions(io.crate.metadata.Functions) ModulesBuilder(org.elasticsearch.common.inject.ModulesBuilder) AggregationImplModule(io.crate.execution.engine.aggregation.impl.AggregationImplModule) Setup(org.openjdk.jmh.annotations.Setup)

Example 3 with AggregationImplModule

use of io.crate.execution.engine.aggregation.impl.AggregationImplModule in project crate by crate.

the class GroupingLongCollectorBenchmark method createGroupingCollector.

@Setup
public void createGroupingCollector() throws Exception {
    IndexWriter iw = new IndexWriter(new ByteBuffersDirectory(), new IndexWriterConfig(new StandardAnalyzer()));
    Functions functions = new ModulesBuilder().add(new AggregationImplModule()).createInjector().getInstance(Functions.class);
    SumAggregation<?> sumAgg = (SumAggregation<?>) functions.getQualified(Signature.aggregate(SumAggregation.NAME, DataTypes.INTEGER.getTypeSignature(), DataTypes.LONG.getTypeSignature()), List.of(DataTypes.INTEGER), DataTypes.INTEGER);
    var memoryManager = new OnHeapMemoryManager(bytes -> {
    });
    groupBySumCollector = createGroupBySumCollector(sumAgg, memoryManager);
    int size = 20_000_000;
    rows = new ArrayList<>(size);
    numbers = new long[size];
    for (int i = 0; i < size; i++) {
        long value = (long) i % 200;
        rows.add(new Row1(value));
        numbers[i] = value;
        var doc = new Document();
        doc.add(new NumericDocValuesField("x", value));
        doc.add(new SortedNumericDocValuesField("y", value));
        iw.addDocument(doc);
    }
    iw.commit();
    iw.forceMerge(1, true);
    searcher = new IndexSearcher(DirectoryReader.open(iw));
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) OnHeapMemoryManager(io.crate.memory.OnHeapMemoryManager) Functions(io.crate.metadata.Functions) Document(org.apache.lucene.document.Document) AggregationImplModule(io.crate.execution.engine.aggregation.impl.AggregationImplModule) Row1(io.crate.data.Row1) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) IndexWriter(org.apache.lucene.index.IndexWriter) ByteBuffersDirectory(org.apache.lucene.store.ByteBuffersDirectory) StandardAnalyzer(org.apache.lucene.analysis.standard.StandardAnalyzer) SumAggregation(io.crate.execution.engine.aggregation.impl.SumAggregation) ModulesBuilder(org.elasticsearch.common.inject.ModulesBuilder) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) Setup(org.openjdk.jmh.annotations.Setup)

Aggregations

AggregationImplModule (io.crate.execution.engine.aggregation.impl.AggregationImplModule)3 OnHeapMemoryManager (io.crate.memory.OnHeapMemoryManager)3 Functions (io.crate.metadata.Functions)3 ModulesBuilder (org.elasticsearch.common.inject.ModulesBuilder)3 Setup (org.openjdk.jmh.annotations.Setup)3 Input (io.crate.data.Input)2 Row1 (io.crate.data.Row1)2 SumAggregation (io.crate.execution.engine.aggregation.impl.SumAggregation)2 InputCollectExpression (io.crate.execution.engine.collect.InputCollectExpression)2 RamAccounting (io.crate.breaker.RamAccounting)1 BatchIterator (io.crate.data.BatchIterator)1 BatchIterators (io.crate.data.BatchIterators)1 InMemoryBatchIterator (io.crate.data.InMemoryBatchIterator)1 Row (io.crate.data.Row)1 SENTINEL (io.crate.data.SentinelRow.SENTINEL)1 MinimumAggregation (io.crate.execution.engine.aggregation.impl.MinimumAggregation)1 CollectExpression (io.crate.execution.engine.collect.CollectExpression)1 AggregateMode (io.crate.expression.symbol.AggregateMode)1 Literal (io.crate.expression.symbol.Literal)1 Signature (io.crate.metadata.functions.Signature)1