Search in sources :

Example 16 with BufferAggregator

use of org.apache.druid.query.aggregation.BufferAggregator in project druid by druid-io.

the class LongAnyAggregationTest method testLongAnyCombiningBufferAggregator.

@Test
public void testLongAnyCombiningBufferAggregator() {
    BufferAggregator agg = combiningAggFactory.factorizeBuffered(colSelectorFactory);
    ByteBuffer buffer = ByteBuffer.wrap(new byte[longAnyAggFactory.getMaxIntermediateSizeWithNulls()]);
    agg.init(buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    Long result = (Long) agg.get(buffer, 0);
    Assert.assertEquals(objects[0], result, 0.0001);
    Assert.assertEquals(objects[0].longValue(), agg.getLong(buffer, 0));
    Assert.assertEquals(objects[0], agg.getLong(buffer, 0), 0.0001);
}
Also used : BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator) ByteBuffer(java.nio.ByteBuffer) InitializedNullHandlingTest(org.apache.druid.testing.InitializedNullHandlingTest) Test(org.junit.Test)

Example 17 with BufferAggregator

use of org.apache.druid.query.aggregation.BufferAggregator in project druid by druid-io.

the class VarianceAggregatorFactoryUnitTest method factorizeBufferedForComplexShouldReturnObjectVectorAggregator.

@Test
public void factorizeBufferedForComplexShouldReturnObjectVectorAggregator() {
    mockType(VarianceAggregatorFactory.TYPE);
    BufferAggregator agg = target.factorizeBuffered(metricFactory);
    Assert.assertNotNull(agg);
    Assert.assertEquals(VarianceBufferAggregator.ObjectVarianceAggregator.class, agg.getClass());
}
Also used : BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator) InitializedNullHandlingTest(org.apache.druid.testing.InitializedNullHandlingTest) Test(org.junit.Test)

Example 18 with BufferAggregator

use of org.apache.druid.query.aggregation.BufferAggregator in project druid by druid-io.

the class ExpressionAggregationBenchmark method compute.

private double compute(final Function<ColumnSelectorFactory, BufferAggregator> aggregatorFactory) {
    final QueryableIndexStorageAdapter adapter = new QueryableIndexStorageAdapter(index);
    final Sequence<Cursor> cursors = adapter.makeCursors(null, index.getDataInterval(), VirtualColumns.EMPTY, Granularities.ALL, false, null);
    final List<Double> results = cursors.map(cursor -> {
        final BufferAggregator bufferAggregator = aggregatorFactory.apply(cursor.getColumnSelectorFactory());
        bufferAggregator.init(aggregationBuffer, 0);
        while (!cursor.isDone()) {
            bufferAggregator.aggregate(aggregationBuffer, 0);
            cursor.advance();
        }
        final Double dbl = (Double) bufferAggregator.get(aggregationBuffer, 0);
        bufferAggregator.close();
        return dbl;
    }).toList();
    return Iterables.getOnlyElement(results);
}
Also used : Iterables(com.google.common.collect.Iterables) DoubleSumAggregatorFactory(org.apache.druid.query.aggregation.DoubleSumAggregatorFactory) BenchmarkMode(org.openjdk.jmh.annotations.BenchmarkMode) Measurement(org.openjdk.jmh.annotations.Measurement) Intervals(org.apache.druid.java.util.common.Intervals) RuntimeShapeInspector(org.apache.druid.query.monomorphicprocessing.RuntimeShapeInspector) SegmentGenerator(org.apache.druid.segment.generator.SegmentGenerator) Blackhole(org.openjdk.jmh.infra.Blackhole) Scope(org.openjdk.jmh.annotations.Scope) Warmup(org.openjdk.jmh.annotations.Warmup) Function(java.util.function.Function) JavaScriptConfig(org.apache.druid.js.JavaScriptConfig) ByteBuffer(java.nio.ByteBuffer) BaseFloatColumnValueSelector(org.apache.druid.segment.BaseFloatColumnValueSelector) BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator) ColumnSelectorFactory(org.apache.druid.segment.ColumnSelectorFactory) ImmutableList(com.google.common.collect.ImmutableList) OutputTimeUnit(org.openjdk.jmh.annotations.OutputTimeUnit) TearDown(org.openjdk.jmh.annotations.TearDown) Sequence(org.apache.druid.java.util.common.guava.Sequence) Setup(org.openjdk.jmh.annotations.Setup) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) VirtualColumns(org.apache.druid.segment.VirtualColumns) GeneratorColumnSchema(org.apache.druid.segment.generator.GeneratorColumnSchema) Mode(org.openjdk.jmh.annotations.Mode) Closer(org.apache.druid.java.util.common.io.Closer) Param(org.openjdk.jmh.annotations.Param) QueryableIndex(org.apache.druid.segment.QueryableIndex) JavaScriptAggregatorFactory(org.apache.druid.query.aggregation.JavaScriptAggregatorFactory) GeneratorSchemaInfo(org.apache.druid.segment.generator.GeneratorSchemaInfo) ValueType(org.apache.druid.segment.column.ValueType) TestExprMacroTable(org.apache.druid.query.expression.TestExprMacroTable) State(org.openjdk.jmh.annotations.State) Benchmark(org.openjdk.jmh.annotations.Benchmark) Granularities(org.apache.druid.java.util.common.granularity.Granularities) TimeUnit(java.util.concurrent.TimeUnit) List(java.util.List) Cursor(org.apache.druid.segment.Cursor) NullHandling(org.apache.druid.common.config.NullHandling) LinearShardSpec(org.apache.druid.timeline.partition.LinearShardSpec) DataSegment(org.apache.druid.timeline.DataSegment) Level(org.openjdk.jmh.annotations.Level) Fork(org.openjdk.jmh.annotations.Fork) QueryableIndexStorageAdapter(org.apache.druid.segment.QueryableIndexStorageAdapter) Cursor(org.apache.druid.segment.Cursor) BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator)

Example 19 with BufferAggregator

use of org.apache.druid.query.aggregation.BufferAggregator in project druid by druid-io.

the class DoubleLastAggregationTest method testDoubleLastCombiningBufferAggregator.

@Test
public void testDoubleLastCombiningBufferAggregator() {
    BufferAggregator agg = combiningAggFactory.factorizeBuffered(colSelectorFactory);
    ByteBuffer buffer = ByteBuffer.wrap(new byte[doubleLastAggFactory.getMaxIntermediateSizeWithNulls()]);
    agg.init(buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    Pair<Long, Double> result = (Pair<Long, Double>) agg.get(buffer, 0);
    Pair<Long, Double> expected = (Pair<Long, Double>) pairs[2];
    Assert.assertEquals(expected.lhs, result.lhs);
    Assert.assertEquals(expected.rhs, result.rhs, 0.0001);
    Assert.assertEquals(expected.rhs.longValue(), agg.getLong(buffer, 0));
    Assert.assertEquals(expected.rhs, agg.getDouble(buffer, 0), 0.0001);
}
Also used : BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator) ByteBuffer(java.nio.ByteBuffer) SerializablePair(org.apache.druid.collections.SerializablePair) Pair(org.apache.druid.java.util.common.Pair) InitializedNullHandlingTest(org.apache.druid.testing.InitializedNullHandlingTest) Test(org.junit.Test)

Example 20 with BufferAggregator

use of org.apache.druid.query.aggregation.BufferAggregator in project druid by druid-io.

the class DoubleLastAggregationTest method testDoubleLastBufferAggregatorWithTimeColumn.

@Test
public void testDoubleLastBufferAggregatorWithTimeColumn() {
    BufferAggregator agg = new DoubleLastAggregatorFactory("billy", "nilly", "customTime").factorizeBuffered(colSelectorFactory);
    ByteBuffer buffer = ByteBuffer.wrap(new byte[doubleLastAggFactory.getMaxIntermediateSizeWithNulls()]);
    agg.init(buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    aggregate(agg, buffer, 0);
    Pair<Long, Double> result = (Pair<Long, Double>) agg.get(buffer, 0);
    Assert.assertEquals(customTimes[1], result.lhs.longValue());
    Assert.assertEquals(doubles[1], result.rhs, 0.0001);
    Assert.assertEquals((long) doubles[1], agg.getLong(buffer, 0));
    Assert.assertEquals(doubles[1], agg.getDouble(buffer, 0), 0.0001);
}
Also used : BufferAggregator(org.apache.druid.query.aggregation.BufferAggregator) ByteBuffer(java.nio.ByteBuffer) SerializablePair(org.apache.druid.collections.SerializablePair) Pair(org.apache.druid.java.util.common.Pair) InitializedNullHandlingTest(org.apache.druid.testing.InitializedNullHandlingTest) Test(org.junit.Test)

Aggregations

BufferAggregator (org.apache.druid.query.aggregation.BufferAggregator)36 ByteBuffer (java.nio.ByteBuffer)33 Test (org.junit.Test)33 InitializedNullHandlingTest (org.apache.druid.testing.InitializedNullHandlingTest)27 Pair (org.apache.druid.java.util.common.Pair)23 SerializablePair (org.apache.druid.collections.SerializablePair)18 SerializablePairLongString (org.apache.druid.query.aggregation.SerializablePairLongString)5 ColumnSelectorFactory (org.apache.druid.segment.ColumnSelectorFactory)2 Cursor (org.apache.druid.segment.Cursor)2 ImmutableList (com.google.common.collect.ImmutableList)1 Iterables (com.google.common.collect.Iterables)1 List (java.util.List)1 TimeUnit (java.util.concurrent.TimeUnit)1 Function (java.util.function.Function)1 NullHandling (org.apache.druid.common.config.NullHandling)1 Intervals (org.apache.druid.java.util.common.Intervals)1 Granularities (org.apache.druid.java.util.common.granularity.Granularities)1 Sequence (org.apache.druid.java.util.common.guava.Sequence)1 Closer (org.apache.druid.java.util.common.io.Closer)1 JavaScriptConfig (org.apache.druid.js.JavaScriptConfig)1