Search in sources :

Example 1 with NilColumnValueSelector

use of org.apache.druid.segment.NilColumnValueSelector in project druid by druid-io.

the class HyperUniquesAggregatorFactory method factorizeBuffered.

@Override
public BufferAggregator factorizeBuffered(ColumnSelectorFactory metricFactory) {
    BaseObjectColumnValueSelector selector = metricFactory.makeColumnValueSelector(fieldName);
    if (selector instanceof NilColumnValueSelector) {
        return NoopBufferAggregator.instance();
    }
    final Class classOfObject = selector.classOfObject();
    if (classOfObject.equals(Object.class) || HyperLogLogCollector.class.isAssignableFrom(classOfObject)) {
        return new HyperUniquesBufferAggregator(selector);
    }
    throw new IAE("Incompatible type for metric[%s], expected a HyperUnique, got a %s", fieldName, classOfObject);
}
Also used : HyperLogLogCollector(org.apache.druid.hll.HyperLogLogCollector) BaseObjectColumnValueSelector(org.apache.druid.segment.BaseObjectColumnValueSelector) IAE(org.apache.druid.java.util.common.IAE) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector)

Example 2 with NilColumnValueSelector

use of org.apache.druid.segment.NilColumnValueSelector in project druid by druid-io.

the class ArrayOfDoublesSketchAggregatorFactory method factorizeBuffered.

@Override
public BufferAggregator factorizeBuffered(final ColumnSelectorFactory metricFactory) {
    if (metricColumns == null) {
        // input is sketches, use merge aggregator
        final BaseObjectColumnValueSelector<ArrayOfDoublesSketch> selector = metricFactory.makeColumnValueSelector(fieldName);
        if (selector instanceof NilColumnValueSelector) {
            return new NoopArrayOfDoublesSketchBufferAggregator(numberOfValues);
        }
        return new ArrayOfDoublesSketchMergeBufferAggregator(selector, nominalEntries, numberOfValues, getMaxIntermediateSizeWithNulls());
    }
    // input is raw data (key and array of values), use build aggregator
    final DimensionSelector keySelector = metricFactory.makeDimensionSelector(new DefaultDimensionSpec(fieldName, fieldName));
    if (DimensionSelector.isNilSelector(keySelector)) {
        return new NoopArrayOfDoublesSketchBufferAggregator(numberOfValues);
    }
    final List<BaseDoubleColumnValueSelector> valueSelectors = new ArrayList<>();
    for (final String column : metricColumns) {
        final BaseDoubleColumnValueSelector valueSelector = metricFactory.makeColumnValueSelector(column);
        valueSelectors.add(valueSelector);
    }
    return new ArrayOfDoublesSketchBuildBufferAggregator(keySelector, valueSelectors, nominalEntries, getMaxIntermediateSizeWithNulls());
}
Also used : BaseDoubleColumnValueSelector(org.apache.druid.segment.BaseDoubleColumnValueSelector) DimensionSelector(org.apache.druid.segment.DimensionSelector) ArrayList(java.util.ArrayList) ArrayOfDoublesSketch(org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec)

Example 3 with NilColumnValueSelector

use of org.apache.druid.segment.NilColumnValueSelector in project druid by druid-io.

the class ArrayOfDoublesSketchAggregatorFactory method factorize.

@Override
public Aggregator factorize(final ColumnSelectorFactory metricFactory) {
    if (metricColumns == null) {
        // input is sketches, use merge aggregator
        final BaseObjectColumnValueSelector<ArrayOfDoublesSketch> selector = metricFactory.makeColumnValueSelector(fieldName);
        if (selector instanceof NilColumnValueSelector) {
            return new NoopArrayOfDoublesSketchAggregator(numberOfValues);
        }
        return new ArrayOfDoublesSketchMergeAggregator(selector, nominalEntries, numberOfValues);
    }
    // input is raw data (key and array of values), use build aggregator
    final DimensionSelector keySelector = metricFactory.makeDimensionSelector(new DefaultDimensionSpec(fieldName, fieldName));
    if (DimensionSelector.isNilSelector(keySelector)) {
        return new NoopArrayOfDoublesSketchAggregator(numberOfValues);
    }
    final List<BaseDoubleColumnValueSelector> valueSelectors = new ArrayList<>();
    for (final String column : metricColumns) {
        final BaseDoubleColumnValueSelector valueSelector = metricFactory.makeColumnValueSelector(column);
        valueSelectors.add(valueSelector);
    }
    return new ArrayOfDoublesSketchBuildAggregator(keySelector, valueSelectors, nominalEntries);
}
Also used : BaseDoubleColumnValueSelector(org.apache.druid.segment.BaseDoubleColumnValueSelector) DimensionSelector(org.apache.druid.segment.DimensionSelector) ArrayList(java.util.ArrayList) ArrayOfDoublesSketch(org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec)

Example 4 with NilColumnValueSelector

use of org.apache.druid.segment.NilColumnValueSelector in project druid by druid-io.

the class ExpressionSelectors method createBindings.

/**
 * Create {@link Expr.ObjectBinding} given a {@link ColumnSelectorFactory} and {@link ExpressionPlan} which
 * provides the set of identifiers which need a binding (list of required columns), and context of whether or not they
 * are used as array or scalar inputs
 */
public static Expr.ObjectBinding createBindings(ColumnSelectorFactory columnSelectorFactory, ExpressionPlan plan) {
    final List<String> columns = plan.getAnalysis().getRequiredBindingsList();
    final Map<String, Pair<ExpressionType, Supplier<Object>>> suppliers = new HashMap<>();
    for (String columnName : columns) {
        final ColumnCapabilities capabilities = columnSelectorFactory.getColumnCapabilities(columnName);
        final boolean multiVal = capabilities != null && capabilities.hasMultipleValues().isTrue();
        final Supplier<Object> supplier;
        final ExpressionType expressionType = ExpressionType.fromColumnType(capabilities);
        final boolean useObjectSupplierForMultiValueStringArray = capabilities != null && // multi-value rows, we can just use the dimension selector, which has the homogenization behavior built-in
        ((!capabilities.is(ValueType.STRING)) || (capabilities.is(ValueType.STRING) && !ExpressionProcessing.isHomogenizeNullMultiValueStringArrays() && !plan.is(ExpressionPlan.Trait.NEEDS_APPLIED))) && // expression has array output
        plan.is(ExpressionPlan.Trait.NON_SCALAR_OUTPUT);
        final boolean homogenizeNullMultiValueStringArrays = plan.is(ExpressionPlan.Trait.NEEDS_APPLIED) || ExpressionProcessing.isHomogenizeNullMultiValueStringArrays();
        if (capabilities == null || capabilities.isArray() || useObjectSupplierForMultiValueStringArray) {
            // Unknown type, array type, or output array uses an Object selector and see if that gives anything useful
            supplier = supplierFromObjectSelector(columnSelectorFactory.makeColumnValueSelector(columnName), homogenizeNullMultiValueStringArrays);
        } else if (capabilities.is(ValueType.FLOAT)) {
            ColumnValueSelector<?> selector = columnSelectorFactory.makeColumnValueSelector(columnName);
            supplier = makeNullableNumericSupplier(selector, selector::getFloat);
        } else if (capabilities.is(ValueType.LONG)) {
            ColumnValueSelector<?> selector = columnSelectorFactory.makeColumnValueSelector(columnName);
            supplier = makeNullableNumericSupplier(selector, selector::getLong);
        } else if (capabilities.is(ValueType.DOUBLE)) {
            ColumnValueSelector<?> selector = columnSelectorFactory.makeColumnValueSelector(columnName);
            supplier = makeNullableNumericSupplier(selector, selector::getDouble);
        } else if (capabilities.is(ValueType.STRING)) {
            supplier = supplierFromDimensionSelector(columnSelectorFactory.makeDimensionSelector(new DefaultDimensionSpec(columnName, columnName)), multiVal, homogenizeNullMultiValueStringArrays);
        } else {
            // complex type just pass straight through
            ColumnValueSelector<?> selector = columnSelectorFactory.makeColumnValueSelector(columnName);
            if (!(selector instanceof NilColumnValueSelector)) {
                supplier = selector::getObject;
            } else {
                supplier = null;
            }
        }
        if (supplier != null) {
            suppliers.put(columnName, new Pair<>(expressionType, supplier));
        }
    }
    if (suppliers.isEmpty()) {
        return InputBindings.nilBindings();
    } else if (suppliers.size() == 1 && columns.size() == 1) {
        // If there's only one column (and it has a supplier), we can skip the Map and just use that supplier when
        // asked for something.
        final String column = Iterables.getOnlyElement(suppliers.keySet());
        final Pair<ExpressionType, Supplier<Object>> supplier = Iterables.getOnlyElement(suppliers.values());
        return new Expr.ObjectBinding() {

            @Nullable
            @Override
            public Object get(String name) {
                // There's only one binding, and it must be the single column, so it can safely be ignored in production.
                assert column.equals(name);
                return supplier.rhs.get();
            }

            @Nullable
            @Override
            public ExpressionType getType(String name) {
                return supplier.lhs;
            }
        };
    } else {
        return InputBindings.withTypedSuppliers(suppliers);
    }
}
Also used : HashMap(java.util.HashMap) ColumnCapabilities(org.apache.druid.segment.column.ColumnCapabilities) DefaultDimensionSpec(org.apache.druid.query.dimension.DefaultDimensionSpec) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector) Expr(org.apache.druid.math.expr.Expr) ExpressionType(org.apache.druid.math.expr.ExpressionType) Nullable(javax.annotation.Nullable) Pair(org.apache.druid.java.util.common.Pair) NonnullPair(org.apache.druid.java.util.common.NonnullPair) ColumnValueSelector(org.apache.druid.segment.ColumnValueSelector) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector) BaseObjectColumnValueSelector(org.apache.druid.segment.BaseObjectColumnValueSelector)

Example 5 with NilColumnValueSelector

use of org.apache.druid.segment.NilColumnValueSelector in project druid by druid-io.

the class HyperUniquesAggregatorFactory method factorize.

@Override
public Aggregator factorize(ColumnSelectorFactory metricFactory) {
    BaseObjectColumnValueSelector selector = metricFactory.makeColumnValueSelector(fieldName);
    if (selector instanceof NilColumnValueSelector) {
        return NoopAggregator.instance();
    }
    final Class classOfObject = selector.classOfObject();
    if (classOfObject.equals(Object.class) || HyperLogLogCollector.class.isAssignableFrom(classOfObject)) {
        return new HyperUniquesAggregator(selector);
    }
    throw new IAE("Incompatible type for metric[%s], expected a HyperUnique, got a %s", fieldName, classOfObject);
}
Also used : HyperLogLogCollector(org.apache.druid.hll.HyperLogLogCollector) BaseObjectColumnValueSelector(org.apache.druid.segment.BaseObjectColumnValueSelector) IAE(org.apache.druid.java.util.common.IAE) NilColumnValueSelector(org.apache.druid.segment.NilColumnValueSelector)

Aggregations

NilColumnValueSelector (org.apache.druid.segment.NilColumnValueSelector)5 DefaultDimensionSpec (org.apache.druid.query.dimension.DefaultDimensionSpec)3 BaseObjectColumnValueSelector (org.apache.druid.segment.BaseObjectColumnValueSelector)3 ArrayList (java.util.ArrayList)2 ArrayOfDoublesSketch (org.apache.datasketches.tuple.arrayofdoubles.ArrayOfDoublesSketch)2 HyperLogLogCollector (org.apache.druid.hll.HyperLogLogCollector)2 IAE (org.apache.druid.java.util.common.IAE)2 BaseDoubleColumnValueSelector (org.apache.druid.segment.BaseDoubleColumnValueSelector)2 DimensionSelector (org.apache.druid.segment.DimensionSelector)2 HashMap (java.util.HashMap)1 Nullable (javax.annotation.Nullable)1 NonnullPair (org.apache.druid.java.util.common.NonnullPair)1 Pair (org.apache.druid.java.util.common.Pair)1 Expr (org.apache.druid.math.expr.Expr)1 ExpressionType (org.apache.druid.math.expr.ExpressionType)1 ColumnValueSelector (org.apache.druid.segment.ColumnValueSelector)1 ColumnCapabilities (org.apache.druid.segment.column.ColumnCapabilities)1