Search in sources :

Example 6 with Aggregation

use of org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation in project hive by apache.

the class VectorUDAFBloomFilter method iterateNoNullsSelectionWithAggregationSelection.

private void iterateNoNullsSelectionWithAggregationSelection(VectorAggregationBufferRow[] aggregationBufferSets, int aggregateIndex, ColumnVector inputColumn, int[] selection, int batchSize) {
    for (int i = 0; i < batchSize; ++i) {
        int row = selection[i];
        Aggregation myagg = getCurrentAggregationBuffer(aggregationBufferSets, aggregateIndex, i);
        valueProcessor.processValue(myagg, inputColumn, row);
    }
}
Also used : Aggregation(org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation)

Example 7 with Aggregation

use of org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation in project hive by apache.

the class VectorUDAFBloomFilter method getAggregationBufferFixedSize.

@Override
public long getAggregationBufferFixedSize() {
    if (bitSetSize < 0) {
        // Not pretty, but we need a way to get the size
        try {
            Aggregation agg = (Aggregation) getNewAggregationBuffer();
            bitSetSize = agg.bf.getBitSet().length;
        } catch (Exception e) {
            throw new RuntimeException("Unexpected error while creating AggregationBuffer", e);
        }
    }
    // BloomFilter: object(BitSet: object(data: long[]), numBits: int, numHashFunctions: int)
    JavaDataModel model = JavaDataModel.get();
    long bloomFilterSize = JavaDataModel.alignUp(model.object() + model.lengthForLongArrayOfSize(bitSetSize), model.memoryAlign());
    return JavaDataModel.alignUp(model.object() + bloomFilterSize + model.primitive1() + model.primitive1(), model.memoryAlign());
}
Also used : Aggregation(org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation) JavaDataModel(org.apache.hadoop.hive.ql.util.JavaDataModel) IOException(java.io.IOException) HiveException(org.apache.hadoop.hive.ql.metadata.HiveException)

Example 8 with Aggregation

use of org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation in project hive by apache.

the class VectorUDAFBloomFilter method iterateNoNullsWithAggregationSelection.

private void iterateNoNullsWithAggregationSelection(VectorAggregationBufferRow[] aggregationBufferSets, int aggregateIndex, ColumnVector inputColumn, int batchSize) {
    for (int i = 0; i < batchSize; ++i) {
        Aggregation myagg = getCurrentAggregationBuffer(aggregationBufferSets, aggregateIndex, i);
        valueProcessor.processValue(myagg, inputColumn, i);
    }
}
Also used : Aggregation(org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation)

Example 9 with Aggregation

use of org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation in project hive by apache.

the class VectorUDAFBloomFilter method getCurrentAggregationBuffer.

private Aggregation getCurrentAggregationBuffer(VectorAggregationBufferRow[] aggregationBufferSets, int aggregateIndex, int row) {
    VectorAggregationBufferRow mySet = aggregationBufferSets[row];
    Aggregation myagg = (Aggregation) mySet.getAggregationBuffer(aggregateIndex);
    return myagg;
}
Also used : Aggregation(org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation) VectorAggregationBufferRow(org.apache.hadoop.hive.ql.exec.vector.VectorAggregationBufferRow)

Aggregations

Aggregation (org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.Aggregation)9 IOException (java.io.IOException)2 BytesColumnVector (org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector)2 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)2 ColumnVector (org.apache.hadoop.hive.ql.exec.vector.ColumnVector)1 DecimalColumnVector (org.apache.hadoop.hive.ql.exec.vector.DecimalColumnVector)1 DoubleColumnVector (org.apache.hadoop.hive.ql.exec.vector.DoubleColumnVector)1 LongColumnVector (org.apache.hadoop.hive.ql.exec.vector.LongColumnVector)1 TimestampColumnVector (org.apache.hadoop.hive.ql.exec.vector.TimestampColumnVector)1 VectorAggregationBufferRow (org.apache.hadoop.hive.ql.exec.vector.VectorAggregationBufferRow)1 JavaDataModel (org.apache.hadoop.hive.ql.util.JavaDataModel)1