Search in sources :

Example 11 with VariableWidthBlock

use of com.facebook.presto.common.block.VariableWidthBlock in project presto by prestodb.

the class SliceDirectSelectiveStreamReader method getBlock.

@Override
public Block getBlock(int[] positions, int positionCount) {
    checkArgument(outputPositionCount > 0, "outputPositionCount must be greater than zero");
    checkState(outputRequired, "This stream reader doesn't produce output");
    checkState(positionCount <= outputPositionCount, "Not enough values");
    checkState(!valuesInUse, "BlockLease hasn't been closed yet");
    if (allNulls) {
        return new RunLengthEncodedBlock(outputType.createBlockBuilder(null, 1).appendNull().build(), positionCount);
    }
    boolean includeNulls = nullsAllowed && presentStream != null;
    if (positionCount != outputPositionCount) {
        compactValues(positions, positionCount, includeNulls);
    }
    Block block = new VariableWidthBlock(positionCount, dataAsSlice, offsets, Optional.ofNullable(includeNulls ? nulls : null));
    dataAsSlice = null;
    data = null;
    offsets = null;
    nulls = null;
    return block;
}
Also used : RunLengthEncodedBlock(com.facebook.presto.common.block.RunLengthEncodedBlock) VariableWidthBlock(com.facebook.presto.common.block.VariableWidthBlock) Block(com.facebook.presto.common.block.Block) RunLengthEncodedBlock(com.facebook.presto.common.block.RunLengthEncodedBlock) VariableWidthBlock(com.facebook.presto.common.block.VariableWidthBlock)

Example 12 with VariableWidthBlock

use of com.facebook.presto.common.block.VariableWidthBlock in project presto by prestodb.

the class SliceDictionaryBatchStreamReader method setDictionaryBlockData.

private void setDictionaryBlockData(byte[] dictionaryData, int[] dictionaryOffsets, int positionCount) {
    verify(positionCount > 0);
    // the engine currently uses identity equality to test if dictionaries are the same
    if (currentDictionaryData != dictionaryData) {
        boolean[] isNullVector = new boolean[positionCount];
        isNullVector[positionCount - 1] = true;
        dictionaryOffsets[positionCount] = dictionaryOffsets[positionCount - 1];
        dictionaryBlock = new VariableWidthBlock(positionCount, wrappedBuffer(dictionaryData), dictionaryOffsets, Optional.of(isNullVector));
        currentDictionaryData = dictionaryData;
    }
}
Also used : VariableWidthBlock(com.facebook.presto.common.block.VariableWidthBlock)

Example 13 with VariableWidthBlock

use of com.facebook.presto.common.block.VariableWidthBlock in project presto by prestodb.

the class TestOptimizedPartitionedOutputOperator method createVariableWidthBlockOverSliceView.

private static Block createVariableWidthBlockOverSliceView(int entries) {
    // Create a slice view whose address starts in the middle of the original slice, and length is half of original slice
    DynamicSliceOutput dynamicSliceOutput = new DynamicSliceOutput(entries * 2);
    for (int i = 0; i < entries * 2; i++) {
        dynamicSliceOutput.writeByte(i);
    }
    Slice slice = dynamicSliceOutput.slice().slice(entries, entries);
    int[] offsets = IntStream.range(0, entries + 1).toArray();
    return new VariableWidthBlock(entries, slice, offsets, Optional.empty());
}
Also used : Slice(io.airlift.slice.Slice) DynamicSliceOutput(io.airlift.slice.DynamicSliceOutput) VariableWidthBlock(com.facebook.presto.common.block.VariableWidthBlock)

Aggregations

VariableWidthBlock (com.facebook.presto.common.block.VariableWidthBlock)13 Slice (io.airlift.slice.Slice)8 Block (com.facebook.presto.common.block.Block)7 RunLengthEncodedBlock (com.facebook.presto.common.block.RunLengthEncodedBlock)7 BinaryValuesDecoder (com.facebook.presto.parquet.batchreader.decoders.ValuesDecoder.BinaryValuesDecoder)4 ValueBuffer (com.facebook.presto.parquet.batchreader.decoders.ValuesDecoder.BinaryValuesDecoder.ValueBuffer)4 ColumnChunk (com.facebook.presto.parquet.reader.ColumnChunk)4 ArrayList (java.util.ArrayList)4 Test (org.testng.annotations.Test)2 GenericInternalException (com.facebook.presto.common.GenericInternalException)1 Page (com.facebook.presto.common.Page)1 ArrayType (com.facebook.presto.common.type.ArrayType)1 IntegerType (com.facebook.presto.common.type.IntegerType)1 Type (com.facebook.presto.common.type.Type)1 VariableWidthType (com.facebook.presto.common.type.VariableWidthType)1 OrcCorruptionException (com.facebook.presto.orc.OrcCorruptionException)1 ConnectorSession (com.facebook.presto.spi.ConnectorSession)1 TestingConnectorSession (com.facebook.presto.testing.TestingConnectorSession)1 DynamicSliceOutput (io.airlift.slice.DynamicSliceOutput)1 DataSchema (org.apache.pinot.common.utils.DataSchema)1