Examples with SliceInput io.airlift.slice.SliceInput used on opensource projects

Example 6 with SliceInput

use of io.airlift.slice.SliceInput in project presto by prestodb.

the class SpoolingOutputBuffer method getPagesFromStorage.

private ListenableFuture<List<SerializedPage>> getPagesFromStorage(ImmutableList.Builder<SerializedPage> resultBuilder, Iterator<HandleInfo> handleIterator, TempStorageHandle handle, GetTracker getTracker) {
    long maxBytes = getTracker.getMaxSize().toBytes();
    long bytes = getTracker.getBytes();
    long pageCount = getTracker.getPageCount();
    try (SliceInput inputStream = new InputStreamSliceInput(tempStorage.open(tempDataOperationContext, handle))) {
        Iterator<SerializedPage> serializedPages = readSerializedPages(inputStream);
        advance(serializedPages, getTracker.getStartPage());
        while (serializedPages.hasNext()) {
            SerializedPage page = serializedPages.next();
            long bytesRead = bytes;
            bytes += page.getRetainedSizeInBytes();
            if (pageCount != 0 && bytes > maxBytes) {
                getTracker.update(bytesRead, pageCount);
                return immediateFuture(resultBuilder.build());
            }
            resultBuilder.add(page);
            pageCount++;
        }
        getTracker.update(bytes, pageCount);
        if (!handleIterator.hasNext()) {
            return immediateFuture(resultBuilder.build());
        }
        return transformAsync(handleIterator.next().getHandleFuture(), input -> getPagesFromStorage(resultBuilder, handleIterator, input, getTracker), executor);
    } catch (IOException e) {
        throw new PrestoException(SPOOLING_STORAGE_ERROR, "Failed to read file from TempStorage", e);
    }
}

Example 7 with SliceInput

use of io.airlift.slice.SliceInput in project presto by prestodb.

the class DigestAndPercentileStateSerializer method deserialize.

@Override
public void deserialize(Block block, int index, DigestAndPercentileState state) {
    SliceInput input = VARBINARY.getSlice(block, index).getInput();
    // read percentile
    state.setPercentile(input.readDouble());
    // read digest
    int length = input.readInt();
    QuantileDigest digest = new QuantileDigest(input.readSlice(length));
    state.setDigest(digest);
    state.addMemoryUsage(state.getDigest().estimatedInMemorySizeInBytes());
}

Example 8 with SliceInput

use of io.airlift.slice.SliceInput in project presto by prestodb.

the class DifferentialEntropyStateSerializer method deserialize.

@Override
public void deserialize(Block block, int index, DifferentialEntropyState state) {
    SliceInput input = VARBINARY.getSlice(block, index).getInput();
    DifferentialEntropyStateStrategy strategy = DifferentialEntropyStateStrategy.deserialize(input);
    if (strategy != null) {
        state.setStrategy(strategy);
    }
}

Example 9 with SliceInput

use of io.airlift.slice.SliceInput in project presto by prestodb.

the class SetDigest method newInstance.

public static SetDigest newInstance(Slice serialized) {
    requireNonNull(serialized, "serialized is null");
    SliceInput input = serialized.getInput();
    checkArgument(input.readByte() == UNCOMPRESSED_FORMAT, "Unexpected version");
    int hllLength = input.readInt();
    Slice serializedHll = Slices.allocate(hllLength);
    input.readBytes(serializedHll, hllLength);
    HyperLogLog hll = HyperLogLog.newInstance(serializedHll);
    Long2ShortRBTreeMap minhash = new Long2ShortRBTreeMap();
    int maxHashes = input.readInt();
    int minhashLength = input.readInt();
    // The values are stored after the keys
    SliceInput valuesInput = serialized.getInput();
    valuesInput.setPosition(input.position() + minhashLength * SIZE_OF_LONG);
    for (int i = 0; i < minhashLength; i++) {
        minhash.put(input.readLong(), valuesInput.readShort());
    }
    return new SetDigest(maxHashes, hll, minhash);
}