Search in sources :

Example 41 with SliceOutput

use of io.airlift.slice.SliceOutput in project presto by prestodb.

the class VariableWidthBlock method copyPositions.

@Override
public Block copyPositions(int[] positions, int offset, int length) {
    checkArrayRange(positions, offset, length);
    int finalLength = 0;
    for (int i = offset; i < offset + length; i++) {
        finalLength += getSliceLength(positions[i]);
    }
    SliceOutput newSlice = Slices.allocate(finalLength).getOutput();
    int[] newOffsets = new int[length + 1];
    boolean[] newValueIsNull = null;
    if (valueIsNull != null) {
        newValueIsNull = new boolean[length];
    }
    for (int i = 0; i < length; i++) {
        int position = positions[offset + i];
        if (!isEntryNull(position)) {
            newSlice.writeBytes(slice, getPositionOffset(position), getSliceLength(position));
        } else if (newValueIsNull != null) {
            newValueIsNull[i] = true;
        }
        newOffsets[i + 1] = newSlice.size();
    }
    return new VariableWidthBlock(0, length, newSlice.slice(), newOffsets, newValueIsNull);
}
Also used : SliceOutput(io.airlift.slice.SliceOutput)

Example 42 with SliceOutput

use of io.airlift.slice.SliceOutput in project presto by prestodb.

the class AbstractTestHiveFileFormats method blockToSlice.

private Slice blockToSlice(Block block) {
    // This function is strictly for testing use only
    SliceOutput sliceOutput = new DynamicSliceOutput(1000);
    BlockSerdeUtil.writeBlock(blockEncodingSerde, sliceOutput, block);
    return sliceOutput.slice();
}
Also used : SliceOutput(io.airlift.slice.SliceOutput) DynamicSliceOutput(io.airlift.slice.DynamicSliceOutput) DynamicSliceOutput(io.airlift.slice.DynamicSliceOutput)

Example 43 with SliceOutput

use of io.airlift.slice.SliceOutput in project presto by prestodb.

the class PagesResponseWriter method writeTo.

@Override
public void writeTo(List<SerializedPage> serializedPages, Class<?> type, Type genericType, Annotation[] annotations, MediaType mediaType, MultivaluedMap<String, Object> httpHeaders, OutputStream output) throws IOException, WebApplicationException {
    try {
        SliceOutput sliceOutput = new OutputStreamSliceOutput(output);
        writeSerializedPages(sliceOutput, serializedPages);
        // We use flush instead of close, because the underlying stream would be closed and that is not allowed.
        sliceOutput.flush();
    } catch (UncheckedIOException e) {
        // This is not a "server" problem so we don't want to log this
        if (!(e.getCause() instanceof EOFException)) {
            throw e;
        }
    }
}
Also used : SliceOutput(io.airlift.slice.SliceOutput) OutputStreamSliceOutput(io.airlift.slice.OutputStreamSliceOutput) EOFException(java.io.EOFException) UncheckedIOException(java.io.UncheckedIOException) OutputStreamSliceOutput(io.airlift.slice.OutputStreamSliceOutput)

Example 44 with SliceOutput

use of io.airlift.slice.SliceOutput in project presto by prestodb.

the class FileFragmentResultCacheManager method cachePages.

private void cachePages(CacheKey key, Path path, List<Page> pages, long resultSize) {
    try {
        Files.createFile(path);
        try (SliceOutput output = new OutputStreamSliceOutput(newOutputStream(path, APPEND))) {
            writePages(pagesSerdeFactory.createPagesSerde(), output, pages.iterator());
            long resultPhysicalBytes = output.size();
            cache.put(key, new CacheEntry(path, resultPhysicalBytes));
            fragmentCacheStats.incrementCacheEntries();
            fragmentCacheStats.addCacheSizeInBytes(resultPhysicalBytes);
        } catch (UncheckedIOException | IOException e) {
            log.warn(e, "%s encountered an error while writing to path %s", Thread.currentThread().getName(), path);
            tryDeleteFile(path);
        }
    } catch (UncheckedIOException | IOException e) {
        log.warn(e, "%s encountered an error while writing to path %s", Thread.currentThread().getName(), path);
        tryDeleteFile(path);
    } finally {
        fragmentCacheStats.addInFlightBytes(-resultSize);
    }
}
Also used : SliceOutput(io.airlift.slice.SliceOutput) OutputStreamSliceOutput(io.airlift.slice.OutputStreamSliceOutput) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) OutputStreamSliceOutput(io.airlift.slice.OutputStreamSliceOutput)

Example 45 with SliceOutput

use of io.airlift.slice.SliceOutput in project presto by prestodb.

the class TestOrcMetadataReader method concatSlice.

static Slice concatSlice(Slice... slices) {
    int totalLength = Arrays.stream(slices).mapToInt(Slice::length).sum();
    Slice value = Slices.allocate(totalLength);
    SliceOutput output = value.getOutput();
    for (Slice slice : slices) {
        output.writeBytes(slice);
    }
    return value;
}
Also used : SliceOutput(io.airlift.slice.SliceOutput) Slice(io.airlift.slice.Slice) Slices.utf8Slice(io.airlift.slice.Slices.utf8Slice)

Aggregations

SliceOutput (io.airlift.slice.SliceOutput)46 DynamicSliceOutput (io.airlift.slice.DynamicSliceOutput)27 Slice (io.airlift.slice.Slice)16 IOException (java.io.IOException)8 Test (org.testng.annotations.Test)6 Block (com.facebook.presto.common.block.Block)5 UncheckedIOException (java.io.UncheckedIOException)5 LiteralParameters (com.facebook.presto.spi.function.LiteralParameters)4 SqlType (com.facebook.presto.spi.function.SqlType)4 UsedByGeneratedCode (com.facebook.presto.annotation.UsedByGeneratedCode)2 Page (com.facebook.presto.common.Page)2 BlockBuilder (com.facebook.presto.common.block.BlockBuilder)2 PrestoException (com.facebook.presto.spi.PrestoException)2 ScalarFunction (com.facebook.presto.spi.function.ScalarFunction)2 SqlNullable (com.facebook.presto.spi.function.SqlNullable)2 ArithmeticUnaryExpression (com.facebook.presto.sql.tree.ArithmeticUnaryExpression)2 BooleanLiteral (com.facebook.presto.sql.tree.BooleanLiteral)2 Cast (com.facebook.presto.sql.tree.Cast)2 JsonUtil.createJsonGenerator (com.facebook.presto.util.JsonUtil.createJsonGenerator)2 JsonGenerator (com.fasterxml.jackson.core.JsonGenerator)2