Search in sources :

Example 6 with SharedBuffer

use of com.facebook.presto.orc.stream.SharedBuffer in project presto by prestodb.

the class StripeReader method readStripeFooter.

public StripeFooter readStripeFooter(StripeId stripeId, StripeInformation stripe, OrcAggregatedMemoryContext systemMemoryUsage) throws IOException {
    long footerOffset = stripe.getOffset() + stripe.getIndexLength() + stripe.getDataLength();
    int footerLength = toIntExact(stripe.getFooterLength());
    // read the footer
    Slice footerSlice = stripeMetadataSource.getStripeFooterSlice(orcDataSource, stripeId, footerOffset, footerLength, cacheable);
    try (InputStream inputStream = new OrcInputStream(orcDataSource.getId(), // Memory is not accounted as the buffer is expected to be tiny and will be immediately discarded
    new SharedBuffer(NOOP_ORC_LOCAL_MEMORY_CONTEXT), footerSlice.getInput(), decompressor, Optional.empty(), systemMemoryUsage, footerLength)) {
        return metadataReader.readStripeFooter(orcDataSource.getId(), types, inputStream);
    }
}
Also used : SharedBuffer(com.facebook.presto.orc.stream.SharedBuffer) OrcInputStream(com.facebook.presto.orc.stream.OrcInputStream) Slice(io.airlift.slice.Slice) ValueInputStream(com.facebook.presto.orc.stream.ValueInputStream) OrcInputStream(com.facebook.presto.orc.stream.OrcInputStream) InputStream(java.io.InputStream) Checkpoints.getDictionaryStreamCheckpoint(com.facebook.presto.orc.checkpoint.Checkpoints.getDictionaryStreamCheckpoint) StreamCheckpoint(com.facebook.presto.orc.checkpoint.StreamCheckpoint)

Aggregations

SharedBuffer (com.facebook.presto.orc.stream.SharedBuffer)6 OrcInputStream (com.facebook.presto.orc.stream.OrcInputStream)4 InputStream (java.io.InputStream)3 Checkpoints.getDictionaryStreamCheckpoint (com.facebook.presto.orc.checkpoint.Checkpoints.getDictionaryStreamCheckpoint)2 StreamCheckpoint (com.facebook.presto.orc.checkpoint.StreamCheckpoint)2 StripeInformation (com.facebook.presto.orc.metadata.StripeInformation)2 DwrfProto (com.facebook.presto.orc.proto.DwrfProto)2 CodedInputStream (com.facebook.presto.orc.protobuf.CodedInputStream)2 Slice (io.airlift.slice.Slice)2 IOException (java.io.IOException)2 RuntimeStats (com.facebook.presto.common.RuntimeStats)1 BlockBuilder (com.facebook.presto.common.block.BlockBuilder)1 DwrfDataEncryptor (com.facebook.presto.orc.DwrfDataEncryptor)1 OrcCorruptionException (com.facebook.presto.orc.OrcCorruptionException)1 OrcDecompressor.createOrcDecompressor (com.facebook.presto.orc.OrcDecompressor.createOrcDecompressor)1 StorageOrcFileTailSource (com.facebook.presto.orc.cache.StorageOrcFileTailSource)1 Footer (com.facebook.presto.orc.metadata.Footer)1 OrcMetadataReader.byteStringToSlice (com.facebook.presto.orc.metadata.OrcMetadataReader.byteStringToSlice)1 StripeFooter (com.facebook.presto.orc.metadata.StripeFooter)1 ColumnStatistics (com.facebook.presto.orc.metadata.statistics.ColumnStatistics)1