Search in sources :

Example 21 with ByteBuff

use of org.apache.hadoop.hbase.nio.ByteBuff in project hbase by apache.

the class HFileBlock method unpack.

/**
   * Retrieves the decompressed/decrypted view of this block. An encoded block remains in its
   * encoded structure. Internal structures are shared between instances where applicable.
   */
HFileBlock unpack(HFileContext fileContext, FSReader reader) throws IOException {
    if (!fileContext.isCompressedOrEncrypted()) {
        // encryption details.
        return this;
    }
    HFileBlock unpacked = new HFileBlock(this);
    // allocates space for the decompressed block
    unpacked.allocateBuffer();
    HFileBlockDecodingContext ctx = blockType == BlockType.ENCODED_DATA ? reader.getBlockDecodingContext() : reader.getDefaultBlockDecodingContext();
    ByteBuff dup = this.buf.duplicate();
    dup.position(this.headerSize());
    dup = dup.slice();
    ctx.prepareDecoding(unpacked.getOnDiskSizeWithoutHeader(), unpacked.getUncompressedSizeWithoutHeader(), unpacked.getBufferWithoutHeader(), dup);
    return unpacked;
}
Also used : HFileBlockDecodingContext(org.apache.hadoop.hbase.io.encoding.HFileBlockDecodingContext) MultiByteBuff(org.apache.hadoop.hbase.nio.MultiByteBuff) SingleByteBuff(org.apache.hadoop.hbase.nio.SingleByteBuff) ByteBuff(org.apache.hadoop.hbase.nio.ByteBuff)

Example 22 with ByteBuff

use of org.apache.hadoop.hbase.nio.ByteBuff in project hbase by apache.

the class HFileBlock method getByteStream.

/**
   * @return a byte stream reading the data + checksum of this block
   */
DataInputStream getByteStream() {
    ByteBuff dup = this.buf.duplicate();
    dup.position(this.headerSize());
    return new DataInputStream(new ByteBuffInputStream(dup));
}
Also used : ByteBuffInputStream(org.apache.hadoop.hbase.io.ByteBuffInputStream) MultiByteBuff(org.apache.hadoop.hbase.nio.MultiByteBuff) SingleByteBuff(org.apache.hadoop.hbase.nio.SingleByteBuff) ByteBuff(org.apache.hadoop.hbase.nio.ByteBuff) DataInputStream(java.io.DataInputStream) FSDataInputStream(org.apache.hadoop.fs.FSDataInputStream)

Example 23 with ByteBuff

use of org.apache.hadoop.hbase.nio.ByteBuff in project hbase by apache.

the class CompoundBloomFilter method contains.

@Override
public boolean contains(byte[] key, int keyOffset, int keyLength, ByteBuff bloom) {
    int block = index.rootBlockContainingKey(key, keyOffset, keyLength);
    if (block < 0) {
        // This key is not in the file.
        return false;
    }
    boolean result;
    HFileBlock bloomBlock = getBloomBlock(block);
    try {
        ByteBuff bloomBuf = bloomBlock.getBufferReadOnly();
        result = BloomFilterUtil.contains(key, keyOffset, keyLength, bloomBuf, bloomBlock.headerSize(), bloomBlock.getUncompressedSizeWithoutHeader(), hash, hashCount);
    } finally {
        // After the use return back the block if it was served from a cache.
        reader.returnBlock(bloomBlock);
    }
    if (numPositivesPerChunk != null && result) {
        // Update statistics. Only used in unit tests.
        ++numPositivesPerChunk[block];
    }
    return result;
}
Also used : ByteBuff(org.apache.hadoop.hbase.nio.ByteBuff)

Aggregations

ByteBuff (org.apache.hadoop.hbase.nio.ByteBuff)23 MultiByteBuff (org.apache.hadoop.hbase.nio.MultiByteBuff)9 ByteBuffer (java.nio.ByteBuffer)8 SingleByteBuff (org.apache.hadoop.hbase.nio.SingleByteBuff)8 Test (org.junit.Test)8 ArrayList (java.util.ArrayList)7 FSDataInputStream (org.apache.hadoop.fs.FSDataInputStream)5 Path (org.apache.hadoop.fs.Path)5 DataInputStream (java.io.DataInputStream)3 Random (java.util.Random)3 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)3 Compression (org.apache.hadoop.hbase.io.compress.Compression)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 DataOutputStream (java.io.DataOutputStream)2 File (java.io.File)2 IOException (java.io.IOException)2 Cell (org.apache.hadoop.hbase.Cell)2 KeyValue (org.apache.hadoop.hbase.KeyValue)2 FSDataInputStreamWrapper (org.apache.hadoop.hbase.io.FSDataInputStreamWrapper)2