Search in sources :

Example 21 with Lz4Compressor

use of io.airlift.compress.lz4.Lz4Compressor in project ignite by apache.

the class CompressionProcessorImpl method compressPageLz4.

/**
 * @param compactPage Compacted page.
 * @param compactSize Compacted page size.
 * @param compressLevel Compression level.
 * @return Compressed page.
 */
private ByteBuffer compressPageLz4(ByteBuffer compactPage, int compactSize, int compressLevel) {
    LZ4Compressor compressor = Lz4.getCompressor(compressLevel);
    ByteBuffer compressedPage = compressBuf.get();
    copyPageHeader(compactPage, compressedPage, compactSize);
    compressor.compress(compactPage, compressedPage);
    compactPage.flip();
    compressedPage.flip();
    return compressedPage;
}
Also used : LZ4Compressor(net.jpountz.lz4.LZ4Compressor) ByteBuffer(java.nio.ByteBuffer)

Example 22 with Lz4Compressor

use of io.airlift.compress.lz4.Lz4Compressor in project druid by druid-io.

the class LZ4Transcoder method compress.

@Override
protected byte[] compress(byte[] in) {
    if (in == null) {
        throw new NullPointerException("Can't compress null");
    }
    LZ4Compressor compressor = lz4Factory.fastCompressor();
    byte[] out = new byte[compressor.maxCompressedLength(in.length)];
    int compressedLength = compressor.compress(in, 0, in.length, out, 0);
    getLogger().debug("Compressed %d bytes to %d", in.length, compressedLength);
    return ByteBuffer.allocate(Integer.BYTES + compressedLength).putInt(in.length).put(out, 0, compressedLength).array();
}
Also used : LZ4Compressor(net.jpountz.lz4.LZ4Compressor)

Aggregations

IOException (java.io.IOException)11 LZ4Compressor (net.jpountz.lz4.LZ4Compressor)10 Lz4Compressor (org.apache.hadoop.io.compress.lz4.Lz4Compressor)10 Test (org.junit.Test)10 Lz4Decompressor (org.apache.hadoop.io.compress.lz4.Lz4Decompressor)5 ByteBuffer (java.nio.ByteBuffer)3 Lz4Compressor (io.airlift.compress.lz4.Lz4Compressor)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)2 DataInputStream (java.io.DataInputStream)2 DataOutputStream (java.io.DataOutputStream)2 LZ4Exception (net.jpountz.lz4.LZ4Exception)2 BlockCompressorStream (org.apache.hadoop.io.compress.BlockCompressorStream)2 BlockDecompressorStream (org.apache.hadoop.io.compress.BlockDecompressorStream)2 DeflateCompressor (com.facebook.presto.orc.zlib.DeflateCompressor)1 InflateDecompressor (com.facebook.presto.orc.zlib.InflateDecompressor)1 ZstdJniCompressor (com.facebook.presto.orc.zstd.ZstdJniCompressor)1 ZstdJniDecompressor (com.facebook.presto.orc.zstd.ZstdJniDecompressor)1 PrestoException (com.facebook.presto.spi.PrestoException)1 PageCompressor (com.facebook.presto.spi.page.PageCompressor)1 PageDecompressor (com.facebook.presto.spi.page.PageDecompressor)1