Examples with Lz4Compressor org.apache.hadoop.io.compress.lz4.Lz4Compressor used on opensource projects

Example 16 with Lz4Compressor

use of org.apache.hadoop.io.compress.lz4.Lz4Compressor in project hadoop by apache.

the class TestLz4CompressorDecompressor method testCompressorSetInputAIOBException.

//test on ArrayIndexOutOfBoundsException in {@code compressor.setInput()}
@Test
public void testCompressorSetInputAIOBException() {
    try {
        Lz4Compressor compressor = new Lz4Compressor();
        compressor.setInput(new byte[] {}, -5, 10);
        fail("testCompressorSetInputAIOBException error !!!");
    } catch (ArrayIndexOutOfBoundsException ex) {
    // expected
    } catch (Exception ex) {
        fail("testCompressorSetInputAIOBException ex error !!!");
    }
}

Example 17 with Lz4Compressor

use of org.apache.hadoop.io.compress.lz4.Lz4Compressor in project hadoop by apache.

the class TestLz4CompressorDecompressor method testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize.

// test Lz4Compressor compressor.compress()  
@Test
public void testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize() {
    int BYTES_SIZE = 1024 * 64 + 1;
    try {
        Lz4Compressor compressor = new Lz4Compressor();
        byte[] bytes = generate(BYTES_SIZE);
        assertTrue("needsInput error !!!", compressor.needsInput());
        compressor.setInput(bytes, 0, bytes.length);
        byte[] emptyBytes = new byte[BYTES_SIZE];
        int csize = compressor.compress(emptyBytes, 0, bytes.length);
        assertTrue("testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize error !!!", csize != 0);
    } catch (Exception ex) {
        fail("testSetInputWithBytesSizeMoreThenDefaultLz4CompressorByfferSize ex error !!!");
    }
}

Example 18 with Lz4Compressor

use of org.apache.hadoop.io.compress.lz4.Lz4Compressor in project hadoop by apache.

the class TestLz4CompressorDecompressor method testCompressorCompressNullPointerException.

//test on NullPointerException in {@code compressor.compress()}  
@Test
public void testCompressorCompressNullPointerException() {
    try {
        Lz4Compressor compressor = new Lz4Compressor();
        byte[] bytes = generate(1024 * 6);
        compressor.setInput(bytes, 0, bytes.length);
        compressor.compress(null, 0, 0);
        fail("testCompressorCompressNullPointerException error !!!");
    } catch (NullPointerException ex) {
    // expected
    } catch (Exception e) {
        fail("testCompressorCompressNullPointerException ex error !!!");
    }
}

Example 19 with Lz4Compressor

use of org.apache.hadoop.io.compress.lz4.Lz4Compressor in project hadoop by apache.

the class TestLz4CompressorDecompressor method testCompressorSetInputNullPointerException.

//test on NullPointerException in {@code compressor.setInput()} 
@Test
public void testCompressorSetInputNullPointerException() {
    try {
        Lz4Compressor compressor = new Lz4Compressor();
        compressor.setInput(null, 0, 10);
        fail("testCompressorSetInputNullPointerException error !!!");
    } catch (NullPointerException ex) {
    // expected
    } catch (Exception e) {
        fail("testCompressorSetInputNullPointerException ex error !!!");
    }
}

Example 20 with Lz4Compressor

use of org.apache.hadoop.io.compress.lz4.Lz4Compressor in project vespa by vespa-engine.

the class Compressor method compress.

/**
 * Compresses some data
 *
 * @param requestedCompression the desired compression type, which will be used if the data is deemed suitable.
 *                             Not all the existing types are actually supported.
 * @param data the data to compress. This array is only read by this method.
 * @param uncompressedSize uncompressedSize the size in bytes of the data array. If this is not present, it is
 *                         assumed that the size is the same as the data array size, i.e that it is completely
 *                         filled with uncompressed data.
 * @return the compression result
 * @throws IllegalArgumentException if the compression type is not supported
 */
public Compression compress(CompressionType requestedCompression, byte[] data, Optional<Integer> uncompressedSize) {
    switch(requestedCompression) {
        case NONE:
            data = uncompressedSize.isPresent() ? Arrays.copyOf(data, uncompressedSize.get()) : data;
            return new Compression(CompressionType.NONE, data.length, data);
        case LZ4:
            int dataSize = uncompressedSize.isPresent() ? uncompressedSize.get() : data.length;
            if (dataSize < compressMinSizeBytes)
                return new Compression(CompressionType.INCOMPRESSIBLE, dataSize, data);
            LZ4Compressor compressor = level < 7 ? factory.fastCompressor() : factory.highCompressor();
            byte[] compressedData = compressor.compress(data, 0, dataSize);
            if (compressedData.length + 8 >= dataSize * compressionThresholdFactor)
                return new Compression(CompressionType.INCOMPRESSIBLE, dataSize, data);
            return new Compression(CompressionType.LZ4, dataSize, compressedData);
        default:
            throw new IllegalArgumentException(requestedCompression + " is not supported");
    }
}