Search in sources :

Example 81 with Checksum

use of java.util.zip.Checksum in project netty by netty.

the class Lz4FrameEncoderTest method newEncoder.

private Lz4FrameEncoder newEncoder(int blockSize, int maxEncodeSize) {
    Checksum checksum = XXHashFactory.fastestInstance().newStreamingHash32(DEFAULT_SEED).asChecksum();
    Lz4FrameEncoder encoder = new Lz4FrameEncoder(LZ4Factory.fastestInstance(), true, blockSize, checksum, maxEncodeSize);
    encoder.handlerAdded(ctx);
    return encoder;
}
Also used : Checksum(java.util.zip.Checksum)

Example 82 with Checksum

use of java.util.zip.Checksum in project neo4j by neo4j.

the class ReadAheadChannelTest method throwOnInvalidChecksum.

@ParameterizedTest
@EnumSource(Constructors.class)
void throwOnInvalidChecksum(Constructor constructor) throws Exception {
    // given
    Checksum checksum = CHECKSUM_FACTORY.get();
    Path file = Path.of("foo.1");
    try (StoreChannel storeChannel = fileSystem.write(file)) {
        ByteBuffer buffer = ByteBuffers.allocate(6, INSTANCE);
        buffer.put((byte) 1);
        checksum.update(1);
        buffer.put((byte) 2);
        checksum.update(2);
        int notChecksumValue = (int) checksum.getValue() + 1;
        buffer.putInt(notChecksumValue);
        buffer.flip();
        storeChannel.writeAll(buffer);
        storeChannel.force(false);
    }
    ReadAheadChannel<StoreChannel> bufferedReader = constructor.apply(fileSystem.read(file), DEFAULT_READ_AHEAD_SIZE);
    assertEquals(1, bufferedReader.get());
    assertEquals(2, bufferedReader.get());
    assertThrows(ChecksumMismatchException.class, bufferedReader::endChecksumAndValidate);
}
Also used : Path(java.nio.file.Path) Checksum(java.util.zip.Checksum) StoreChannel(org.neo4j.io.fs.StoreChannel) ByteBuffer(java.nio.ByteBuffer) EnumSource(org.junit.jupiter.params.provider.EnumSource) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 83 with Checksum

use of java.util.zip.Checksum in project neo4j by neo4j.

the class PhysicalFlushableChecksumChannelTest method beginCehcksumShouldResetCalculations.

@Test
void beginCehcksumShouldResetCalculations() throws IOException {
    final Path firstFile = directory.homePath().resolve("file1");
    StoreChannel storeChannel = fileSystem.write(firstFile);
    int channelChecksum;
    try (PhysicalFlushableChecksumChannel channel = new PhysicalFlushableChecksumChannel(storeChannel, new HeapScopedBuffer(100, INSTANCE))) {
        channel.put((byte) 5);
        channel.beginChecksum();
        channel.put((byte) 10);
        channelChecksum = channel.putChecksum();
    }
    int fileSize = (int) fileSystem.getFileSize(firstFile);
    assertEquals(Byte.BYTES + Byte.BYTES + Integer.BYTES, fileSize);
    byte[] writtenBytes = new byte[fileSize];
    try (InputStream in = Files.newInputStream(firstFile)) {
        in.read(writtenBytes);
    }
    ByteBuffer buffer = ByteBuffer.wrap(writtenBytes);
    Checksum checksum = CHECKSUM_FACTORY.get();
    checksum.update(10);
    assertEquals(checksum.getValue(), channelChecksum);
    assertEquals(5, buffer.get());
    assertEquals(10, buffer.get());
    assertEquals(checksum.getValue(), buffer.getInt());
}
Also used : Path(java.nio.file.Path) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) PhysicalFlushableChecksumChannel(org.neo4j.io.fs.PhysicalFlushableChecksumChannel) InputStream(java.io.InputStream) Checksum(java.util.zip.Checksum) StoreChannel(org.neo4j.io.fs.StoreChannel) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.jupiter.api.Test)

Example 84 with Checksum

use of java.util.zip.Checksum in project neo4j by neo4j.

the class PhysicalFlushableChecksumChannelTest method calculateChecksum.

@Test
void calculateChecksum() throws IOException {
    final Path firstFile = directory.homePath().resolve("file1");
    StoreChannel storeChannel = fileSystem.write(firstFile);
    int channelChecksum;
    try (PhysicalFlushableChecksumChannel channel = new PhysicalFlushableChecksumChannel(storeChannel, new HeapScopedBuffer(100, INSTANCE))) {
        channel.beginChecksum();
        channel.put((byte) 10);
        channelChecksum = channel.putChecksum();
    }
    int fileSize = (int) fileSystem.getFileSize(firstFile);
    assertEquals(Byte.BYTES + Integer.BYTES, fileSize);
    byte[] writtenBytes = new byte[fileSize];
    try (InputStream in = Files.newInputStream(firstFile)) {
        in.read(writtenBytes);
    }
    ByteBuffer buffer = ByteBuffer.wrap(writtenBytes);
    Checksum checksum = CHECKSUM_FACTORY.get();
    checksum.update(10);
    assertEquals(checksum.getValue(), channelChecksum);
    assertEquals(10, buffer.get());
    assertEquals(checksum.getValue(), buffer.getInt());
}
Also used : Path(java.nio.file.Path) HeapScopedBuffer(org.neo4j.io.memory.HeapScopedBuffer) PhysicalFlushableChecksumChannel(org.neo4j.io.fs.PhysicalFlushableChecksumChannel) InputStream(java.io.InputStream) Checksum(java.util.zip.Checksum) StoreChannel(org.neo4j.io.fs.StoreChannel) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.jupiter.api.Test)

Aggregations

Checksum (java.util.zip.Checksum)84 CRC32 (java.util.zip.CRC32)29 IOException (java.io.IOException)16 ByteBuffer (java.nio.ByteBuffer)15 Adler32 (java.util.zip.Adler32)12 File (java.io.File)8 InputStream (java.io.InputStream)7 FileInputStream (java.io.FileInputStream)6 Path (java.nio.file.Path)6 EOFException (java.io.EOFException)5 Test (org.junit.Test)5 Test (org.junit.jupiter.api.Test)5 StoreChannel (org.neo4j.io.fs.StoreChannel)5 CheckedInputStream (java.util.zip.CheckedInputStream)4 BufferedOutputStream (java.io.BufferedOutputStream)3 ByteArrayInputStream (java.io.ByteArrayInputStream)3 UnsupportedEncodingException (java.io.UnsupportedEncodingException)3 ArrayList (java.util.ArrayList)3 PureJavaCrc32 (org.apache.hadoop.util.PureJavaCrc32)3 BinaryInputArchive (org.apache.jute.BinaryInputArchive)3