Search in sources :

Example 11 with Checksum

use of java.util.zip.Checksum in project cassandra by apache.

the class IncomingTcpConnection method receiveMessages.

// Not closing constructed DataInputPlus's as the stream needs to remain open.
@SuppressWarnings("resource")
private void receiveMessages() throws IOException {
    // handshake (true) endpoint versions
    DataOutputStream out = new DataOutputStream(socket.getOutputStream());
    // if this version is < the MS version the other node is trying
    // to connect with, the other node will disconnect
    out.writeInt(MessagingService.current_version);
    out.flush();
    DataInputPlus in = new DataInputStreamPlus(socket.getInputStream());
    int maxVersion = in.readInt();
    // outbound side will reconnect if necessary to upgrade version
    assert version <= MessagingService.current_version;
    from = CompactEndpointSerializationHelper.deserialize(in);
    // record the (true) version of the endpoint
    MessagingService.instance().setVersion(from, maxVersion);
    logger.trace("Set version for {} to {} (will use {})", from, maxVersion, MessagingService.instance().getVersion(from));
    if (compressed) {
        logger.trace("Upgrading incoming connection to be compressed");
        LZ4FastDecompressor decompressor = LZ4Factory.fastestInstance().fastDecompressor();
        Checksum checksum = XXHashFactory.fastestInstance().newStreamingHash32(OutboundTcpConnection.LZ4_HASH_SEED).asChecksum();
        in = new DataInputStreamPlus(new LZ4BlockInputStream(socket.getInputStream(), decompressor, checksum));
    } else {
        ReadableByteChannel channel = socket.getChannel();
        in = new NIODataInputStream(channel != null ? channel : Channels.newChannel(socket.getInputStream()), BUFFER_SIZE);
    }
    while (true) {
        MessagingService.validateMagic(in.readInt());
        receiveMessage(in, version);
    }
}
Also used : ReadableByteChannel(java.nio.channels.ReadableByteChannel) LZ4FastDecompressor(net.jpountz.lz4.LZ4FastDecompressor) Checksum(java.util.zip.Checksum) DataInputStreamPlus(org.apache.cassandra.io.util.DataInputPlus.DataInputStreamPlus) DataInputPlus(org.apache.cassandra.io.util.DataInputPlus) LZ4BlockInputStream(net.jpountz.lz4.LZ4BlockInputStream) NIODataInputStream(org.apache.cassandra.io.util.NIODataInputStream)

Example 12 with Checksum

use of java.util.zip.Checksum in project guava by google.

the class ChecksumHashFunctionTest method assertChecksum.

private static void assertChecksum(Supplier<Checksum> supplier, String input) {
    byte[] bytes = HashTestUtils.ascii(input);
    Checksum checksum = supplier.get();
    checksum.update(bytes, 0, bytes.length);
    long value = checksum.getValue();
    String toString = "name";
    HashFunction func = new ChecksumHashFunction(supplier, 32, toString);
    assertEquals(toString, func.toString());
    assertEquals(value, func.hashBytes(bytes).padToLong());
}
Also used : Checksum(java.util.zip.Checksum)

Example 13 with Checksum

use of java.util.zip.Checksum in project lucene-solr by apache.

the class TestBufferedChecksum method testRandom.

public void testRandom() {
    Checksum c1 = new CRC32();
    Checksum c2 = new BufferedChecksum(new CRC32());
    int iterations = atLeast(10000);
    for (int i = 0; i < iterations; i++) {
        switch(random().nextInt(4)) {
            case 0:
                // update(byte[], int, int)
                int length = random().nextInt(1024);
                byte[] bytes = new byte[length];
                random().nextBytes(bytes);
                c1.update(bytes, 0, bytes.length);
                c2.update(bytes, 0, bytes.length);
                break;
            case 1:
                // update(int)
                int b = random().nextInt(256);
                c1.update(b);
                c2.update(b);
                break;
            case 2:
                // reset()
                c1.reset();
                c2.reset();
                break;
            case 3:
                // getValue()
                assertEquals(c1.getValue(), c2.getValue());
                break;
        }
    }
    assertEquals(c1.getValue(), c2.getValue());
}
Also used : CRC32(java.util.zip.CRC32) Checksum(java.util.zip.Checksum)

Example 14 with Checksum

use of java.util.zip.Checksum in project poi by apache.

the class IOUtils method calculateChecksum.

/**
     * Calculate checksum on input data
     */
public static long calculateChecksum(byte[] data) {
    Checksum sum = new CRC32();
    sum.update(data, 0, data.length);
    return sum.getValue();
}
Also used : CRC32(java.util.zip.CRC32) Checksum(java.util.zip.Checksum)

Example 15 with Checksum

use of java.util.zip.Checksum in project cassandra by apache.

the class CommitLogTest method testRecoveryWithBadSizeChecksum.

@Test
public void testRecoveryWithBadSizeChecksum() throws Exception {
    Checksum checksum = new CRC32();
    checksum.update(100);
    testRecoveryWithBadSizeArgument(100, 100, ~checksum.getValue());
}
Also used : CRC32(java.util.zip.CRC32) Checksum(java.util.zip.Checksum)

Aggregations

Checksum (java.util.zip.Checksum)35 CRC32 (java.util.zip.CRC32)14 IOException (java.io.IOException)10 Adler32 (java.util.zip.Adler32)9 File (java.io.File)6 EOFException (java.io.EOFException)3 FileInputStream (java.io.FileInputStream)3 ByteBuffer (java.nio.ByteBuffer)3 ArrayList (java.util.ArrayList)3 BufferedOutputStream (java.io.BufferedOutputStream)2 ByteArrayInputStream (java.io.ByteArrayInputStream)2 FileOutputStream (java.io.FileOutputStream)2 RandomAccessFile (java.io.RandomAccessFile)2 Date (java.util.Date)2 DataChecksum (org.apache.hadoop.util.DataChecksum)2 PureJavaCrc32 (org.apache.hadoop.util.PureJavaCrc32)2 BinaryInputArchive (org.apache.jute.BinaryInputArchive)2 Record (org.apache.jute.Record)2 FileHeader (org.apache.zookeeper_voltpatches.server.persistence.FileHeader)2 ByteBuf (io.netty.buffer.ByteBuf)1