Search in sources :

Example 6 with CheckedOutputStream

use of java.util.zip.CheckedOutputStream in project voltdb by VoltDB.

the class FileSnap method serialize.

/**
     * serialize the datatree and session into the file snapshot
     * @param dt the datatree to be serialized
     * @param sessions the sessions to be serialized
     * @param snapShot the file to store snapshot into
     */
@Override
public synchronized void serialize(DataTree dt, Map<Long, Long> sessions, File snapShot) throws IOException {
    if (!close) {
        OutputStream sessOS = new BufferedOutputStream(new FileOutputStream(snapShot));
        CheckedOutputStream crcOut = new CheckedOutputStream(sessOS, new Adler32());
        //CheckedOutputStream cout = new CheckedOutputStream()
        OutputArchive oa = BinaryOutputArchive.getArchive(crcOut);
        FileHeader header = new FileHeader(SNAP_MAGIC, VERSION, dbId);
        serialize(dt, sessions, oa, header);
        long val = crcOut.getChecksum().getValue();
        oa.writeLong(val, "val");
        oa.writeString("/", "path");
        sessOS.flush();
        crcOut.close();
        sessOS.close();
    }
}
Also used : BinaryOutputArchive(org.apache.jute_voltpatches.BinaryOutputArchive) OutputArchive(org.apache.jute_voltpatches.OutputArchive) BufferedOutputStream(java.io.BufferedOutputStream) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream) FileOutputStream(java.io.FileOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream) BufferedOutputStream(java.io.BufferedOutputStream) FileHeader(org.apache.zookeeper_voltpatches.server.persistence.FileHeader) Adler32(java.util.zip.Adler32)

Example 7 with CheckedOutputStream

use of java.util.zip.CheckedOutputStream in project processdash by dtuma.

the class ListingHashcodeCalculator method getListingHashcode.

/**
     * Computes a hashcode for the items in a resource collection.
     * 
     * The hashcode is sensitive to:
     * <ul>
     * <li>The names of the resources that appear both in the collection and in
     * the <code>resourceNames</code> parameter</li>
     * <li>The checksums of those files in the resource collection</li>
     * </ul>
     * 
     * Thus, if the list of resources in the collection changes, or if the
     * contents of a resource change, this hashcode is likely to change as well.
     * 
     * @param collection
     *                the resource collection to examine
     * @param resourceNames
     *                the names of resources to include in the hash
     * @return a hashcode for the names and checksums of the listed files
     */
public static long getListingHashcode(ResourceCollectionInfo collection, List<String> resourceNames) {
    String[] sortedNames = new String[resourceNames.size()];
    for (int i = 0; i < sortedNames.length; i++) {
        sortedNames[i] = resourceNames.get(i).toLowerCase();
    }
    Arrays.sort(sortedNames);
    Adler32 cksum = new Adler32();
    try {
        DataOutputStream out = new DataOutputStream(new CheckedOutputStream(NULL_OUT, cksum));
        for (String resourceName : sortedNames) {
            long lastMod = collection.getLastModified(resourceName);
            if (lastMod < 1)
                continue;
            Long checksum = collection.getChecksum(resourceName);
            if (checksum == null)
                continue;
            out.writeUTF(resourceName);
            out.writeLong(checksum);
        }
    } catch (IOException e) {
    // can't happen
    }
    return cksum.getValue();
}
Also used : DataOutputStream(java.io.DataOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream) IOException(java.io.IOException) Adler32(java.util.zip.Adler32)

Example 8 with CheckedOutputStream

use of java.util.zip.CheckedOutputStream in project elasticsearch by elastic.

the class CompressedXContent method crc32.

private static int crc32(BytesReference data) {
    OutputStream dummy = new OutputStream() {

        @Override
        public void write(int b) throws IOException {
        // no-op
        }

        @Override
        public void write(byte[] b, int off, int len) throws IOException {
        // no-op
        }
    };
    CRC32 crc32 = new CRC32();
    try {
        data.writeTo(new CheckedOutputStream(dummy, crc32));
    } catch (IOException bogus) {
        // cannot happen
        throw new Error(bogus);
    }
    return (int) crc32.getValue();
}
Also used : CRC32(java.util.zip.CRC32) OutputStream(java.io.OutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream) IOException(java.io.IOException)

Example 9 with CheckedOutputStream

use of java.util.zip.CheckedOutputStream in project hadoop by apache.

the class TestIndexCache method writeFile.

private static void writeFile(FileSystem fs, Path f, long fill, int parts) throws IOException {
    FSDataOutputStream out = fs.create(f, false);
    CheckedOutputStream iout = new CheckedOutputStream(out, new CRC32());
    DataOutputStream dout = new DataOutputStream(iout);
    for (int i = 0; i < parts; ++i) {
        for (int j = 0; j < MapTask.MAP_OUTPUT_INDEX_RECORD_LENGTH / 8; ++j) {
            dout.writeLong(fill);
        }
    }
    out.writeLong(iout.getChecksum().getValue());
    dout.close();
}
Also used : CRC32(java.util.zip.CRC32) FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) DataOutputStream(java.io.DataOutputStream) FSDataOutputStream(org.apache.hadoop.fs.FSDataOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream)

Example 10 with CheckedOutputStream

use of java.util.zip.CheckedOutputStream in project jdk8u_jdk by JetBrains.

the class ZipMeUp method getManifestAsBytes.

static byte[] getManifestAsBytes(int nchars) throws IOException {
    crc.reset();
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    CheckedOutputStream cos = new CheckedOutputStream(baos, crc);
    PrintStream ps = new PrintStream(cos);
    ps.println("Manifest-Version: 1.0");
    ps.print("Main-Class: ");
    for (int i = 0; i < nchars - SOME_KLASS.length(); i++) {
        ps.print(i % 10);
    }
    ps.println(SOME_KLASS);
    cos.flush();
    cos.close();
    ps.close();
    return baos.toByteArray();
}
Also used : PrintStream(java.io.PrintStream) ByteArrayOutputStream(java.io.ByteArrayOutputStream) CheckedOutputStream(java.util.zip.CheckedOutputStream)

Aggregations

CheckedOutputStream (java.util.zip.CheckedOutputStream)10 IOException (java.io.IOException)4 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)4 DataOutputStream (java.io.DataOutputStream)3 OutputStream (java.io.OutputStream)3 Adler32 (java.util.zip.Adler32)3 CRC32 (java.util.zip.CRC32)3 BufferedOutputStream (java.io.BufferedOutputStream)2 FileOutputStream (java.io.FileOutputStream)2 Path (org.apache.hadoop.fs.Path)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 InputStream (java.io.InputStream)1 PrintStream (java.io.PrintStream)1 JarEntry (java.util.jar.JarEntry)1 Checksum (java.util.zip.Checksum)1 ChecksumException (org.apache.hadoop.fs.ChecksumException)1 FileSystem (org.apache.hadoop.fs.FileSystem)1 PureJavaCrc32 (org.apache.hadoop.util.PureJavaCrc32)1 BinaryOutputArchive (org.apache.jute.BinaryOutputArchive)1 OutputArchive (org.apache.jute.OutputArchive)1