Search in sources :

Example 46 with Output

use of com.esotericsoftware.kryo.io.Output in project jetcache by alibaba.

the class KryoValueEncoder method apply.

@Override
public byte[] apply(Object value) {
    try {
        Kryo kryo = kryoThreadLocal.get();
        ByteArrayOutputStream bos = new ByteArrayOutputStream(256);
        Output output = new Output(bos);
        output.writeInt(IDENTITY_NUMBER);
        kryo.writeClassAndObject(output, value);
        output.close();
        return bos.toByteArray();
    } catch (Exception e) {
        StringBuilder sb = new StringBuilder("Kryo Encode error. ");
        sb.append("msg=").append(e.getMessage());
        throw new CacheEncodeException(sb.toString(), e);
    }
}
Also used : Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo)

Example 47 with Output

use of com.esotericsoftware.kryo.io.Output in project beam by apache.

the class AvroCoderTest method testKryoSerialization.

/**
   * Confirm that we can serialize and deserialize an AvroCoder object using Kryo.
   * (BEAM-626).
   *
   * @throws Exception
   */
@Test
public void testKryoSerialization() throws Exception {
    Pojo value = new Pojo("Hello", 42);
    AvroCoder<Pojo> coder = AvroCoder.of(Pojo.class);
    //Kryo instantiation
    Kryo kryo = new Kryo();
    kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
    //Serialization of object without any memoization
    ByteArrayOutputStream coderWithoutMemoizationBos = new ByteArrayOutputStream();
    try (Output output = new Output(coderWithoutMemoizationBos)) {
        kryo.writeObject(output, coder);
    }
    // Force thread local memoization to store values.
    CoderProperties.coderDecodeEncodeEqual(coder, value);
    // Serialization of object with memoized fields
    ByteArrayOutputStream coderWithMemoizationBos = new ByteArrayOutputStream();
    try (Output output = new Output(coderWithMemoizationBos)) {
        kryo.writeObject(output, coder);
    }
    // Copy empty and memoized variants of the Coder
    ByteArrayInputStream bisWithoutMemoization = new ByteArrayInputStream(coderWithoutMemoizationBos.toByteArray());
    AvroCoder<Pojo> copiedWithoutMemoization = (AvroCoder<Pojo>) kryo.readObject(new Input(bisWithoutMemoization), AvroCoder.class);
    ByteArrayInputStream bisWithMemoization = new ByteArrayInputStream(coderWithMemoizationBos.toByteArray());
    AvroCoder<Pojo> copiedWithMemoization = (AvroCoder<Pojo>) kryo.readObject(new Input(bisWithMemoization), AvroCoder.class);
    CoderProperties.coderDecodeEncodeEqual(copiedWithoutMemoization, value);
    CoderProperties.coderDecodeEncodeEqual(copiedWithMemoization, value);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) StdInstantiatorStrategy(org.objenesis.strategy.StdInstantiatorStrategy) ByteArrayInputStream(java.io.ByteArrayInputStream) Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Example 48 with Output

use of com.esotericsoftware.kryo.io.Output in project apex-core by apache.

the class Journal method write.

final void write(Recoverable op) {
    if (replayMode.get()) {
        throw new IllegalStateException("Request to write while journal is replaying operations");
    }
    Integer classId = RecoverableOperation.getId(op.getClass());
    if (classId == null) {
        throw new IllegalArgumentException("Class not registered " + op.getClass());
    }
    while (true) {
        final Output out = output.get();
        if (out != null) {
            // need to atomically write id, operation and flush the output stream
            synchronized (out) {
                try {
                    LOG.debug("WAL write {}", RecoverableOperation.get(classId));
                    out.writeInt(classId);
                    op.write(out);
                    out.flush();
                    break;
                } catch (KryoException e) {
                    // stream or null leading to the current stream being closed
                    if (output.get() == out) {
                        throw e;
                    }
                }
            }
        } else {
            LOG.warn("Journal output stream is null. Skipping write to the WAL.");
            break;
        }
    }
}
Also used : KryoException(com.esotericsoftware.kryo.KryoException) Output(com.esotericsoftware.kryo.io.Output)

Example 49 with Output

use of com.esotericsoftware.kryo.io.Output in project apex-core by apache.

the class DefaultStatefulStreamCodecTest method testVirginKryo.

@Test
public void testVirginKryo() {
    Kryo coder = new Kryo();
    Kryo decoder = new Kryo();
    ClassIdPair cip = new ClassIdPair();
    Output output = new Output(4096, Integer.MAX_VALUE);
    coder.writeClassAndObject(output, cip);
    Input input = new Input();
    input.setBuffer(output.toBytes());
    decoder.readClassAndObject(input);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) Output(com.esotericsoftware.kryo.io.Output) ClassIdPair(com.datatorrent.stram.codec.DefaultStatefulStreamCodec.ClassIdPair) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Example 50 with Output

use of com.esotericsoftware.kryo.io.Output in project apex-core by apache.

the class OperatorDiscoveryTest method testTypeGraphSerializer.

@Test
public void testTypeGraphSerializer() throws Exception {
    String[] classFilePath = getClassFileInClasspath();
    OperatorDiscoverer operatorDiscoverer = new OperatorDiscoverer(classFilePath);
    operatorDiscoverer.buildTypeGraph();
    // make sure (de)serialization of type graph works without problem
    Kryo kryo = new Kryo();
    TypeGraph.TypeGraphSerializer tgs = new TypeGraph.TypeGraphSerializer();
    kryo.register(TypeGraph.class, tgs);
    ByteArrayOutputStream baos = new ByteArrayOutputStream(1024 * 1024 * 20);
    Output output = new Output(baos);
    kryo.writeObject(output, operatorDiscoverer.getTypeGraph());
    output.close();
    Input input = new Input(new ByteArrayInputStream(baos.toByteArray()));
    TypeGraph tg = kryo.readObject(input, TypeGraph.class);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) ByteArrayInputStream(java.io.ByteArrayInputStream) Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Aggregations

Output (com.esotericsoftware.kryo.io.Output)67 Kryo (com.esotericsoftware.kryo.Kryo)34 ByteArrayOutputStream (java.io.ByteArrayOutputStream)34 Input (com.esotericsoftware.kryo.io.Input)28 ByteArrayInputStream (java.io.ByteArrayInputStream)13 Test (org.junit.Test)11 FileOutputStream (java.io.FileOutputStream)7 IOException (java.io.IOException)6 Test (org.testng.annotations.Test)6 KryoException (com.esotericsoftware.kryo.KryoException)5 BaseTest (org.broadinstitute.hellbender.utils.test.BaseTest)5 StdInstantiatorStrategy (org.objenesis.strategy.StdInstantiatorStrategy)4 SAMFileHeader (htsjdk.samtools.SAMFileHeader)3 HashMap (java.util.HashMap)3 ByteArrayOutputStream (org.apache.commons.io.output.ByteArrayOutputStream)3 Schema (co.cask.cdap.api.data.schema.Schema)2 Slice (com.datatorrent.netlet.util.Slice)2 File (java.io.File)2 FileNotFoundException (java.io.FileNotFoundException)2 ObjectOutput (java.io.ObjectOutput)2