Search in sources :

Example 66 with Kryo

use of com.esotericsoftware.kryo.Kryo in project jetcache by alibaba.

the class KryoValueEncoder method apply.

@Override
public byte[] apply(Object value) {
    try {
        Kryo kryo = kryoThreadLocal.get();
        ByteArrayOutputStream bos = new ByteArrayOutputStream(256);
        Output output = new Output(bos);
        output.writeInt(IDENTITY_NUMBER);
        kryo.writeClassAndObject(output, value);
        output.close();
        return bos.toByteArray();
    } catch (Exception e) {
        StringBuilder sb = new StringBuilder("Kryo Encode error. ");
        sb.append("msg=").append(e.getMessage());
        throw new CacheEncodeException(sb.toString(), e);
    }
}
Also used : Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo)

Example 67 with Kryo

use of com.esotericsoftware.kryo.Kryo in project beam by apache.

the class AvroCoderTest method testKryoSerialization.

/**
   * Confirm that we can serialize and deserialize an AvroCoder object using Kryo.
   * (BEAM-626).
   *
   * @throws Exception
   */
@Test
public void testKryoSerialization() throws Exception {
    Pojo value = new Pojo("Hello", 42);
    AvroCoder<Pojo> coder = AvroCoder.of(Pojo.class);
    //Kryo instantiation
    Kryo kryo = new Kryo();
    kryo.setInstantiatorStrategy(new StdInstantiatorStrategy());
    //Serialization of object without any memoization
    ByteArrayOutputStream coderWithoutMemoizationBos = new ByteArrayOutputStream();
    try (Output output = new Output(coderWithoutMemoizationBos)) {
        kryo.writeObject(output, coder);
    }
    // Force thread local memoization to store values.
    CoderProperties.coderDecodeEncodeEqual(coder, value);
    // Serialization of object with memoized fields
    ByteArrayOutputStream coderWithMemoizationBos = new ByteArrayOutputStream();
    try (Output output = new Output(coderWithMemoizationBos)) {
        kryo.writeObject(output, coder);
    }
    // Copy empty and memoized variants of the Coder
    ByteArrayInputStream bisWithoutMemoization = new ByteArrayInputStream(coderWithoutMemoizationBos.toByteArray());
    AvroCoder<Pojo> copiedWithoutMemoization = (AvroCoder<Pojo>) kryo.readObject(new Input(bisWithoutMemoization), AvroCoder.class);
    ByteArrayInputStream bisWithMemoization = new ByteArrayInputStream(coderWithMemoizationBos.toByteArray());
    AvroCoder<Pojo> copiedWithMemoization = (AvroCoder<Pojo>) kryo.readObject(new Input(bisWithMemoization), AvroCoder.class);
    CoderProperties.coderDecodeEncodeEqual(copiedWithoutMemoization, value);
    CoderProperties.coderDecodeEncodeEqual(copiedWithMemoization, value);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) StdInstantiatorStrategy(org.objenesis.strategy.StdInstantiatorStrategy) ByteArrayInputStream(java.io.ByteArrayInputStream) Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Example 68 with Kryo

use of com.esotericsoftware.kryo.Kryo in project apex-core by apache.

the class DefaultStatefulStreamCodecTest method testVirginKryo.

@Test
public void testVirginKryo() {
    Kryo coder = new Kryo();
    Kryo decoder = new Kryo();
    ClassIdPair cip = new ClassIdPair();
    Output output = new Output(4096, Integer.MAX_VALUE);
    coder.writeClassAndObject(output, cip);
    Input input = new Input();
    input.setBuffer(output.toBytes());
    decoder.readClassAndObject(input);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) Output(com.esotericsoftware.kryo.io.Output) ClassIdPair(com.datatorrent.stram.codec.DefaultStatefulStreamCodec.ClassIdPair) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Example 69 with Kryo

use of com.esotericsoftware.kryo.Kryo in project apex-core by apache.

the class OperatorDiscoveryTest method testTypeGraphSerializer.

@Test
public void testTypeGraphSerializer() throws Exception {
    String[] classFilePath = getClassFileInClasspath();
    OperatorDiscoverer operatorDiscoverer = new OperatorDiscoverer(classFilePath);
    operatorDiscoverer.buildTypeGraph();
    // make sure (de)serialization of type graph works without problem
    Kryo kryo = new Kryo();
    TypeGraph.TypeGraphSerializer tgs = new TypeGraph.TypeGraphSerializer();
    kryo.register(TypeGraph.class, tgs);
    ByteArrayOutputStream baos = new ByteArrayOutputStream(1024 * 1024 * 20);
    Output output = new Output(baos);
    kryo.writeObject(output, operatorDiscoverer.getTypeGraph());
    output.close();
    Input input = new Input(new ByteArrayInputStream(baos.toByteArray()));
    TypeGraph tg = kryo.readObject(input, TypeGraph.class);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) ByteArrayInputStream(java.io.ByteArrayInputStream) Output(com.esotericsoftware.kryo.io.Output) ByteArrayOutputStream(java.io.ByteArrayOutputStream) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test)

Example 70 with Kryo

use of com.esotericsoftware.kryo.Kryo in project flink by apache.

the class StateDescriptorPassingTest method validateListStateDescriptorConfigured.

private void validateListStateDescriptorConfigured(SingleOutputStreamOperator<?> result) {
    OneInputTransformation<?, ?> transform = (OneInputTransformation<?, ?>) result.getTransformation();
    WindowOperator<?, ?, ?, ?, ?> op = (WindowOperator<?, ?, ?, ?, ?>) transform.getOperator();
    StateDescriptor<?, ?> descr = op.getStateDescriptor();
    assertTrue(descr instanceof ListStateDescriptor);
    ListStateDescriptor<?> listDescr = (ListStateDescriptor<?>) descr;
    // this would be the first statement to fail if state descriptors were not properly initialized
    TypeSerializer<?> serializer = listDescr.getSerializer();
    assertTrue(serializer instanceof ListSerializer);
    TypeSerializer<?> elementSerializer = listDescr.getElementSerializer();
    assertTrue(elementSerializer instanceof KryoSerializer);
    Kryo kryo = ((KryoSerializer<?>) elementSerializer).getKryo();
    assertTrue("serializer registration was not properly passed on", kryo.getSerializer(File.class) instanceof JavaSerializer);
}
Also used : WindowOperator(org.apache.flink.streaming.runtime.operators.windowing.WindowOperator) ListSerializer(org.apache.flink.api.common.typeutils.base.ListSerializer) ListStateDescriptor(org.apache.flink.api.common.state.ListStateDescriptor) OneInputTransformation(org.apache.flink.streaming.api.transformations.OneInputTransformation) JavaSerializer(com.esotericsoftware.kryo.serializers.JavaSerializer) Kryo(com.esotericsoftware.kryo.Kryo) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer)

Aggregations

Kryo (com.esotericsoftware.kryo.Kryo)94 Input (com.esotericsoftware.kryo.io.Input)37 Output (com.esotericsoftware.kryo.io.Output)34 Test (org.junit.Test)26 ByteArrayOutputStream (java.io.ByteArrayOutputStream)21 ByteArrayInputStream (java.io.ByteArrayInputStream)17 StdInstantiatorStrategy (org.objenesis.strategy.StdInstantiatorStrategy)14 File (java.io.File)10 CollectorTestSink (org.apache.apex.malhar.lib.testbench.CollectorTestSink)10 List (java.util.List)9 Map (java.util.Map)8 Test (org.testng.annotations.Test)8 ArrayList (java.util.ArrayList)7 Path (org.apache.hadoop.fs.Path)7 BigIntegerSerializer (com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer)5 FileNotFoundException (java.io.FileNotFoundException)5 IOException (java.io.IOException)5 BaseTest (org.broadinstitute.hellbender.utils.test.BaseTest)5 DefaultPartition (com.datatorrent.api.DefaultPartition)4 CountDownLatch (java.util.concurrent.CountDownLatch)4