Search in sources :

Example 21 with Kryo

use of com.esotericsoftware.kryo.Kryo in project flink by apache.

the class KryoSerializer method getKryoInstance.

// --------------------------------------------------------------------------------------------
/**
	 * Returns the Chill Kryo Serializer which is implictly added to the classpath via flink-runtime.
	 * Falls back to the default Kryo serializer if it can't be found.
	 * @return The Kryo serializer instance.
	 */
private Kryo getKryoInstance() {
    try {
        // check if ScalaKryoInstantiator is in class path (coming from Twitter's Chill library).
        // This will be true if Flink's Scala API is used.
        Class<?> chillInstantiatorClazz = Class.forName("com.twitter.chill.ScalaKryoInstantiator");
        Object chillInstantiator = chillInstantiatorClazz.newInstance();
        // obtain a Kryo instance through Twitter Chill
        Method m = chillInstantiatorClazz.getMethod("newKryo");
        return (Kryo) m.invoke(chillInstantiator);
    } catch (ClassNotFoundException | InstantiationException | NoSuchMethodException | IllegalAccessException | InvocationTargetException e) {
        LOG.warn("Falling back to default Kryo serializer because Chill serializer couldn't be found.", e);
        Kryo.DefaultInstantiatorStrategy initStrategy = new Kryo.DefaultInstantiatorStrategy();
        initStrategy.setFallbackInstantiatorStrategy(new StdInstantiatorStrategy());
        Kryo kryo = new Kryo();
        kryo.setInstantiatorStrategy(initStrategy);
        return kryo;
    }
}
Also used : Method(java.lang.reflect.Method) InvocationTargetException(java.lang.reflect.InvocationTargetException) StdInstantiatorStrategy(org.objenesis.strategy.StdInstantiatorStrategy) Kryo(com.esotericsoftware.kryo.Kryo)

Example 22 with Kryo

use of com.esotericsoftware.kryo.Kryo in project flink by apache.

the class KryoGenericTypeSerializerTest method validateReferenceMappingEnabled.

@Test
public void validateReferenceMappingEnabled() {
    KryoSerializer<String> serializer = new KryoSerializer<>(String.class, new ExecutionConfig());
    Kryo kryo = serializer.getKryo();
    assertTrue(kryo.getReferences());
}
Also used : ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) Kryo(com.esotericsoftware.kryo.Kryo) Test(org.junit.Test) AbstractGenericTypeSerializerTest(org.apache.flink.api.java.typeutils.runtime.AbstractGenericTypeSerializerTest)

Example 23 with Kryo

use of com.esotericsoftware.kryo.Kryo in project Alkahest-Coffee by AlkahestDev.

the class MultiplayerTools method register.

public static void register(EndPoint endpoint) {
    Kryo serializer = endpoint.getKryo();
    //Can't register Connection so will have to switch with Integer
    serializer.register(HashMap.class);
    serializer.register(Rectangle.class);
    serializer.register(ClientControlObject[].class);
    serializer.register(ClientControlObject.class);
    serializer.register(PlayerSoldier.class);
    serializer.register(ClientKeysUpdate.class);
    serializer.register(ClientPickedLoadout.class);
    serializer.register(ClientInfoRequest.class);
    serializer.register(ClientConnectionRequest.class);
    serializer.register(ClientPickedTeam.class);
    serializer.register(ClientSentChatMessage.class);
    serializer.register(ClientKeysUpdate.class);
    serializer.register(ServerSummary.class);
    serializer.register(ServerResponse.class);
    serializer.register(ServerResponse.ResponseCode.class);
    serializer.register(ServerDetailedSummary.class);
    serializer.register(ServerGameCountdown.class);
    serializer.register(ServerSentChatMessage.class);
    serializer.register(ServerPlayerPositions.class);
    serializer.register(ServerGameStarted.class);
    serializer.register(ServerNotifyGame.class);
}
Also used : Kryo(com.esotericsoftware.kryo.Kryo)

Example 24 with Kryo

use of com.esotericsoftware.kryo.Kryo in project apex-core by apache.

the class TypeGraphFactory method createTypeGraphProtoType.

public static TypeGraph createTypeGraphProtoType() {
    Input input = null;
    try {
        input = new Input(new ByteArrayInputStream(preComputeGraph));
        Kryo kryo = new Kryo();
        TypeGraphSerializer tgs = new TypeGraphSerializer();
        kryo.register(TypeGraph.class, tgs);
        return kryo.readObject(input, TypeGraph.class);
    } finally {
        IOUtils.closeQuietly(input);
    }
}
Also used : Input(com.esotericsoftware.kryo.io.Input) ByteArrayInputStream(java.io.ByteArrayInputStream) TypeGraphSerializer(com.datatorrent.stram.webapp.TypeGraph.TypeGraphSerializer) Kryo(com.esotericsoftware.kryo.Kryo)

Example 25 with Kryo

use of com.esotericsoftware.kryo.Kryo in project heron by twitter.

the class SerializationFactory method getKryo.

/**
   * Get kryo based on conf
   *
   * @param conf the config
   * @return Kryo
   */
@SuppressWarnings({ "rawtypes", "unchecked" })
public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    k.register(byte[].class);
    k.register(ListDelegate.class);
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    // k.register(TransactionAttempt.class);
    k.register(Values.class);
    // k.register(org.apache.storm.metric.api.IMetricsConsumer.DataPoint.class);
    // k.register(org.apache.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    /*
        try {
            JavaBridge.registerPrimitives(k);
            JavaBridge.registerCollections(k);
        } catch(Exception e) {
            throw new RuntimeException(e);
        }
        */
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (String klassName : registrations.keySet()) {
        String serializerClassName = registrations.get(klassName);
        try {
            Class klass = Class.forName(klassName);
            Class serializerClass = null;
            if (serializerClassName != null) {
                serializerClass = Class.forName(serializerClassName);
            }
            LOG.info("Doing kryo.register for class " + klass);
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName);
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(org.apache.storm.serialization.types.ArrayListSerializer) HashMapSerializer(org.apache.storm.serialization.types.HashMapSerializer) HashSetSerializer(org.apache.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayList(java.util.ArrayList) List(java.util.List) Kryo(com.esotericsoftware.kryo.Kryo)

Aggregations

Kryo (com.esotericsoftware.kryo.Kryo)71 Input (com.esotericsoftware.kryo.io.Input)31 Output (com.esotericsoftware.kryo.io.Output)29 ByteArrayOutputStream (java.io.ByteArrayOutputStream)17 ByteArrayInputStream (java.io.ByteArrayInputStream)16 StdInstantiatorStrategy (org.objenesis.strategy.StdInstantiatorStrategy)13 Test (org.junit.Test)10 Test (org.testng.annotations.Test)8 ArrayList (java.util.ArrayList)6 BigIntegerSerializer (com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer)5 File (java.io.File)5 FileNotFoundException (java.io.FileNotFoundException)5 IOException (java.io.IOException)5 List (java.util.List)5 Map (java.util.Map)5 BaseTest (org.broadinstitute.hellbender.utils.test.BaseTest)5 ArrayListSerializer (backtype.storm.serialization.types.ArrayListSerializer)3 HashMapSerializer (backtype.storm.serialization.types.HashMapSerializer)3 HashSetSerializer (backtype.storm.serialization.types.HashSetSerializer)3 Serializer (com.esotericsoftware.kryo.Serializer)3