Examples with BigIntegerSerializer com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer used on opensource projects

Search in sources :

Example 1 with BigIntegerSerializer

use of com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer in project storm by nathanmarz.

the class SerializationFactory method getKryo.

public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    k.register(byte[].class);
    /* tuple payload serializer is specified via configuration */
    String payloadSerializerName = (String) conf.get(Config.TOPOLOGY_TUPLE_SERIALIZER);
    try {
        Class serializerClass = Class.forName(payloadSerializerName);
        Serializer serializer = resolveSerializerInstance(k, ListDelegate.class, serializerClass, conf);
        k.register(ListDelegate.class, serializer);
    } catch (ClassNotFoundException ex) {
        throw new RuntimeException(ex);
    }
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    k.register(TransactionAttempt.class);
    k.register(Values.class);
    k.register(backtype.storm.metric.api.IMetricsConsumer.DataPoint.class);
    k.register(backtype.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    try {
        JavaBridge.registerPrimitives(k);
        JavaBridge.registerCollections(k);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (String klassName : registrations.keySet()) {
        String serializerClassName = registrations.get(klassName);
        try {
            Class klass = Class.forName(klassName);
            Class serializerClass = null;
            if (serializerClassName != null)
                serializerClass = Class.forName(serializerClassName);
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass, conf));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName);
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(backtype.storm.serialization.types.ArrayListSerializer) HashMapSerializer(backtype.storm.serialization.types.HashMapSerializer) HashSetSerializer(backtype.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayList(java.util.ArrayList) List(java.util.List) Kryo(com.esotericsoftware.kryo.Kryo) HashMapSerializer(backtype.storm.serialization.types.HashMapSerializer) HashSetSerializer(backtype.storm.serialization.types.HashSetSerializer) ListDelegateSerializer(backtype.storm.serialization.types.ListDelegateSerializer) Serializer(com.esotericsoftware.kryo.Serializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayListSerializer(backtype.storm.serialization.types.ArrayListSerializer)

Example 2 with BigIntegerSerializer

use of com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer in project jstorm by alibaba.

the class SerializationFactory method getKryo.

public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    if (WorkerClassLoader.getInstance() != null)
        k.setClassLoader(WorkerClassLoader.getInstance());
    k.register(byte[].class);
    /* tuple payload serializer is specified via configuration */
    String payloadSerializerName = (String) conf.get(Config.TOPOLOGY_TUPLE_SERIALIZER);
    try {
        Class serializerClass = Class.forName(payloadSerializerName, true, k.getClassLoader());
        Serializer serializer = resolveSerializerInstance(k, ListDelegate.class, serializerClass, conf);
        k.register(ListDelegate.class, serializer);
    } catch (ClassNotFoundException ex) {
        throw new RuntimeException(ex);
    }
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    k.register(TransactionAttempt.class);
    k.register(Values.class);
    k.register(backtype.storm.metric.api.IMetricsConsumer.DataPoint.class);
    k.register(backtype.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    k.register(ConsList.class);
    k.register(BatchGroupId.class, new BatchGroupIdSerializer());
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (String klassName : registrations.keySet()) {
        String serializerClassName = registrations.get(klassName);
        try {
            Class klass = Class.forName(klassName, true, k.getClassLoader());
            Class serializerClass = null;
            if (serializerClassName != null)
                serializerClass = Class.forName(serializerClassName, true, k.getClassLoader());
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass, conf));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName, true, k.getClassLoader());
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(backtype.storm.serialization.types.ArrayListSerializer) BatchGroupIdSerializer(com.alibaba.jstorm.transactional.BatchGroupIdSerializer) HashMapSerializer(backtype.storm.serialization.types.HashMapSerializer) HashSetSerializer(backtype.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ConsList(storm.trident.tuple.ConsList) Kryo(com.esotericsoftware.kryo.Kryo) Serializer(com.esotericsoftware.kryo.Serializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) BatchGroupIdSerializer(com.alibaba.jstorm.transactional.BatchGroupIdSerializer) HashMapSerializer(backtype.storm.serialization.types.HashMapSerializer) HashSetSerializer(backtype.storm.serialization.types.HashSetSerializer) ArrayListSerializer(backtype.storm.serialization.types.ArrayListSerializer)

Example 3 with BigIntegerSerializer

use of com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer in project heron by twitter.

the class SerializationFactory method getKryo.

/**
   * Get kryo based on conf
   *
   * @param conf the config
   * @return Kryo
   */
@SuppressWarnings({ "rawtypes", "unchecked" })
public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    k.register(byte[].class);
    k.register(ListDelegate.class);
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    // k.register(TransactionAttempt.class);
    k.register(Values.class);
    // k.register(org.apache.storm.metric.api.IMetricsConsumer.DataPoint.class);
    // k.register(org.apache.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    /*
        try {
            JavaBridge.registerPrimitives(k);
            JavaBridge.registerCollections(k);
        } catch(Exception e) {
            throw new RuntimeException(e);
        }
        */
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (String klassName : registrations.keySet()) {
        String serializerClassName = registrations.get(klassName);
        try {
            Class klass = Class.forName(klassName);
            Class serializerClass = null;
            if (serializerClassName != null) {
                serializerClass = Class.forName(serializerClassName);
            }
            LOG.info("Doing kryo.register for class " + klass);
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName);
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(org.apache.storm.serialization.types.ArrayListSerializer) HashMapSerializer(org.apache.storm.serialization.types.HashMapSerializer) HashSetSerializer(org.apache.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayList(java.util.ArrayList) List(java.util.List) Kryo(com.esotericsoftware.kryo.Kryo)

Example 4 with BigIntegerSerializer

use of com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer in project storm by apache.

the class SerializationFactory method getKryo.

public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    k.register(byte[].class);
    /* tuple payload serializer is specified via configuration */
    String payloadSerializerName = (String) conf.get(Config.TOPOLOGY_TUPLE_SERIALIZER);
    try {
        Class serializerClass = Class.forName(payloadSerializerName);
        Serializer serializer = resolveSerializerInstance(k, ListDelegate.class, serializerClass, conf);
        k.register(ListDelegate.class, serializer);
    } catch (ClassNotFoundException ex) {
        throw new RuntimeException(ex);
    }
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    k.register(TransactionAttempt.class);
    k.register(Values.class);
    k.register(org.apache.storm.metric.api.IMetricsConsumer.DataPoint.class);
    k.register(org.apache.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    k.register(ConsList.class);
    try {
        JavaBridge.registerPrimitives(k);
        JavaBridge.registerCollections(k);
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (Map.Entry<String, String> entry : registrations.entrySet()) {
        String serializerClassName = entry.getValue();
        try {
            Class klass = Class.forName(entry.getKey());
            Class serializerClass = null;
            if (serializerClassName != null)
                serializerClass = Class.forName(serializerClassName);
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass, conf));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName);
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(org.apache.storm.serialization.types.ArrayListSerializer) HashMapSerializer(org.apache.storm.serialization.types.HashMapSerializer) HashSetSerializer(org.apache.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayList(java.util.ArrayList) ConsList(org.apache.storm.trident.tuple.ConsList) List(java.util.List) HashMap(java.util.HashMap) Map(java.util.Map) TreeMap(java.util.TreeMap) Kryo(com.esotericsoftware.kryo.Kryo) ArrayListSerializer(org.apache.storm.serialization.types.ArrayListSerializer) Serializer(com.esotericsoftware.kryo.Serializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) HashMapSerializer(org.apache.storm.serialization.types.HashMapSerializer) HashSetSerializer(org.apache.storm.serialization.types.HashSetSerializer)

Example 5 with BigIntegerSerializer

use of com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer in project heron by twitter.

the class SerializationFactory method getKryo.

/**
   * Get kryo based on conf
   * @param conf the config
   * @return Kryo
   */
@SuppressWarnings({ "rawtypes", "unchecked" })
public static Kryo getKryo(Map conf) {
    IKryoFactory kryoFactory = (IKryoFactory) Utils.newInstance((String) conf.get(Config.TOPOLOGY_KRYO_FACTORY));
    Kryo k = kryoFactory.getKryo(conf);
    k.register(byte[].class);
    k.register(ListDelegate.class);
    k.register(ArrayList.class, new ArrayListSerializer());
    k.register(HashMap.class, new HashMapSerializer());
    k.register(HashSet.class, new HashSetSerializer());
    k.register(BigInteger.class, new BigIntegerSerializer());
    // k.register(TransactionAttempt.class);
    k.register(Values.class);
    // k.register(backtype.storm.metric.api.IMetricsConsumer.DataPoint.class);
    // k.register(backtype.storm.metric.api.IMetricsConsumer.TaskInfo.class);
    /*
        try {
            JavaBridge.registerPrimitives(k);
            JavaBridge.registerCollections(k);
        } catch(Exception e) {
            throw new RuntimeException(e);
        }
        */
    Map<String, String> registrations = normalizeKryoRegister(conf);
    kryoFactory.preRegister(k, conf);
    boolean skipMissing = (Boolean) conf.get(Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS);
    for (String klassName : registrations.keySet()) {
        String serializerClassName = registrations.get(klassName);
        try {
            Class klass = Class.forName(klassName);
            Class serializerClass = null;
            if (serializerClassName != null) {
                serializerClass = Class.forName(serializerClassName);
            }
            LOG.info("Doing kryo.register for class " + klass);
            if (serializerClass == null) {
                k.register(klass);
            } else {
                k.register(klass, resolveSerializerInstance(k, klass, serializerClass));
            }
        } catch (ClassNotFoundException e) {
            if (skipMissing) {
                LOG.info("Could not find serialization or class for " + serializerClassName + ". Skipping registration...");
            } else {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postRegister(k, conf);
    if (conf.get(Config.TOPOLOGY_KRYO_DECORATORS) != null) {
        for (String klassName : (List<String>) conf.get(Config.TOPOLOGY_KRYO_DECORATORS)) {
            try {
                Class klass = Class.forName(klassName);
                IKryoDecorator decorator = (IKryoDecorator) klass.newInstance();
                decorator.decorate(k);
            } catch (ClassNotFoundException e) {
                if (skipMissing) {
                    LOG.info("Could not find kryo decorator named " + klassName + ". Skipping registration...");
                } else {
                    throw new RuntimeException(e);
                }
            } catch (InstantiationException e) {
                throw new RuntimeException(e);
            } catch (IllegalAccessException e) {
                throw new RuntimeException(e);
            }
        }
    }
    kryoFactory.postDecorate(k, conf);
    return k;
}
Also used : ArrayListSerializer(backtype.storm.serialization.types.ArrayListSerializer) HashMapSerializer(backtype.storm.serialization.types.HashMapSerializer) HashSetSerializer(backtype.storm.serialization.types.HashSetSerializer) BigIntegerSerializer(com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer) ArrayList(java.util.ArrayList) List(java.util.List) Kryo(com.esotericsoftware.kryo.Kryo)

Aggregations

Kryo (com.esotericsoftware.kryo.Kryo)5 BigIntegerSerializer (com.esotericsoftware.kryo.serializers.DefaultSerializers.BigIntegerSerializer)5 ArrayList (java.util.ArrayList)4 List (java.util.List)4 ArrayListSerializer (backtype.storm.serialization.types.ArrayListSerializer)3 HashMapSerializer (backtype.storm.serialization.types.HashMapSerializer)3 HashSetSerializer (backtype.storm.serialization.types.HashSetSerializer)3 Serializer (com.esotericsoftware.kryo.Serializer)3 ArrayListSerializer (org.apache.storm.serialization.types.ArrayListSerializer)2 HashMapSerializer (org.apache.storm.serialization.types.HashMapSerializer)2 HashSetSerializer (org.apache.storm.serialization.types.HashSetSerializer)2 ListDelegateSerializer (backtype.storm.serialization.types.ListDelegateSerializer)1 BatchGroupIdSerializer (com.alibaba.jstorm.transactional.BatchGroupIdSerializer)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 TreeMap (java.util.TreeMap)1 ConsList (org.apache.storm.trident.tuple.ConsList)1 ConsList (storm.trident.tuple.ConsList)1
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial