Search in sources :

Example 11 with Input

use of com.esotericsoftware.kryo.io.Input in project apex-core by apache.

the class Journal method replay.

final void replay(final InputStream input) {
    if (replayMode.compareAndSet(false, true)) {
        Input in = new Input(input);
        try {
            LOG.debug("Start replaying WAL");
            while (!in.eof()) {
                final int opId = in.readInt();
                final RecoverableOperation recoverableOperation = RecoverableOperation.get(opId);
                if (recoverableOperation == null) {
                    throw new IllegalArgumentException("No reader registered for id " + opId);
                }
                LOG.debug("Replaying {}", recoverableOperation);
                switch(recoverableOperation) {
                    case OPERATOR_STATE:
                    case CONTAINER_STATE:
                        recoverableOperation.operation.read(scm.getPhysicalPlan(), in);
                        break;
                    case OPERATOR_PROPERTY:
                    case PHYSICAL_OPERATOR_PROPERTY:
                        recoverableOperation.operation.read(scm, in);
                        break;
                    default:
                        throw new IllegalArgumentException("Unsupported recoverable operation " + recoverableOperation);
                }
            }
        } finally {
            LOG.debug("Done replaying WAL");
            replayMode.set(false);
        }
    } else {
        throw new IllegalStateException("Request to replay while journal is already replaying other operations");
    }
}
Also used : Input(com.esotericsoftware.kryo.io.Input)

Example 12 with Input

use of com.esotericsoftware.kryo.io.Input in project apex-core by apache.

the class FSStorageAgent method retrieve.

public static Object retrieve(InputStream stream) {
    synchronized (kryo) {
        kryo.setClassLoader(Thread.currentThread().getContextClassLoader());
        Input input = new Input(stream);
        return kryo.readClassAndObject(input);
    }
}
Also used : Input(com.esotericsoftware.kryo.io.Input)

Example 13 with Input

use of com.esotericsoftware.kryo.io.Input in project heron by twitter.

the class HeronPluggableSerializerDelegate method initialize.

@Override
@SuppressWarnings("rawtypes")
public void initialize(Map config) {
    kryo = SerializationFactory.getKryo(config);
    kryoOut = new Output(2000, 2000000000);
    kryoIn = new Input(1);
}
Also used : Input(com.esotericsoftware.kryo.io.Input) Output(com.esotericsoftware.kryo.io.Output)

Example 14 with Input

use of com.esotericsoftware.kryo.io.Input in project intellij-community by JetBrains.

the class ExternalProjectSerializer method load.

@Nullable
public ExternalProject load(@NotNull ProjectSystemId externalSystemId, File externalProjectPath) {
    LOG.debug("Attempt to load project data from: " + externalProjectPath);
    ExternalProject externalProject = null;
    try {
        final File configurationFile = getProjectConfigurationFile(externalSystemId, externalProjectPath);
        if (!configurationFile.isFile())
            return null;
        Input input = new Input(new FileInputStream(configurationFile));
        try {
            externalProject = myKryo.readObject(input, DefaultExternalProject.class);
        } finally {
            StreamUtil.closeStream(input);
        }
    } catch (Exception e) {
        LOG.debug(e);
    }
    if (externalProject != null) {
        LOG.debug("Loaded project: " + externalProject.getProjectDir());
    }
    return externalProject;
}
Also used : Input(com.esotericsoftware.kryo.io.Input) File(java.io.File) FileInputStream(java.io.FileInputStream) FileNotFoundException(java.io.FileNotFoundException) Nullable(org.jetbrains.annotations.Nullable)

Example 15 with Input

use of com.esotericsoftware.kryo.io.Input in project intellij-community by JetBrains.

the class ExternalProjectSerializer method configureKryo.

private void configureKryo() {
    myKryo.setAutoReset(true);
    myKryo.setRegistrationRequired(true);
    Log.set(Log.LEVEL_WARN);
    myKryo.register(ArrayList.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new ArrayList();
        }
    });
    myKryo.register(HashMap.class, new MapSerializer() {

        @Override
        protected Map create(Kryo kryo, Input input, Class<Map> type) {
            return new HashMap();
        }
    });
    myKryo.register(HashSet.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new HashSet();
        }
    });
    myKryo.register(File.class, new FileSerializer());
    myKryo.register(DefaultExternalProject.class, new FieldSerializer<DefaultExternalProject>(myKryo, DefaultExternalProject.class) {

        @Override
        protected DefaultExternalProject create(Kryo kryo, Input input, Class<DefaultExternalProject> type) {
            return new DefaultExternalProject();
        }
    });
    myKryo.register(DefaultExternalTask.class, new FieldSerializer<DefaultExternalTask>(myKryo, DefaultExternalTask.class) {

        @Override
        protected DefaultExternalTask create(Kryo kryo, Input input, Class<DefaultExternalTask> type) {
            return new DefaultExternalTask();
        }
    });
    myKryo.register(DefaultExternalPlugin.class, new FieldSerializer<DefaultExternalPlugin>(myKryo, DefaultExternalPlugin.class) {

        @Override
        protected DefaultExternalPlugin create(Kryo kryo, Input input, Class<DefaultExternalPlugin> type) {
            return new DefaultExternalPlugin();
        }
    });
    myKryo.register(DefaultExternalSourceSet.class, new FieldSerializer<DefaultExternalSourceSet>(myKryo, DefaultExternalSourceSet.class) {

        @Override
        protected DefaultExternalSourceSet create(Kryo kryo, Input input, Class<DefaultExternalSourceSet> type) {
            return new DefaultExternalSourceSet();
        }
    });
    myKryo.register(DefaultExternalSourceDirectorySet.class, new FieldSerializer<DefaultExternalSourceDirectorySet>(myKryo, DefaultExternalSourceDirectorySet.class) {

        @Override
        protected DefaultExternalSourceDirectorySet create(Kryo kryo, Input input, Class<DefaultExternalSourceDirectorySet> type) {
            return new DefaultExternalSourceDirectorySet();
        }
    });
    myKryo.register(DefaultExternalFilter.class, new FieldSerializer<DefaultExternalFilter>(myKryo, DefaultExternalFilter.class) {

        @Override
        protected DefaultExternalFilter create(Kryo kryo, Input input, Class<DefaultExternalFilter> type) {
            return new DefaultExternalFilter();
        }
    });
    myKryo.register(ExternalSystemSourceType.class, new DefaultSerializers.EnumSerializer(ExternalSystemSourceType.class));
    myKryo.register(DefaultExternalProjectDependency.class, new FieldSerializer<DefaultExternalProjectDependency>(myKryo, DefaultExternalProjectDependency.class) {

        @Override
        protected DefaultExternalProjectDependency create(Kryo kryo, Input input, Class<DefaultExternalProjectDependency> type) {
            return new DefaultExternalProjectDependency();
        }
    });
    myKryo.register(DefaultFileCollectionDependency.class, new FieldSerializer<DefaultFileCollectionDependency>(myKryo, DefaultFileCollectionDependency.class) {

        @Override
        protected DefaultFileCollectionDependency create(Kryo kryo, Input input, Class<DefaultFileCollectionDependency> type) {
            return new DefaultFileCollectionDependency();
        }
    });
    myKryo.register(DefaultExternalLibraryDependency.class, new FieldSerializer<DefaultExternalLibraryDependency>(myKryo, DefaultExternalLibraryDependency.class) {

        @Override
        protected DefaultExternalLibraryDependency create(Kryo kryo, Input input, Class<DefaultExternalLibraryDependency> type) {
            return new DefaultExternalLibraryDependency();
        }
    });
    myKryo.register(DefaultUnresolvedExternalDependency.class, new FieldSerializer<DefaultUnresolvedExternalDependency>(myKryo, DefaultUnresolvedExternalDependency.class) {

        @Override
        protected DefaultUnresolvedExternalDependency create(Kryo kryo, Input input, Class<DefaultUnresolvedExternalDependency> type) {
            return new DefaultUnresolvedExternalDependency();
        }
    });
    myKryo.register(DefaultExternalDependencyId.class, new FieldSerializer<DefaultExternalDependencyId>(myKryo, DefaultExternalDependencyId.class) {

        @Override
        protected DefaultExternalDependencyId create(Kryo kryo, Input input, Class<DefaultExternalDependencyId> type) {
            return new DefaultExternalDependencyId();
        }
    });
    myKryo.register(LinkedHashSet.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new LinkedHashSet();
        }
    });
    myKryo.register(HashSet.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new HashSet();
        }
    });
    myKryo.register(THashSet.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new THashSet();
        }
    });
    myKryo.register(Set.class, new CollectionSerializer() {

        @Override
        protected Collection create(Kryo kryo, Input input, Class<Collection> type) {
            return new HashSet();
        }
    });
    myKryo.register(THashMap.class, new MapSerializer() {

        @Override
        protected Map create(Kryo kryo, Input input, Class<Map> type) {
            return new THashMap();
        }
    });
}
Also used : THashMap(gnu.trove.THashMap) DefaultExternalDependencyId(org.jetbrains.plugins.gradle.DefaultExternalDependencyId) Input(com.esotericsoftware.kryo.io.Input) MapSerializer(com.esotericsoftware.kryo.serializers.MapSerializer) THashSet(gnu.trove.THashSet) THashSet(gnu.trove.THashSet) THashMap(gnu.trove.THashMap) ExternalSystemSourceType(com.intellij.openapi.externalSystem.model.project.ExternalSystemSourceType) THashMap(gnu.trove.THashMap) DefaultSerializers(com.esotericsoftware.kryo.serializers.DefaultSerializers) CollectionSerializer(com.esotericsoftware.kryo.serializers.CollectionSerializer) Kryo(com.esotericsoftware.kryo.Kryo)

Aggregations

Input (com.esotericsoftware.kryo.io.Input)49 Kryo (com.esotericsoftware.kryo.Kryo)31 Output (com.esotericsoftware.kryo.io.Output)21 ByteArrayInputStream (java.io.ByteArrayInputStream)19 ByteArrayOutputStream (java.io.ByteArrayOutputStream)15 Test (org.junit.Test)8 Test (org.testng.annotations.Test)8 FileInputStream (java.io.FileInputStream)6 IOException (java.io.IOException)5 BaseTest (org.broadinstitute.hellbender.utils.test.BaseTest)5 StdInstantiatorStrategy (org.objenesis.strategy.StdInstantiatorStrategy)4 File (java.io.File)3 FileNotFoundException (java.io.FileNotFoundException)3 HashMap (java.util.HashMap)3 Map (java.util.Map)3 Schema (co.cask.cdap.api.data.schema.Schema)2 SAMFileHeader (htsjdk.samtools.SAMFileHeader)2 ArrayList (java.util.ArrayList)2 HiveKey (org.apache.hadoop.hive.ql.io.HiveKey)2 HiveException (org.apache.hadoop.hive.ql.metadata.HiveException)2