Search in sources :

Example 6 with KryoSerializer

use of org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer in project flink by apache.

the class MapStateDescriptorTest method testMapStateDescriptorEagerSerializer.

@Test
public void testMapStateDescriptorEagerSerializer() throws Exception {
    TypeSerializer<Integer> keySerializer = new KryoSerializer<>(Integer.class, new ExecutionConfig());
    TypeSerializer<String> valueSerializer = new KryoSerializer<>(String.class, new ExecutionConfig());
    MapStateDescriptor<Integer, String> descr = new MapStateDescriptor<>("testName", keySerializer, valueSerializer);
    assertEquals("testName", descr.getName());
    assertNotNull(descr.getSerializer());
    assertTrue(descr.getSerializer() instanceof MapSerializer);
    assertNotNull(descr.getKeySerializer());
    assertEquals(keySerializer, descr.getKeySerializer());
    assertNotNull(descr.getValueSerializer());
    assertEquals(valueSerializer, descr.getValueSerializer());
    MapStateDescriptor<Integer, String> copy = CommonTestUtils.createCopySerializable(descr);
    assertEquals("testName", copy.getName());
    assertNotNull(copy.getSerializer());
    assertTrue(copy.getSerializer() instanceof MapSerializer);
    assertNotNull(copy.getKeySerializer());
    assertEquals(keySerializer, copy.getKeySerializer());
    assertNotNull(copy.getValueSerializer());
    assertEquals(valueSerializer, copy.getValueSerializer());
}
Also used : MapSerializer(org.apache.flink.api.common.typeutils.base.MapSerializer) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer) Test(org.junit.Test)

Example 7 with KryoSerializer

use of org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer in project flink by apache.

the class ReducingStateDescriptorTest method testValueStateDescriptorLazySerializer.

@Test
public void testValueStateDescriptorLazySerializer() throws Exception {
    @SuppressWarnings("unchecked") ReduceFunction<Path> reducer = mock(ReduceFunction.class);
    // some different registered value
    ExecutionConfig cfg = new ExecutionConfig();
    cfg.registerKryoType(TaskInfo.class);
    ReducingStateDescriptor<Path> descr = new ReducingStateDescriptor<Path>("testName", reducer, Path.class);
    try {
        descr.getSerializer();
        fail("should cause an exception");
    } catch (IllegalStateException ignored) {
    }
    descr.initializeSerializerUnlessSet(cfg);
    assertNotNull(descr.getSerializer());
    assertTrue(descr.getSerializer() instanceof KryoSerializer);
    assertTrue(((KryoSerializer<?>) descr.getSerializer()).getKryo().getRegistration(TaskInfo.class).getId() > 0);
}
Also used : Path(org.apache.flink.core.fs.Path) TaskInfo(org.apache.flink.api.common.TaskInfo) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer) Test(org.junit.Test)

Example 8 with KryoSerializer

use of org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer in project flink by apache.

the class ValueStateDescriptorTest method testValueStateDescriptorLazySerializer.

@Test
public void testValueStateDescriptorLazySerializer() throws Exception {
    // some default value that goes to the generic serializer
    Path defaultValue = new Path(new File(ConfigConstants.DEFAULT_TASK_MANAGER_TMP_PATH).toURI());
    // some different registered value
    ExecutionConfig cfg = new ExecutionConfig();
    cfg.registerKryoType(TaskInfo.class);
    ValueStateDescriptor<Path> descr = new ValueStateDescriptor<Path>("testName", Path.class, defaultValue);
    try {
        descr.getSerializer();
        fail("should cause an exception");
    } catch (IllegalStateException ignored) {
    }
    descr.initializeSerializerUnlessSet(cfg);
    assertNotNull(descr.getSerializer());
    assertTrue(descr.getSerializer() instanceof KryoSerializer);
    assertTrue(((KryoSerializer<?>) descr.getSerializer()).getKryo().getRegistration(TaskInfo.class).getId() > 0);
}
Also used : Path(org.apache.flink.core.fs.Path) TaskInfo(org.apache.flink.api.common.TaskInfo) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) File(java.io.File) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer) Test(org.junit.Test)

Example 9 with KryoSerializer

use of org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer in project flink by apache.

the class ValueStateDescriptorTest method testValueStateDescriptorEagerSerializer.

@Test
public void testValueStateDescriptorEagerSerializer() throws Exception {
    TypeSerializer<String> serializer = new KryoSerializer<>(String.class, new ExecutionConfig());
    String defaultValue = "le-value-default";
    ValueStateDescriptor<String> descr = new ValueStateDescriptor<String>("testName", serializer, defaultValue);
    assertEquals("testName", descr.getName());
    assertEquals(defaultValue, descr.getDefaultValue());
    assertNotNull(descr.getSerializer());
    assertEquals(serializer, descr.getSerializer());
    ValueStateDescriptor<String> copy = CommonTestUtils.createCopySerializable(descr);
    assertEquals("testName", copy.getName());
    assertEquals(defaultValue, copy.getDefaultValue());
    assertNotNull(copy.getSerializer());
    assertEquals(serializer, copy.getSerializer());
}
Also used : ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer) Test(org.junit.Test)

Example 10 with KryoSerializer

use of org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer in project flink by apache.

the class ListStateDescriptorTest method testValueStateDescriptorEagerSerializer.

@Test
public void testValueStateDescriptorEagerSerializer() throws Exception {
    TypeSerializer<String> serializer = new KryoSerializer<>(String.class, new ExecutionConfig());
    ListStateDescriptor<String> descr = new ListStateDescriptor<String>("testName", serializer);
    assertEquals("testName", descr.getName());
    assertNotNull(descr.getSerializer());
    assertTrue(descr.getSerializer() instanceof ListSerializer);
    assertNotNull(descr.getElementSerializer());
    assertEquals(serializer, descr.getElementSerializer());
    ListStateDescriptor<String> copy = CommonTestUtils.createCopySerializable(descr);
    assertEquals("testName", copy.getName());
    assertNotNull(copy.getSerializer());
    assertTrue(copy.getSerializer() instanceof ListSerializer);
    assertNotNull(copy.getElementSerializer());
    assertEquals(serializer, copy.getElementSerializer());
}
Also used : ListSerializer(org.apache.flink.api.common.typeutils.base.ListSerializer) ExecutionConfig(org.apache.flink.api.common.ExecutionConfig) KryoSerializer(org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer) Test(org.junit.Test)

Aggregations

KryoSerializer (org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer)17 ExecutionConfig (org.apache.flink.api.common.ExecutionConfig)15 Test (org.junit.Test)14 TaskInfo (org.apache.flink.api.common.TaskInfo)9 Path (org.apache.flink.core.fs.Path)9 AtomicReference (java.util.concurrent.atomic.AtomicReference)5 ListStateDescriptor (org.apache.flink.api.common.state.ListStateDescriptor)4 ListSerializer (org.apache.flink.api.common.typeutils.base.ListSerializer)4 FoldingStateDescriptor (org.apache.flink.api.common.state.FoldingStateDescriptor)3 MapStateDescriptor (org.apache.flink.api.common.state.MapStateDescriptor)3 Kryo (com.esotericsoftware.kryo.Kryo)2 JavaSerializer (com.esotericsoftware.kryo.serializers.JavaSerializer)2 ReducingStateDescriptor (org.apache.flink.api.common.state.ReducingStateDescriptor)2 StateDescriptor (org.apache.flink.api.common.state.StateDescriptor)2 ValueStateDescriptor (org.apache.flink.api.common.state.ValueStateDescriptor)2 MapSerializer (org.apache.flink.api.common.typeutils.base.MapSerializer)2 OneInputTransformation (org.apache.flink.streaming.api.transformations.OneInputTransformation)2 WindowOperator (org.apache.flink.streaming.runtime.operators.windowing.WindowOperator)2 File (java.io.File)1 FoldFunction (org.apache.flink.api.common.functions.FoldFunction)1