Search in sources :

Example 11 with StateMigrationException

use of org.apache.flink.util.StateMigrationException in project flink by apache.

the class StateBackendTestBase method testValueStateRestoreWithWrongSerializers.

@Test
@SuppressWarnings("unchecked")
public void testValueStateRestoreWithWrongSerializers() throws Exception {
    assumeTrue(supportsMetaInfoVerification());
    CheckpointStreamFactory streamFactory = createStreamFactory();
    SharedStateRegistry sharedStateRegistry = new SharedStateRegistryImpl();
    CheckpointableKeyedStateBackend<Integer> backend = createKeyedBackend(IntSerializer.INSTANCE);
    try {
        ValueStateDescriptor<String> kvId = new ValueStateDescriptor<>("id", String.class);
        ValueState<String> state = backend.getPartitionedState(VoidNamespace.INSTANCE, VoidNamespaceSerializer.INSTANCE, kvId);
        backend.setCurrentKey(1);
        state.update("1");
        backend.setCurrentKey(2);
        state.update("2");
        // draw a snapshot
        KeyedStateHandle snapshot1 = runSnapshot(backend.snapshot(682375462378L, 2, streamFactory, CheckpointOptions.forCheckpointWithDefaultLocation()), sharedStateRegistry);
        backend.dispose();
        // restore the first snapshot and validate it
        backend = restoreKeyedBackend(IntSerializer.INSTANCE, snapshot1);
        snapshot1.discardState();
        @SuppressWarnings("unchecked") TypeSerializer<String> fakeStringSerializer = (TypeSerializer<String>) (TypeSerializer<?>) FloatSerializer.INSTANCE;
        try {
            kvId = new ValueStateDescriptor<>("id", fakeStringSerializer);
            state = backend.getPartitionedState(VoidNamespace.INSTANCE, VoidNamespaceSerializer.INSTANCE, kvId);
            state.value();
            fail("should recognize wrong serializers");
        } catch (StateMigrationException ignored) {
        // expected
        }
    } finally {
        IOUtils.closeQuietly(backend);
        backend.dispose();
    }
}
Also used : BlockerCheckpointStreamFactory(org.apache.flink.runtime.util.BlockerCheckpointStreamFactory) ValueStateDescriptor(org.apache.flink.api.common.state.ValueStateDescriptor) StateMigrationException(org.apache.flink.util.StateMigrationException) TypeSerializer(org.apache.flink.api.common.typeutils.TypeSerializer) Test(org.junit.Test)

Example 12 with StateMigrationException

use of org.apache.flink.util.StateMigrationException in project flink by apache.

the class StateBackendTestBase method testRestoreWithWrongKeySerializer.

@Test
public void testRestoreWithWrongKeySerializer() throws Exception {
    assumeTrue(supportsMetaInfoVerification());
    CheckpointStreamFactory streamFactory = createStreamFactory();
    SharedStateRegistry sharedStateRegistry = new SharedStateRegistryImpl();
    ValueStateDescriptor<String> kvId = new ValueStateDescriptor<>("id", String.class);
    KeyedStateHandle snapshot = null;
    // use an IntSerializer at first
    CheckpointableKeyedStateBackend<Integer> backend = createKeyedBackend(IntSerializer.INSTANCE);
    try {
        ValueState<String> state = backend.getPartitionedState(VoidNamespace.INSTANCE, VoidNamespaceSerializer.INSTANCE, kvId);
        // write some state
        backend.setCurrentKey(1);
        state.update("1");
        backend.setCurrentKey(2);
        state.update("2");
        // draw a snapshot
        snapshot = runSnapshot(backend.snapshot(682375462378L, 2, streamFactory, CheckpointOptions.forCheckpointWithDefaultLocation()), sharedStateRegistry);
    } finally {
        IOUtils.closeQuietly(backend);
        backend.dispose();
    }
    // restore with the wrong key serializer
    try {
        restoreKeyedBackend(DoubleSerializer.INSTANCE, snapshot);
        fail("should recognize wrong key serializer");
    } catch (StateMigrationException ignored) {
    // expected
    } catch (BackendBuildingException ignored) {
        assertTrue(ignored.getCause() instanceof StateMigrationException);
    }
}
Also used : ValueStateDescriptor(org.apache.flink.api.common.state.ValueStateDescriptor) StateMigrationException(org.apache.flink.util.StateMigrationException) BlockerCheckpointStreamFactory(org.apache.flink.runtime.util.BlockerCheckpointStreamFactory) Test(org.junit.Test)

Aggregations

StateMigrationException (org.apache.flink.util.StateMigrationException)12 BlockerCheckpointStreamFactory (org.apache.flink.runtime.util.BlockerCheckpointStreamFactory)5 FlinkRuntimeException (org.apache.flink.util.FlinkRuntimeException)5 Test (org.junit.Test)5 TypeSerializer (org.apache.flink.api.common.typeutils.TypeSerializer)4 IOException (java.io.IOException)3 RocksDBException (org.rocksdb.RocksDBException)3 ValueStateDescriptor (org.apache.flink.api.common.state.ValueStateDescriptor)2 MapSerializer (org.apache.flink.api.common.typeutils.base.MapSerializer)2 Nonnull (javax.annotation.Nonnull)1 ListStateDescriptor (org.apache.flink.api.common.state.ListStateDescriptor)1 MapStateDescriptor (org.apache.flink.api.common.state.MapStateDescriptor)1 ReducingStateDescriptor (org.apache.flink.api.common.state.ReducingStateDescriptor)1 State (org.apache.flink.api.common.state.State)1 TypeSerializerSnapshot (org.apache.flink.api.common.typeutils.TypeSerializerSnapshot)1 ListSerializer (org.apache.flink.api.common.typeutils.base.ListSerializer)1 MapSerializerSnapshot (org.apache.flink.api.common.typeutils.base.MapSerializerSnapshot)1 FSDataInputStream (org.apache.flink.core.fs.FSDataInputStream)1 DataInputDeserializer (org.apache.flink.core.memory.DataInputDeserializer)1 DataInputViewStreamWrapper (org.apache.flink.core.memory.DataInputViewStreamWrapper)1