Search in sources :

Example 11 with CheckpointStorage

use of org.apache.flink.runtime.state.CheckpointStorage in project flink by apache.

the class ChangelogStateBackendLoadingTest method testApplicationEnableChangelogStateBackend.

@Test
public void testApplicationEnableChangelogStateBackend() throws Exception {
    final StateBackend backend = StateBackendLoader.fromApplicationOrConfigOrDefault(null, TernaryBoolean.TRUE, config(false), cl, null);
    final CheckpointStorage storage = CheckpointStorageLoader.load(null, null, backend, config(), cl, null);
    assertDelegateStateBackend(backend, HashMapStateBackend.class, storage, JobManagerCheckpointStorage.class);
}
Also used : CheckpointStorage(org.apache.flink.runtime.state.CheckpointStorage) JobManagerCheckpointStorage(org.apache.flink.runtime.state.storage.JobManagerCheckpointStorage) EmbeddedRocksDBStateBackend(org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend) StateBackend(org.apache.flink.runtime.state.StateBackend) OperatorStateBackend(org.apache.flink.runtime.state.OperatorStateBackend) DelegatingStateBackend(org.apache.flink.runtime.state.delegate.DelegatingStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) AbstractStateBackend(org.apache.flink.runtime.state.AbstractStateBackend) AbstractKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend) HashMapStateBackend(org.apache.flink.runtime.state.hashmap.HashMapStateBackend) ConfigurableStateBackend(org.apache.flink.runtime.state.ConfigurableStateBackend) Test(org.junit.Test)

Example 12 with CheckpointStorage

use of org.apache.flink.runtime.state.CheckpointStorage in project flink by apache.

the class ChangelogStateBackendLoadingTest method testLoadingStateBackend.

private void testLoadingStateBackend(String backendName, Class<?> delegatedStateBackendClass, Class<?> storageClass, boolean configOnly) throws Exception {
    final Configuration config = config(backendName, true);
    StateBackend backend;
    StateBackend appBackend = StateBackendLoader.loadStateBackendFromConfig(config, cl, null);
    if (configOnly) {
        backend = StateBackendLoader.fromApplicationOrConfigOrDefault(null, TernaryBoolean.UNDEFINED, config, cl, null);
    } else {
        backend = StateBackendLoader.fromApplicationOrConfigOrDefault(appBackend, TernaryBoolean.TRUE, config, cl, null);
    }
    final CheckpointStorage storage = CheckpointStorageLoader.load(null, null, backend, config, cl, null);
    assertDelegateStateBackend(backend, delegatedStateBackendClass, storage, storageClass);
}
Also used : Configuration(org.apache.flink.configuration.Configuration) CheckpointStorage(org.apache.flink.runtime.state.CheckpointStorage) JobManagerCheckpointStorage(org.apache.flink.runtime.state.storage.JobManagerCheckpointStorage) EmbeddedRocksDBStateBackend(org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend) StateBackend(org.apache.flink.runtime.state.StateBackend) OperatorStateBackend(org.apache.flink.runtime.state.OperatorStateBackend) DelegatingStateBackend(org.apache.flink.runtime.state.delegate.DelegatingStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) AbstractStateBackend(org.apache.flink.runtime.state.AbstractStateBackend) AbstractKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend) HashMapStateBackend(org.apache.flink.runtime.state.hashmap.HashMapStateBackend) ConfigurableStateBackend(org.apache.flink.runtime.state.ConfigurableStateBackend)

Example 13 with CheckpointStorage

use of org.apache.flink.runtime.state.CheckpointStorage in project flink by apache.

the class ChangelogStateBackendLoadingTest method testApplicationDefinedChangelogStateBackend.

@Test
public void testApplicationDefinedChangelogStateBackend() throws Exception {
    final StateBackend appBackend = new MockStateBackend();
    // "rocksdb" should not take effect
    final StateBackend backend = StateBackendLoader.fromApplicationOrConfigOrDefault(appBackend, TernaryBoolean.TRUE, config("rocksdb", false), cl, null);
    final CheckpointStorage storage = CheckpointStorageLoader.load(null, null, backend, config(), cl, null);
    assertDelegateStateBackend(backend, MockStateBackend.class, storage, MockStateBackend.class);
    assertTrue(((MockStateBackend) (((ChangelogStateBackend) backend).getDelegatedStateBackend())).isConfigUpdated());
}
Also used : CheckpointStorage(org.apache.flink.runtime.state.CheckpointStorage) JobManagerCheckpointStorage(org.apache.flink.runtime.state.storage.JobManagerCheckpointStorage) EmbeddedRocksDBStateBackend(org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend) StateBackend(org.apache.flink.runtime.state.StateBackend) OperatorStateBackend(org.apache.flink.runtime.state.OperatorStateBackend) DelegatingStateBackend(org.apache.flink.runtime.state.delegate.DelegatingStateBackend) MemoryStateBackend(org.apache.flink.runtime.state.memory.MemoryStateBackend) AbstractStateBackend(org.apache.flink.runtime.state.AbstractStateBackend) AbstractKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend) HashMapStateBackend(org.apache.flink.runtime.state.hashmap.HashMapStateBackend) ConfigurableStateBackend(org.apache.flink.runtime.state.ConfigurableStateBackend) Test(org.junit.Test)

Aggregations

CheckpointStorage (org.apache.flink.runtime.state.CheckpointStorage)13 StateBackend (org.apache.flink.runtime.state.StateBackend)10 JobManagerCheckpointStorage (org.apache.flink.runtime.state.storage.JobManagerCheckpointStorage)7 AbstractKeyedStateBackend (org.apache.flink.runtime.state.AbstractKeyedStateBackend)6 OperatorStateBackend (org.apache.flink.runtime.state.OperatorStateBackend)6 HashMapStateBackend (org.apache.flink.runtime.state.hashmap.HashMapStateBackend)6 EmbeddedRocksDBStateBackend (org.apache.flink.contrib.streaming.state.EmbeddedRocksDBStateBackend)5 AbstractStateBackend (org.apache.flink.runtime.state.AbstractStateBackend)5 ConfigurableStateBackend (org.apache.flink.runtime.state.ConfigurableStateBackend)5 DelegatingStateBackend (org.apache.flink.runtime.state.delegate.DelegatingStateBackend)5 MemoryStateBackend (org.apache.flink.runtime.state.memory.MemoryStateBackend)5 Test (org.junit.Test)5 JobCheckpointingSettings (org.apache.flink.runtime.jobgraph.tasks.JobCheckpointingSettings)4 IOException (java.io.IOException)3 CheckpointCoordinatorConfiguration (org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration)3 ArrayList (java.util.ArrayList)2 Nonnull (javax.annotation.Nonnull)2 MasterTriggerRestoreHook (org.apache.flink.runtime.checkpoint.MasterTriggerRestoreHook)2 SerializedValue (org.apache.flink.util.SerializedValue)2 File (java.io.File)1