Search in sources :

Example 11 with CompletedCheckpointStorageLocation

use of org.apache.flink.runtime.state.CompletedCheckpointStorageLocation in project flink by apache.

the class CheckpointMetadataLoadingTest method testUnmatchedCoordinatorOnlyStateFails.

/**
 * Tests that savepoint loading fails when there is non-restored coordinator state only, and
 * non-restored state is not allowed.
 */
@Test
public void testUnmatchedCoordinatorOnlyStateFails() throws Exception {
    final OperatorID operatorID = new OperatorID();
    final int maxParallelism = 1234;
    final OperatorState state = new OperatorState(operatorID, maxParallelism / 2, maxParallelism);
    state.setCoordinatorState(new ByteStreamStateHandle("coordinatorState", new byte[0]));
    final CompletedCheckpointStorageLocation testSavepoint = createSavepointWithOperatorState(42L, state);
    final Map<JobVertexID, ExecutionJobVertex> tasks = Collections.emptyMap();
    try {
        Checkpoints.loadAndValidateCheckpoint(new JobID(), tasks, testSavepoint, cl, false, CheckpointProperties.forSavepoint(false, SavepointFormatType.CANONICAL), RestoreMode.NO_CLAIM);
        fail("Did not throw expected Exception");
    } catch (IllegalStateException expected) {
        assertTrue(expected.getMessage().contains("allowNonRestoredState"));
    }
}
Also used : ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) ByteStreamStateHandle(org.apache.flink.runtime.state.memory.ByteStreamStateHandle) TestCompletedCheckpointStorageLocation(org.apache.flink.runtime.state.testutils.TestCompletedCheckpointStorageLocation) CompletedCheckpointStorageLocation(org.apache.flink.runtime.state.CompletedCheckpointStorageLocation) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 12 with CompletedCheckpointStorageLocation

use of org.apache.flink.runtime.state.CompletedCheckpointStorageLocation in project flink by apache.

the class CheckpointMetadataLoadingTest method testMaxParallelismMismatch.

/**
 * Tests that savepoint loading fails when there is a max-parallelism mismatch.
 */
@Test
public void testMaxParallelismMismatch() throws Exception {
    final OperatorID operatorId = new OperatorID();
    final int parallelism = 128128;
    final CompletedCheckpointStorageLocation testSavepoint = createSavepointWithOperatorSubtaskState(242L, operatorId, parallelism);
    final Map<JobVertexID, ExecutionJobVertex> tasks = createTasks(operatorId, parallelism, parallelism + 1);
    try {
        Checkpoints.loadAndValidateCheckpoint(new JobID(), tasks, testSavepoint, cl, false, CheckpointProperties.forSavepoint(false, SavepointFormatType.CANONICAL), RestoreMode.NO_CLAIM);
        fail("Did not throw expected Exception");
    } catch (IllegalStateException expected) {
        assertTrue(expected.getMessage().contains("Max parallelism mismatch"));
    }
}
Also used : ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) TestCompletedCheckpointStorageLocation(org.apache.flink.runtime.state.testutils.TestCompletedCheckpointStorageLocation) CompletedCheckpointStorageLocation(org.apache.flink.runtime.state.CompletedCheckpointStorageLocation) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 13 with CompletedCheckpointStorageLocation

use of org.apache.flink.runtime.state.CompletedCheckpointStorageLocation in project flink by apache.

the class CheckpointMetadataLoadingTest method testNonRestoredStateWhenAllowed.

/**
 * Tests that savepoint loading succeeds when there is non-restored state and it is not allowed.
 */
@Test
public void testNonRestoredStateWhenAllowed() throws Exception {
    final OperatorID operatorId = new OperatorID();
    final int parallelism = 9;
    final CompletedCheckpointStorageLocation testSavepoint = createSavepointWithOperatorSubtaskState(242L, operatorId, parallelism);
    final Map<JobVertexID, ExecutionJobVertex> tasks = Collections.emptyMap();
    final CompletedCheckpoint loaded = Checkpoints.loadAndValidateCheckpoint(new JobID(), tasks, testSavepoint, cl, true, CheckpointProperties.forSavepoint(false, SavepointFormatType.CANONICAL), RestoreMode.NO_CLAIM);
    assertTrue(loaded.getOperatorStates().isEmpty());
}
Also used : ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) TestCompletedCheckpointStorageLocation(org.apache.flink.runtime.state.testutils.TestCompletedCheckpointStorageLocation) CompletedCheckpointStorageLocation(org.apache.flink.runtime.state.CompletedCheckpointStorageLocation) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Example 14 with CompletedCheckpointStorageLocation

use of org.apache.flink.runtime.state.CompletedCheckpointStorageLocation in project flink by apache.

the class CheckpointMetadataLoadingTest method testNonRestoredStateWhenDisallowed.

/**
 * Tests that savepoint loading fails when there is non-restored state, but it is not allowed.
 */
@Test
public void testNonRestoredStateWhenDisallowed() throws Exception {
    final OperatorID operatorId = new OperatorID();
    final int parallelism = 9;
    final CompletedCheckpointStorageLocation testSavepoint = createSavepointWithOperatorSubtaskState(242L, operatorId, parallelism);
    final Map<JobVertexID, ExecutionJobVertex> tasks = Collections.emptyMap();
    try {
        Checkpoints.loadAndValidateCheckpoint(new JobID(), tasks, testSavepoint, cl, false, CheckpointProperties.forSavepoint(false, SavepointFormatType.CANONICAL), RestoreMode.NO_CLAIM);
        fail("Did not throw expected Exception");
    } catch (IllegalStateException expected) {
        assertTrue(expected.getMessage().contains("allowNonRestoredState"));
    }
}
Also used : ExecutionJobVertex(org.apache.flink.runtime.executiongraph.ExecutionJobVertex) JobVertexID(org.apache.flink.runtime.jobgraph.JobVertexID) OperatorID(org.apache.flink.runtime.jobgraph.OperatorID) TestCompletedCheckpointStorageLocation(org.apache.flink.runtime.state.testutils.TestCompletedCheckpointStorageLocation) CompletedCheckpointStorageLocation(org.apache.flink.runtime.state.CompletedCheckpointStorageLocation) JobID(org.apache.flink.api.common.JobID) Test(org.junit.Test)

Aggregations

CompletedCheckpointStorageLocation (org.apache.flink.runtime.state.CompletedCheckpointStorageLocation)14 Test (org.junit.Test)8 JobID (org.apache.flink.api.common.JobID)6 CheckpointMetadataOutputStream (org.apache.flink.runtime.state.CheckpointMetadataOutputStream)6 ExecutionJobVertex (org.apache.flink.runtime.executiongraph.ExecutionJobVertex)5 JobVertexID (org.apache.flink.runtime.jobgraph.JobVertexID)5 OperatorID (org.apache.flink.runtime.jobgraph.OperatorID)5 TestCompletedCheckpointStorageLocation (org.apache.flink.runtime.state.testutils.TestCompletedCheckpointStorageLocation)5 CheckpointStorageLocation (org.apache.flink.runtime.state.CheckpointStorageLocation)4 Path (org.apache.flink.core.fs.Path)3 CheckpointMetadata (org.apache.flink.runtime.checkpoint.metadata.CheckpointMetadata)3 CheckpointStorageAccess (org.apache.flink.runtime.state.CheckpointStorageAccess)3 StreamStateHandle (org.apache.flink.runtime.state.StreamStateHandle)3 MemoryBackendCheckpointStorageAccess (org.apache.flink.runtime.state.memory.MemoryBackendCheckpointStorageAccess)3 File (java.io.File)2 FileSystem (org.apache.flink.core.fs.FileSystem)2 DataInputStream (java.io.DataInputStream)1 IOException (java.io.IOException)1 InputStream (java.io.InputStream)1 URI (java.net.URI)1