Search in sources :

Example 11 with ComponentMainThreadExecutor

use of org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor in project flink by apache.

the class OperatorCoordinatorHolderTest method testCheckpointFailsIfSendingEventFailedBeforeTrigger.

@Test
public void testCheckpointFailsIfSendingEventFailedBeforeTrigger() throws Exception {
    final ReorderableManualExecutorService executor = new ReorderableManualExecutorService();
    final ComponentMainThreadExecutor mainThreadExecutor = new ComponentMainThreadExecutorServiceAdapter((ScheduledExecutorService) executor, Thread.currentThread());
    CompletableFuture<Acknowledge> eventSendingResult = new CompletableFuture<>();
    final EventReceivingTasks tasks = EventReceivingTasks.createForRunningTasksWithRpcResult(eventSendingResult);
    final OperatorCoordinatorHolder holder = createCoordinatorHolder(tasks, TestingOperatorCoordinator::new, mainThreadExecutor);
    // Send one event without finishing it.
    getCoordinator(holder).getSubtaskGateway(0).sendEvent(new TestOperatorEvent(0));
    executor.triggerAll();
    // Finish the event sending. This will insert one runnable that handles
    // failed events to the executor. And we delay this runnable to
    // simulates checkpoints triggered before the failure get processed.
    executor.setDelayNewRunnables(true);
    eventSendingResult.completeExceptionally(new RuntimeException("Artificial"));
    executor.setDelayNewRunnables(false);
    // Trigger one checkpoint, the checkpoint should not be confirmed
    // before the failure get triggered.
    CompletableFuture<byte[]> checkpointResult = new CompletableFuture<>();
    holder.checkpointCoordinator(1, checkpointResult);
    executor.triggerAll();
    getCoordinator(holder).getLastTriggeredCheckpoint().complete(new byte[0]);
    executor.triggerAll();
    assertFalse(checkpointResult.isDone());
    // Then the failure finally get processed by fail the corresponding tasks.
    executor.executeAllDelayedRunnables();
    executor.triggerAll();
    // The checkpoint would be finally confirmed.
    assertTrue(checkpointResult.isCompletedExceptionally());
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) ComponentMainThreadExecutor(org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor) Acknowledge(org.apache.flink.runtime.messages.Acknowledge) ComponentMainThreadExecutorServiceAdapter(org.apache.flink.runtime.concurrent.ComponentMainThreadExecutorServiceAdapter) Test(org.junit.Test)

Aggregations

ComponentMainThreadExecutor (org.apache.flink.runtime.concurrent.ComponentMainThreadExecutor)11 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)5 LogicalSlot (org.apache.flink.runtime.jobmaster.LogicalSlot)5 CompletableFuture (java.util.concurrent.CompletableFuture)4 FlinkException (org.apache.flink.util.FlinkException)4 Test (org.junit.Test)4 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)3 ComponentMainThreadExecutorServiceAdapter (org.apache.flink.runtime.concurrent.ComponentMainThreadExecutorServiceAdapter)3 ExecutionState (org.apache.flink.runtime.execution.ExecutionState)3 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)3 Duration (java.time.Duration)2 ArrayList (java.util.ArrayList)2 Collections (java.util.Collections)2 Iterator (java.util.Iterator)2 List (java.util.List)2 ScheduledFuture (java.util.concurrent.ScheduledFuture)2 TimeUnit (java.util.concurrent.TimeUnit)2 TimeoutException (java.util.concurrent.TimeoutException)2 JobID (org.apache.flink.api.common.JobID)2 JobStatus (org.apache.flink.api.common.JobStatus)2