Search in sources :

Example 6 with TestRestartBackoffTimeStrategy

use of org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy in project flink by apache.

the class ExecutionGraphRestartTest method testFailExecutionAfterCancel.

/**
 * Tests that a graph is not restarted after cancellation via a call to {@link
 * Execution#fail(Throwable)}. This can happen when a slot is released concurrently with
 * cancellation.
 */
@Test
public void testFailExecutionAfterCancel() throws Exception {
    try (SlotPool slotPool = SlotPoolUtils.createDeclarativeSlotPoolBridge()) {
        SchedulerBase scheduler = SchedulerTestingUtils.newSchedulerBuilder(createJobGraphToCancel(), mainThreadExecutor).setExecutionSlotAllocatorFactory(createExecutionSlotAllocatorFactory(slotPool)).setRestartBackoffTimeStrategy(new TestRestartBackoffTimeStrategy(false, Long.MAX_VALUE)).setDelayExecutor(taskRestartExecutor).build();
        ExecutionGraph eg = scheduler.getExecutionGraph();
        startScheduling(scheduler);
        offerSlots(slotPool, 1);
        // Fail right after cancel (for example with concurrent slot release)
        scheduler.cancel();
        for (ExecutionVertex v : eg.getAllExecutionVertices()) {
            v.getCurrentExecutionAttempt().fail(new Exception("Test Exception"));
        }
        assertEquals(JobStatus.CANCELED, eg.getTerminationFuture().get());
        Execution execution = eg.getAllExecutionVertices().iterator().next().getCurrentExecutionAttempt();
        execution.completeCancelling();
        assertEquals(JobStatus.CANCELED, eg.getState());
    }
}
Also used : TestRestartBackoffTimeStrategy(org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy) SchedulerBase(org.apache.flink.runtime.scheduler.SchedulerBase) IOException(java.io.IOException) SlotPool(org.apache.flink.runtime.jobmaster.slotpool.SlotPool) Test(org.junit.Test)

Example 7 with TestRestartBackoffTimeStrategy

use of org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy in project flink by apache.

the class ExecutionGraphSuspendTest method testSuspendWhileRestarting.

/**
 * Tests that we can suspend a job when in state RESTARTING.
 */
@Test
public void testSuspendWhileRestarting() throws Exception {
    final ManuallyTriggeredScheduledExecutor taskRestartExecutor = new ManuallyTriggeredScheduledExecutor();
    final SchedulerBase scheduler = SchedulerTestingUtils.newSchedulerBuilder(JobGraphTestUtils.emptyJobGraph(), ComponentMainThreadExecutorServiceAdapter.forMainThread()).setRestartBackoffTimeStrategy(new TestRestartBackoffTimeStrategy(true, Long.MAX_VALUE)).setDelayExecutor(taskRestartExecutor).build();
    scheduler.startScheduling();
    final ExecutionGraph eg = scheduler.getExecutionGraph();
    assertEquals(JobStatus.RUNNING, eg.getState());
    ExecutionGraphTestUtils.switchAllVerticesToRunning(eg);
    scheduler.handleGlobalFailure(new Exception("test"));
    assertEquals(JobStatus.RESTARTING, eg.getState());
    ExecutionGraphTestUtils.completeCancellingForAllVertices(eg);
    assertEquals(JobStatus.RESTARTING, eg.getState());
    scheduler.closeAsync();
    assertEquals(JobStatus.SUSPENDED, eg.getState());
    taskRestartExecutor.triggerScheduledTasks();
    assertEquals(JobStatus.SUSPENDED, eg.getState());
}
Also used : TestRestartBackoffTimeStrategy(org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy) SchedulerBase(org.apache.flink.runtime.scheduler.SchedulerBase) ManuallyTriggeredScheduledExecutor(org.apache.flink.util.concurrent.ManuallyTriggeredScheduledExecutor) Test(org.junit.Test)

Example 8 with TestRestartBackoffTimeStrategy

use of org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy in project flink by apache.

the class AdaptiveSchedulerTest method testHowToHandleFailureAllowedByStrategy.

@Test
public void testHowToHandleFailureAllowedByStrategy() throws Exception {
    final TestRestartBackoffTimeStrategy restartBackoffTimeStrategy = new TestRestartBackoffTimeStrategy(true, 1234);
    final AdaptiveScheduler scheduler = new AdaptiveSchedulerBuilder(createJobGraph(), mainThreadExecutor).setRestartBackoffTimeStrategy(restartBackoffTimeStrategy).build();
    final FailureResult failureResult = scheduler.howToHandleFailure(new Exception("test"));
    assertThat(failureResult.canRestart()).isTrue();
    assertThat(failureResult.getBackoffTime().toMillis()).isEqualTo(restartBackoffTimeStrategy.getBackoffTime());
}
Also used : TestRestartBackoffTimeStrategy(org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy) TaskNotRunningException(org.apache.flink.runtime.operators.coordination.TaskNotRunningException) CheckpointException(org.apache.flink.runtime.checkpoint.CheckpointException) FlinkException(org.apache.flink.util.FlinkException) PartitionProducerDisposedException(org.apache.flink.runtime.jobmanager.PartitionProducerDisposedException) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) SuppressRestartsException(org.apache.flink.runtime.execution.SuppressRestartsException) Test(org.junit.Test) ArchivedExecutionGraphTest(org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest) DefaultSchedulerTest(org.apache.flink.runtime.scheduler.DefaultSchedulerTest)

Aggregations

TestRestartBackoffTimeStrategy (org.apache.flink.runtime.executiongraph.failover.flip1.TestRestartBackoffTimeStrategy)8 Test (org.junit.Test)7 IOException (java.io.IOException)6 SchedulerBase (org.apache.flink.runtime.scheduler.SchedulerBase)6 SlotPool (org.apache.flink.runtime.jobmaster.slotpool.SlotPool)5 ExecutionException (java.util.concurrent.ExecutionException)1 Configuration (org.apache.flink.configuration.Configuration)1 CheckpointException (org.apache.flink.runtime.checkpoint.CheckpointException)1 ResourceID (org.apache.flink.runtime.clusterframework.types.ResourceID)1 SuppressRestartsException (org.apache.flink.runtime.execution.SuppressRestartsException)1 ArchivedExecutionGraphTest (org.apache.flink.runtime.executiongraph.ArchivedExecutionGraphTest)1 TestingJobMasterPartitionTracker (org.apache.flink.runtime.io.network.partition.TestingJobMasterPartitionTracker)1 JobGraph (org.apache.flink.runtime.jobgraph.JobGraph)1 JobVertex (org.apache.flink.runtime.jobgraph.JobVertex)1 PartitionProducerDisposedException (org.apache.flink.runtime.jobmanager.PartitionProducerDisposedException)1 TaskNotRunningException (org.apache.flink.runtime.operators.coordination.TaskNotRunningException)1 DefaultSchedulerTest (org.apache.flink.runtime.scheduler.DefaultSchedulerTest)1 TestingShuffleMaster (org.apache.flink.runtime.shuffle.TestingShuffleMaster)1 DirectScheduledExecutorService (org.apache.flink.runtime.testutils.DirectScheduledExecutorService)1 FlinkException (org.apache.flink.util.FlinkException)1