Search in sources :

Example 16 with ManualClock

use of org.apache.flink.util.clock.ManualClock in project flink by apache.

the class ExponentialDelayRestartBackoffTimeStrategyTest method testInitialBackoff.

@Test
public void testInitialBackoff() throws Exception {
    long initialBackoffMS = 42L;
    final ExponentialDelayRestartBackoffTimeStrategy restartStrategy = new ExponentialDelayRestartBackoffTimeStrategy(new ManualClock(), initialBackoffMS, 45L, 2.0, 8L, 0);
    assertThat(restartStrategy.getBackoffTime(), is(initialBackoffMS));
}
Also used : ManualClock(org.apache.flink.util.clock.ManualClock) Test(org.junit.Test)

Example 17 with ManualClock

use of org.apache.flink.util.clock.ManualClock in project flink by apache.

the class ExponentialDelayRestartBackoffTimeStrategyTest method testResetBackoff.

@Test
public void testResetBackoff() throws Exception {
    final long initialBackoffMS = 1L;
    final long resetBackoffThresholdMS = 8L;
    final ManualClock clock = new ManualClock();
    final ExponentialDelayRestartBackoffTimeStrategy restartStrategy = new ExponentialDelayRestartBackoffTimeStrategy(clock, initialBackoffMS, 5L, 2.0, resetBackoffThresholdMS, 0.25);
    clock.advanceTime(resetBackoffThresholdMS + restartStrategy.getBackoffTime() - 1, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertThat("Backoff should be increased", restartStrategy.getBackoffTime(), is(2L));
    clock.advanceTime(resetBackoffThresholdMS + restartStrategy.getBackoffTime(), TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertThat("Backoff should be reset", restartStrategy.getBackoffTime(), is(initialBackoffMS));
}
Also used : ManualClock(org.apache.flink.util.clock.ManualClock) Test(org.junit.Test)

Example 18 with ManualClock

use of org.apache.flink.util.clock.ManualClock in project flink by apache.

the class ExponentialDelayRestartBackoffTimeStrategyTest method testMultipleSettings.

@Test
public void testMultipleSettings() throws Exception {
    ManualClock clock = new ManualClock();
    final long initialBackoffMS = 1L;
    final long maxBackoffMS = 9L;
    double backoffMultiplier = 2.0;
    final long resetBackoffThresholdMS = 8L;
    double jitterFactor = 0.25;
    final ExponentialDelayRestartBackoffTimeStrategy restartStrategy = new ExponentialDelayRestartBackoffTimeStrategy(clock, initialBackoffMS, maxBackoffMS, backoffMultiplier, resetBackoffThresholdMS, jitterFactor);
    // ensure initial data
    assertTrue(restartStrategy.canRestart());
    assertThat(restartStrategy.getBackoffTime(), is(initialBackoffMS));
    // ensure backoff time is initial after the first failure
    clock.advanceTime(50, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertThat(restartStrategy.getBackoffTime(), is(initialBackoffMS));
    // ensure backoff increases until threshold is reached
    clock.advanceTime(4, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertThat(restartStrategy.getBackoffTime(), is(2L));
    clock.advanceTime(3, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertCorrectRandomRange(restartStrategy::getBackoffTime, 3L, 4L, 5L);
    clock.advanceTime(7, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertCorrectRandomRange(restartStrategy::getBackoffTime, 6L, 7L, 8L, 9L);
    // ensure backoff is reset after threshold is reached
    clock.advanceTime(resetBackoffThresholdMS + 9 + 1, TimeUnit.MILLISECONDS);
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertThat(restartStrategy.getBackoffTime(), is(1L));
    // ensure backoff still increases
    restartStrategy.notifyFailure(failure);
    assertTrue(restartStrategy.canRestart());
    assertThat(restartStrategy.getBackoffTime(), is(2L));
}
Also used : ManualClock(org.apache.flink.util.clock.ManualClock) Test(org.junit.Test)

Example 19 with ManualClock

use of org.apache.flink.util.clock.ManualClock in project flink by apache.

the class CheckpointBarrierTrackerTest method testTwoLastBarriersOneByOne.

@Test
public void testTwoLastBarriersOneByOne() throws Exception {
    BufferOrEvent[] sequence = { // start checkpoint 1
    createBarrier(1, 1), // start checkpoint 2
    createBarrier(2, 1), // finish the checkpoint 1
    createBarrier(1, 0), // finish the checkpoint 2
    createBarrier(2, 0) };
    ValidatingCheckpointHandler validator = new ValidatingCheckpointHandler();
    ManualClock manualClock = new ManualClock();
    inputGate = createCheckpointedInputGate(2, sequence, validator, manualClock);
    for (BufferOrEvent boe : sequence) {
        assertEquals(boe, inputGate.pollNext().get());
        manualClock.advanceTime(Duration.ofSeconds(1));
    }
    assertEquals(Duration.ofSeconds(2).toNanos(), validator.lastAlignmentDurationNanos.get().longValue());
}
Also used : ManualClock(org.apache.flink.util.clock.ManualClock) BufferOrEvent(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent) Test(org.junit.Test)

Example 20 with ManualClock

use of org.apache.flink.util.clock.ManualClock in project flink by apache.

the class CheckpointBarrierTrackerTest method testNextFirstCheckpointBarrierOvertakesCancellationBarrier.

@Test
public void testNextFirstCheckpointBarrierOvertakesCancellationBarrier() throws Exception {
    BufferOrEvent[] sequence = { // start checkpoint 1
    createBarrier(1, 1), // start checkpoint 2(just suppose checkpoint 1 was canceled)
    createBarrier(2, 1), // cancellation barrier of checkpoint 1
    createCancellationBarrier(1, 0), // finish the checkpoint 2
    createBarrier(2, 0) };
    ValidatingCheckpointHandler validator = new ValidatingCheckpointHandler();
    ManualClock manualClock = new ManualClock();
    inputGate = createCheckpointedInputGate(2, sequence, validator, manualClock);
    for (BufferOrEvent boe : sequence) {
        assertEquals(boe, inputGate.pollNext().get());
        manualClock.advanceTime(Duration.ofSeconds(1));
    }
    assertEquals(Duration.ofSeconds(2).toNanos(), validator.lastAlignmentDurationNanos.get().longValue());
}
Also used : ManualClock(org.apache.flink.util.clock.ManualClock) BufferOrEvent(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent) Test(org.junit.Test)

Aggregations

ManualClock (org.apache.flink.util.clock.ManualClock)43 Test (org.junit.Test)30 Test (org.junit.jupiter.api.Test)10 ExecutionAttemptID (org.apache.flink.runtime.executiongraph.ExecutionAttemptID)7 IdlenessTimer (org.apache.flink.api.common.eventtime.WatermarksWithIdleness.IdlenessTimer)5 JobID (org.apache.flink.api.common.JobID)3 CompletableFuture (java.util.concurrent.CompletableFuture)2 Time (org.apache.flink.api.common.time.Time)2 BufferOrEvent (org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent)2 ExecutionException (java.util.concurrent.ExecutionException)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 CheckpointRequestDeciderTest.regularCheckpoint (org.apache.flink.runtime.checkpoint.CheckpointRequestDeciderTest.regularCheckpoint)1 ResourceID (org.apache.flink.runtime.clusterframework.types.ResourceID)1 ManuallyTriggeredScheduledExecutor (org.apache.flink.util.concurrent.ManuallyTriggeredScheduledExecutor)1 Before (org.junit.Before)1