Search in sources :

Example 26 with FakeTimer

use of org.apache.hadoop.util.FakeTimer in project hadoop by apache.

the class TestThrottledAsyncChecker method testCancellation.

@Test(timeout = 60000)
public void testCancellation() throws Exception {
    LatchedCheckable target = new LatchedCheckable();
    final FakeTimer timer = new FakeTimer();
    final LatchedCallback callback = new LatchedCallback(target);
    ThrottledAsyncChecker<Boolean, Boolean> checker = new ThrottledAsyncChecker<>(timer, MIN_ERROR_CHECK_GAP, 0, getExecutorService());
    Optional<ListenableFuture<Boolean>> olf = checker.schedule(target, true);
    if (olf.isPresent()) {
        Futures.addCallback(olf.get(), callback);
    }
    // Request immediate cancellation.
    checker.shutdownAndWait(0, TimeUnit.MILLISECONDS);
    try {
        assertFalse(olf.get().get());
        fail("Failed to get expected InterruptedException");
    } catch (ExecutionException ee) {
        assertTrue(ee.getCause() instanceof InterruptedException);
    }
    callback.failureLatch.await();
}
Also used : ListenableFuture(com.google.common.util.concurrent.ListenableFuture) ExecutionException(java.util.concurrent.ExecutionException) FakeTimer(org.apache.hadoop.util.FakeTimer) Test(org.junit.Test)

Example 27 with FakeTimer

use of org.apache.hadoop.util.FakeTimer in project hadoop by apache.

the class TestThrottledAsyncChecker method testConcurrentChecks.

@Test(timeout = 60000)
public void testConcurrentChecks() throws Exception {
    LatchedCheckable target = new LatchedCheckable();
    final FakeTimer timer = new FakeTimer();
    ThrottledAsyncChecker<Boolean, Boolean> checker = new ThrottledAsyncChecker<>(timer, MIN_ERROR_CHECK_GAP, 0, getExecutorService());
    final Optional<ListenableFuture<Boolean>> olf1 = checker.schedule(target, true);
    final Optional<ListenableFuture<Boolean>> olf2 = checker.schedule(target, true);
    // Ensure that concurrent requests return the future object
    // for the first caller.
    assertTrue(olf1.isPresent());
    assertFalse(olf2.isPresent());
    // Unblock the latch and wait for it to finish execution.
    target.latch.countDown();
    olf1.get().get();
    GenericTestUtils.waitFor(new Supplier<Boolean>() {

        @Override
        public Boolean get() {
            // We should get an absent Optional.
            // This can take a short while until the internal callback in
            // ThrottledAsyncChecker is scheduled for execution.
            // Also this should not trigger a new check operation as the timer
            // was not advanced. If it does trigger a new check then the test
            // will fail with a timeout.
            final Optional<ListenableFuture<Boolean>> olf3 = checker.schedule(target, true);
            return !olf3.isPresent();
        }
    }, 100, 10000);
}
Also used : Optional(com.google.common.base.Optional) ListenableFuture(com.google.common.util.concurrent.ListenableFuture) FakeTimer(org.apache.hadoop.util.FakeTimer) Test(org.junit.Test)

Example 28 with FakeTimer

use of org.apache.hadoop.util.FakeTimer in project hadoop by apache.

the class TestThrottledAsyncChecker method testExceptionCaching.

/**
   * Ensure that the exception from a failed check is cached
   * and returned without re-running the check when the minimum
   * gap has not elapsed.
   *
   * @throws Exception
   */
@Test(timeout = 60000)
public void testExceptionCaching() throws Exception {
    final ThrowingCheckable target1 = new ThrowingCheckable();
    final FakeTimer timer = new FakeTimer();
    ThrottledAsyncChecker<Boolean, Boolean> checker = new ThrottledAsyncChecker<>(timer, MIN_ERROR_CHECK_GAP, 0, getExecutorService());
    assertTrue(checker.schedule(target1, true).isPresent());
    waitTestCheckableCheckCount(target1, 1L);
    assertFalse(checker.schedule(target1, true).isPresent());
    waitTestCheckableCheckCount(target1, 1L);
}
Also used : FakeTimer(org.apache.hadoop.util.FakeTimer) Test(org.junit.Test)

Example 29 with FakeTimer

use of org.apache.hadoop.util.FakeTimer in project hadoop by apache.

the class TestThrottledAsyncChecker method testContextIsPassed.

/**
   * Ensure that the context is passed through to the Checkable#check
   * method.
   * @throws Exception
   */
@Test(timeout = 60000)
public void testContextIsPassed() throws Exception {
    final NoOpCheckable target1 = new NoOpCheckable();
    final FakeTimer timer = new FakeTimer();
    ThrottledAsyncChecker<Boolean, Boolean> checker = new ThrottledAsyncChecker<>(timer, MIN_ERROR_CHECK_GAP, 0, getExecutorService());
    assertTrue(checker.schedule(target1, true).isPresent());
    waitTestCheckableCheckCount(target1, 1L);
    timer.advance(MIN_ERROR_CHECK_GAP + 1);
    assertTrue(checker.schedule(target1, false).isPresent());
    waitTestCheckableCheckCount(target1, 2L);
}
Also used : FakeTimer(org.apache.hadoop.util.FakeTimer) Test(org.junit.Test)

Example 30 with FakeTimer

use of org.apache.hadoop.util.FakeTimer in project hadoop by apache.

the class TestThrottledAsyncCheckerTimeout method testDiskCheckTimeoutInvokesOneCallbackOnly.

@Test(timeout = 2000)
public void testDiskCheckTimeoutInvokesOneCallbackOnly() throws Exception {
    LOG.info("Executing {}", testName.getMethodName());
    final DummyCheckable target = new DummyCheckable();
    final FakeTimer timer = new FakeTimer();
    ThrottledAsyncChecker<Boolean, Boolean> checker = new ThrottledAsyncChecker<>(timer, 0, DISK_CHECK_TIMEOUT, getExecutorService());
    FutureCallback<Boolean> futureCallback = mock(FutureCallback.class);
    // Acquire lock to halt disk checker. Release after timeout occurs.
    lock.lock();
    final Optional<ListenableFuture<Boolean>> olf1 = checker.schedule(target, true);
    assertTrue(olf1.isPresent());
    Futures.addCallback(olf1.get(), futureCallback);
    // Wait for the callback
    Thread.sleep(DISK_CHECK_TIMEOUT);
    // Verify that timeout results in only 1 onFailure call and 0 onSuccess
    // calls.
    verify(futureCallback, times(1)).onFailure(any());
    verify(futureCallback, times(0)).onSuccess(any());
    // Release lock so that target can acquire it.
    lock.unlock();
    final Optional<ListenableFuture<Boolean>> olf2 = checker.schedule(target, true);
    assertTrue(olf2.isPresent());
    Futures.addCallback(olf2.get(), futureCallback);
    // Wait for the callback
    Thread.sleep(DISK_CHECK_TIME);
    // Verify that normal check (dummy) results in only 1 onSuccess call.
    // Number of times onFailure is invoked should remain the same - 1.
    verify(futureCallback, times(1)).onFailure(any());
    verify(futureCallback, times(1)).onSuccess(any());
}
Also used : ListenableFuture(com.google.common.util.concurrent.ListenableFuture) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) FakeTimer(org.apache.hadoop.util.FakeTimer) Test(org.junit.Test)

Aggregations

FakeTimer (org.apache.hadoop.util.FakeTimer)33 Test (org.junit.Test)30 Groups (org.apache.hadoop.security.Groups)10 HdfsConfiguration (org.apache.hadoop.hdfs.HdfsConfiguration)9 Configuration (org.apache.hadoop.conf.Configuration)8 StorageLocation (org.apache.hadoop.hdfs.server.datanode.StorageLocation)6 ListenableFuture (com.google.common.util.concurrent.ListenableFuture)5 IOException (java.io.IOException)5 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 AtomicLong (java.util.concurrent.atomic.AtomicLong)3 TimeoutException (java.util.concurrent.TimeoutException)2 LogCapturer (org.apache.hadoop.test.GenericTestUtils.LogCapturer)2 Before (org.junit.Before)2 Optional (com.google.common.base.Optional)1 File (java.io.File)1 FileOutputStream (java.io.FileOutputStream)1 OutputStreamWriter (java.io.OutputStreamWriter)1 Writer (java.io.Writer)1 ArrayList (java.util.ArrayList)1 CountDownLatch (java.util.concurrent.CountDownLatch)1