Search in sources :

Example 21 with AsyncReadWriteUpdateTable

use of org.apache.samza.table.AsyncReadWriteUpdateTable in project samza by apache.

the class TestAsyncRetriableTable method testUpdateWithPermFailureOnMaxCount.

@Test
public void testUpdateWithPermFailureOnMaxCount() {
    TableRetryPolicy policy = new TableRetryPolicy();
    policy.withFixedBackoff(Duration.ofMillis(5));
    policy.withStopAfterAttempts(5);
    TableReadFunction<String, String> readFn = mock(TableReadFunction.class);
    TableWriteFunction<String, String, String> writeFn = mock(TableWriteFunction.class);
    doReturn(true).when(writeFn).isRetriable(any());
    CompletableFuture<String> future = new CompletableFuture();
    future.completeExceptionally(new RuntimeException("test exception"));
    doReturn(future).when(writeFn).updateAsync(any(), any());
    AsyncReadWriteUpdateTable delegate = new AsyncRemoteTable(readFn, writeFn);
    AsyncRetriableTable table = new AsyncRetriableTable("t1", delegate, null, policy, schedExec, readFn, writeFn);
    table.init(TestRemoteTable.getMockContext());
    try {
        table.updateAsync(1, 2).join();
        fail();
    } catch (Throwable t) {
    }
    verify(writeFn, atLeast(6)).updateAsync(any(), any());
    assertEquals(5, table.writeRetryMetrics.retryCount.getCount());
    assertEquals(0, table.writeRetryMetrics.successCount.getCount());
    assertEquals(1, table.writeRetryMetrics.permFailureCount.getCount());
    assertTrue(table.writeRetryMetrics.retryTimer.getSnapshot().getMax() > 0);
}
Also used : CompletableFuture(java.util.concurrent.CompletableFuture) AsyncRemoteTable(org.apache.samza.table.remote.AsyncRemoteTable) AsyncReadWriteUpdateTable(org.apache.samza.table.AsyncReadWriteUpdateTable) Test(org.junit.Test)

Example 22 with AsyncReadWriteUpdateTable

use of org.apache.samza.table.AsyncReadWriteUpdateTable in project samza by apache.

the class TestAsyncRetriableTable method testGetAllWithOneRetry.

@Test
public void testGetAllWithOneRetry() {
    TableRetryPolicy policy = new TableRetryPolicy();
    policy.withFixedBackoff(Duration.ofMillis(10));
    TableReadFunction<String, String> readFn = mock(TableReadFunction.class);
    doReturn(true).when(readFn).isRetriable(any());
    AtomicInteger times = new AtomicInteger();
    Map<String, String> map = new HashMap<>();
    map.put("foo1", "bar1");
    map.put("foo2", "bar2");
    doAnswer(invocation -> {
        CompletableFuture<Map<String, String>> future = new CompletableFuture();
        if (times.get() > 0) {
            future.complete(map);
        } else {
            times.incrementAndGet();
            future.completeExceptionally(new RuntimeException("test exception"));
        }
        return future;
    }).when(readFn).getAllAsync(anyCollection());
    AsyncReadWriteUpdateTable delegate = new AsyncRemoteTable(readFn, null);
    AsyncRetriableTable table = new AsyncRetriableTable("t1", delegate, policy, null, schedExec, readFn, null);
    table.init(TestRemoteTable.getMockContext());
    assertEquals(map, table.getAllAsync(Arrays.asList("foo1", "foo2")).join());
    verify(readFn, times(2)).getAllAsync(any());
    assertEquals(1, table.readRetryMetrics.retryCount.getCount());
    assertEquals(0, table.readRetryMetrics.successCount.getCount());
    assertEquals(0, table.readRetryMetrics.permFailureCount.getCount());
    assertTrue(table.readRetryMetrics.retryTimer.getSnapshot().getMax() > 0);
}
Also used : AsyncRemoteTable(org.apache.samza.table.remote.AsyncRemoteTable) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) AsyncReadWriteUpdateTable(org.apache.samza.table.AsyncReadWriteUpdateTable) HashMap(java.util.HashMap) Map(java.util.Map) Test(org.junit.Test)

Aggregations

AsyncReadWriteUpdateTable (org.apache.samza.table.AsyncReadWriteUpdateTable)22 AsyncRemoteTable (org.apache.samza.table.remote.AsyncRemoteTable)21 Test (org.junit.Test)21 CompletableFuture (java.util.concurrent.CompletableFuture)12 TableReadFunction (org.apache.samza.table.remote.TableReadFunction)8 HashMap (java.util.HashMap)7 TableWriteFunction (org.apache.samza.table.remote.TableWriteFunction)6 Map (java.util.Map)5 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 Entry (org.apache.samza.storage.kv.Entry)4 TableRateLimiter (org.apache.samza.table.remote.TableRateLimiter)3 AsyncRateLimitedTable (org.apache.samza.table.ratelimit.AsyncRateLimitedTable)2 AsyncRetriableTable (org.apache.samza.table.retry.AsyncRetriableTable)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 ThreadPoolExecutor (java.util.concurrent.ThreadPoolExecutor)1 Config (org.apache.samza.config.Config)1 JavaTableConfig (org.apache.samza.config.JavaTableConfig)1 MapConfig (org.apache.samza.config.MapConfig)1 TaskName (org.apache.samza.container.TaskName)1 ContainerContext (org.apache.samza.context.ContainerContext)1