Search in sources :

Example 31 with InMemorySystemDescriptor

use of org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor in project samza by apache.

the class AsyncStreamTaskIntegrationTest method testAsyncTaskWithMultiplePartitionMultithreaded.

@Test
public void testAsyncTaskWithMultiplePartitionMultithreaded() throws Exception {
    Map<Integer, List<KV>> inputPartitionData = new HashMap<>();
    Map<Integer, List<Integer>> expectedOutputPartitionData = new HashMap<>();
    genData(inputPartitionData, expectedOutputPartitionData);
    InMemorySystemDescriptor isd = new InMemorySystemDescriptor("async-test");
    InMemoryInputDescriptor<KV> imid = isd.getInputDescriptor("ints", new NoOpSerde<>());
    InMemoryOutputDescriptor imod = isd.getOutputDescriptor("ints-out", new NoOpSerde<>());
    TestRunner.of(MyAsyncStreamTask.class).addInputStream(imid, inputPartitionData).addOutputStream(imod, 5).addConfig("task.max.concurrency", "4").run(Duration.ofSeconds(2));
    StreamAssert.containsInAnyOrder(expectedOutputPartitionData, imod, Duration.ofMillis(1000));
}
Also used : InMemoryOutputDescriptor(org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) List(java.util.List) KV(org.apache.samza.operators.KV) InMemorySystemDescriptor(org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor) Test(org.junit.Test)

Example 32 with InMemorySystemDescriptor

use of org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor in project samza by apache.

the class AsyncStreamTaskIntegrationTest method testAsyncTaskWithSinglePartitionUsingStreamAssert.

@Test
public void testAsyncTaskWithSinglePartitionUsingStreamAssert() throws Exception {
    List<Integer> inputList = Arrays.asList(1, 2, 3, 4, 5);
    List<Integer> outputList = Arrays.asList(50, 10, 20, 30, 40);
    InMemorySystemDescriptor isd = new InMemorySystemDescriptor("async-test");
    InMemoryInputDescriptor<Integer> imid = isd.getInputDescriptor("ints", new NoOpSerde<Integer>());
    InMemoryOutputDescriptor imod = isd.getOutputDescriptor("ints-out", new NoOpSerde<>());
    TestRunner.of(MyAsyncStreamTask.class).addInputStream(imid, inputList).addOutputStream(imod, 1).run(Duration.ofSeconds(2));
    StreamAssert.containsInAnyOrder(outputList, imod, Duration.ofMillis(1000));
}
Also used : InMemoryOutputDescriptor(org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor) InMemorySystemDescriptor(org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor) Test(org.junit.Test)

Example 33 with InMemorySystemDescriptor

use of org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor in project samza by apache.

the class AsyncStreamTaskIntegrationTest method testSamzaJobTimeoutFailureForAsyncTask.

/**
 * Job should fail because it times out too soon
 */
@Test(expected = SamzaException.class)
public void testSamzaJobTimeoutFailureForAsyncTask() {
    InMemorySystemDescriptor isd = new InMemorySystemDescriptor("async-test");
    InMemoryInputDescriptor<Integer> imid = isd.getInputDescriptor("ints", new NoOpSerde<>());
    InMemoryOutputDescriptor imod = isd.getOutputDescriptor("ints-out", new NoOpSerde<>());
    TestRunner.of(MyAsyncStreamTask.class).addInputStream(imid, Arrays.asList(1, 2, 3, 4)).addOutputStream(imod, 1).run(Duration.ofMillis(1));
}
Also used : InMemoryOutputDescriptor(org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor) InMemorySystemDescriptor(org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor) Test(org.junit.Test)

Example 34 with InMemorySystemDescriptor

use of org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor in project samza by apache.

the class AsyncStreamTaskIntegrationTest method testAsyncTaskWithMultiplePartition.

@Test
public void testAsyncTaskWithMultiplePartition() throws Exception {
    Map<Integer, List<KV>> inputPartitionData = new HashMap<>();
    Map<Integer, List<Integer>> expectedOutputPartitionData = new HashMap<>();
    genData(inputPartitionData, expectedOutputPartitionData);
    InMemorySystemDescriptor isd = new InMemorySystemDescriptor("async-test");
    InMemoryInputDescriptor<KV> imid = isd.getInputDescriptor("ints", new NoOpSerde<KV>());
    InMemoryOutputDescriptor imod = isd.getOutputDescriptor("ints-out", new NoOpSerde<>());
    TestRunner.of(MyAsyncStreamTask.class).addInputStream(imid, inputPartitionData).addOutputStream(imod, 5).run(Duration.ofSeconds(2));
    StreamAssert.containsInOrder(expectedOutputPartitionData, imod, Duration.ofMillis(1000));
}
Also used : InMemoryOutputDescriptor(org.apache.samza.test.framework.system.descriptors.InMemoryOutputDescriptor) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) List(java.util.List) KV(org.apache.samza.operators.KV) InMemorySystemDescriptor(org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor) Test(org.junit.Test)

Aggregations

InMemorySystemDescriptor (org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor)34 Test (org.junit.Test)29 KV (org.apache.samza.operators.KV)17 ArrayList (java.util.ArrayList)16 List (java.util.List)16 NoOpSerde (org.apache.samza.serializers.NoOpSerde)13 HashMap (java.util.HashMap)12 StreamApplication (org.apache.samza.application.StreamApplication)12 Duration (java.time.Duration)11 InMemoryInputDescriptor (org.apache.samza.test.framework.system.descriptors.InMemoryInputDescriptor)11 PageView (org.apache.samza.test.table.TestTableData.PageView)11 DelegatingSystemDescriptor (org.apache.samza.system.descriptors.DelegatingSystemDescriptor)10 GenericInputDescriptor (org.apache.samza.system.descriptors.GenericInputDescriptor)10 Table (org.apache.samza.table.Table)10 TestRunner (org.apache.samza.test.framework.TestRunner)10 EnrichedPageView (org.apache.samza.test.table.TestTableData.EnrichedPageView)10 Profile (org.apache.samza.test.table.TestTableData.Profile)10 Arrays (java.util.Arrays)9 StreamApplicationDescriptor (org.apache.samza.application.descriptors.StreamApplicationDescriptor)9 Assert (org.junit.Assert)9