Search in sources :

Example 6 with PageView

use of org.apache.samza.test.operator.data.PageView in project samza by apache.

the class BroadcastAssertApp method describe.

@Override
public void describe(StreamApplicationDescriptor appDescriptor) {
    Config config = appDescriptor.getConfig();
    String inputTopic = config.get(INPUT_TOPIC_NAME_PROP);
    final JsonSerdeV2<PageView> serde = new JsonSerdeV2<>(PageView.class);
    KafkaSystemDescriptor ksd = new KafkaSystemDescriptor(SYSTEM);
    KafkaInputDescriptor<PageView> isd = ksd.getInputDescriptor(inputTopic, serde);
    final MessageStream<PageView> broadcastPageViews = appDescriptor.getInputStream(isd).broadcast(serde, "pv");
    /**
     * Each task will see all the pageview events
     */
    MessageStreamAssert.that("Each task contains all broadcast PageView events", broadcastPageViews, serde).forEachTask().containsInAnyOrder(Arrays.asList(new PageView("v1", "p1", "u1"), new PageView("v2", "p2", "u1"), new PageView("v3", "p1", "u2"), new PageView("v4", "p3", "u2")));
}
Also used : PageView(org.apache.samza.test.operator.data.PageView) Config(org.apache.samza.config.Config) KafkaSystemDescriptor(org.apache.samza.system.kafka.descriptors.KafkaSystemDescriptor) JsonSerdeV2(org.apache.samza.serializers.JsonSerdeV2)

Example 7 with PageView

use of org.apache.samza.test.operator.data.PageView in project samza by apache.

the class TestAsyncFlatMap method runTest.

private List<PageView> runTest(Map<String, String> configs) {
    InMemorySystemDescriptor isd = new InMemorySystemDescriptor(TEST_SYSTEM);
    InMemoryInputDescriptor<PageView> pageViewStreamDesc = isd.getInputDescriptor(PAGE_VIEW_STREAM, new NoOpSerde<>());
    InMemoryOutputDescriptor<PageView> outputStreamDesc = isd.getOutputDescriptor(NON_GUEST_PAGE_VIEW_STREAM, new NoOpSerde<>());
    TestRunner.of(new AsyncFlatMapExample()).addInputStream(pageViewStreamDesc, PAGE_VIEWS).addOutputStream(outputStreamDesc, 1).addConfig(new MapConfig(configs)).run(Duration.ofSeconds(10));
    Map<Integer, List<PageView>> result = TestRunner.consumeStream(outputStreamDesc, Duration.ofMillis(1000));
    return result.values().stream().flatMap(List::stream).collect(Collectors.toList());
}
Also used : PageView(org.apache.samza.test.operator.data.PageView) ImmutableList(com.google.common.collect.ImmutableList) List(java.util.List) MapConfig(org.apache.samza.config.MapConfig) InMemorySystemDescriptor(org.apache.samza.test.framework.system.descriptors.InMemorySystemDescriptor)

Aggregations

PageView (org.apache.samza.test.operator.data.PageView)7 JsonSerdeV2 (org.apache.samza.serializers.JsonSerdeV2)5 KafkaSystemDescriptor (org.apache.samza.system.kafka.descriptors.KafkaSystemDescriptor)5 Config (org.apache.samza.config.Config)4 KV (org.apache.samza.operators.KV)4 Duration (java.time.Duration)3 List (java.util.List)3 StreamApplication (org.apache.samza.application.StreamApplication)3 StreamApplicationDescriptor (org.apache.samza.application.descriptors.StreamApplicationDescriptor)3 MessageStream (org.apache.samza.operators.MessageStream)3 Windows (org.apache.samza.operators.windows.Windows)3 KVSerde (org.apache.samza.serializers.KVSerde)3 StringSerde (org.apache.samza.serializers.StringSerde)3 KafkaInputDescriptor (org.apache.samza.system.kafka.descriptors.KafkaInputDescriptor)3 ImmutableList (com.google.common.collect.ImmutableList)2 OutputStream (org.apache.samza.operators.OutputStream)2 ApplicationRunner (org.apache.samza.runtime.ApplicationRunner)2 ApplicationRunners (org.apache.samza.runtime.ApplicationRunners)2 IntegerSerde (org.apache.samza.serializers.IntegerSerde)2 KafkaOutputDescriptor (org.apache.samza.system.kafka.descriptors.KafkaOutputDescriptor)2