Examples with IncomingKafkaRecord io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord used on opensource projects

Example 1 with IncomingKafkaRecord

use of io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord in project smallrye-reactive-messaging by smallrye.

the class ReactiveKafkaConsumerTest method testThrottledAcknowledgementWithDefaultValues.

@Test
public void testThrottledAcknowledgementWithDefaultValues() throws Exception {
    String groupId = UUID.randomUUID().toString();
    MapBasedConfig config = createConsumerConfig(groupId).put("topic", topic);
    Multi<IncomingKafkaRecord<Integer, String>> stream = createSource(config, groupId).getStream().invoke(IncomingKafkaRecord::ack);
    sendReceive(stream, 0, 100, 0, 100);
    restartAndCheck(config, groupId, 1);
}

Example 2 with IncomingKafkaRecord

use of io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord in project smallrye-reactive-messaging by smallrye.

the class ReactiveKafkaConsumerTest method testWithMultipleConsumersWithAMultipleConsumeGroup.

@Test
public void testWithMultipleConsumersWithAMultipleConsumeGroup() throws Exception {
    int count = 100;
    CountDownLatch latch = new CountDownLatch(count * partitions);
    List<KafkaSource<Integer, String>> sources = new ArrayList<>();
    for (int i = 0; i < partitions; i++) {
        String groupId = UUID.randomUUID().toString();
        MapBasedConfig config = createConsumerConfig(groupId).with("topic", topic).with(ConsumerConfig.CLIENT_ID_CONFIG, "client-" + i).with(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        KafkaSource<Integer, String> source = createSource(config, groupId);
        sources.add(source);
        source.getStream().invoke(rec -> {
            onReceive(rec);
            latch.countDown();
        }).subscribe().with(IncomingKafkaRecord::ack);
        await().until(() -> !source.getConsumer().getAssignments().await().indefinitely().isEmpty());
    }
    sendMessages(0, count);
    waitForMessages(latch);
    assertThat(sources).hasSize(4);
    assertThat(sources).allSatisfy(s -> assertThat(s.getConsumer().getAssignments().await().indefinitely()).hasSize(partitions));
    sources.forEach(KafkaSource::closeQuietly);
}

Example 3 with IncomingKafkaRecord

use of io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord in project smallrye-reactive-messaging by smallrye.

the class ReactiveKafkaConsumerTest method testAutoCommitAcknowledgement.

@Test
public void testAutoCommitAcknowledgement() throws Exception {
    String groupId = UUID.randomUUID().toString();
    MapBasedConfig config = createConsumerConfig(groupId).with("topic", topic).with(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, true);
    Multi<IncomingKafkaRecord<Integer, String>> stream = createSource(config, groupId).getStream().invoke(IncomingKafkaRecord::ack);
    sendReceive(stream, 0, 100, 0, 100);
    restartAndCheck(config, groupId, 1);
}

Example 4 with IncomingKafkaRecord

use of io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord in project smallrye-reactive-messaging by smallrye.

the class ReactiveKafkaConsumerTest method testMessageProcessingDelay.

/**
 * Tests that delays in message processing dont cause session timeouts.
 * Kafka consumer heartbeat thread should keep the session alive.
 */
@Test
public void testMessageProcessingDelay() throws Exception {
    int count = 3;
    String groupId = UUID.randomUUID().toString();
    MapBasedConfig config = createConsumerConfig(groupId).put("topic", topic);
    KafkaSource<Integer, String> source = createSource(config, groupId);
    AssertSubscriber<IncomingKafkaRecord<Integer, String>> subscriber = source.getStream().select().first(count).subscribe().withSubscriber(AssertSubscriber.create(0));
    sendMessages(0, count);
    for (int i = 0; i < count; i++) {
        subscriber.request(1);
        int l = i + 1;
        await().until(() -> subscriber.getItems().size() == l);
        Thread.sleep(sessionTimeoutMillis + 1000);
    }
    subscriber.awaitCompletion();
}

Example 5 with IncomingKafkaRecord

use of io.smallrye.reactive.messaging.kafka.IncomingKafkaRecord in project smallrye-reactive-messaging by smallrye.

the class ReactiveKafkaConsumerTest method testAcknowledgementUsingThrottledStrategy.

@Test
public void testAcknowledgementUsingThrottledStrategy() throws Exception {
    String groupId = UUID.randomUUID().toString();
    MapBasedConfig config = createConsumerConfig(groupId).with("topic", topic);
    Multi<IncomingKafkaRecord<Integer, String>> stream = createSource(config, groupId).getStream().invoke(IncomingKafkaRecord::ack);
    sendReceive(stream, 0, 100, 0, 100);
    waitForCommits(source, 100);
    // Close consumer and create another one. First consumer should commit final offset on close.
    // Second consumer should receive only new messages.
    cancelSubscriptions();
    clearReceivedMessages();
    config.with(ConsumerConfig.CLIENT_ID_CONFIG, "second-consumer");
    Multi<IncomingKafkaRecord<Integer, String>> stream2 = createSource(config, groupId).getStream();
    sendReceive(stream2, 100, 100, 100, 100);
}