Search in sources :

Example 1 with IncomingKafkaRecordBatchMetadata

use of io.smallrye.reactive.messaging.kafka.api.IncomingKafkaRecordBatchMetadata in project smallrye-reactive-messaging by smallrye.

the class BatchConsumerTest method testIncomingConsumingMessageWithMetadata.

@Test
void testIncomingConsumingMessageWithMetadata() {
    String newTopic = UUID.randomUUID().toString();
    companion.topics().createAndWait(newTopic, 3);
    KafkaMapBasedConfig config = kafkaConfig("mp.messaging.incoming.kafka").put("value.deserializer", StringDeserializer.class.getName()).put("auto.offset.reset", "earliest").put("topic", newTopic).put("batch", true);
    BeanConsumingMessageWithBatchMetadata bean = runApplication(config, BeanConsumingMessageWithBatchMetadata.class);
    companion.produceStrings().usingGenerator(i -> new ProducerRecord<>(newTopic, "k-" + i, "v-" + i), 10);
    await().until(() -> bean.metadata().stream().mapToInt(m -> m.getRecords().count()).sum() == 10);
    Map<TopicPartition, List<ConsumerRecord<String, String>>> records = new HashMap<>();
    for (IncomingKafkaRecordBatchMetadata<String, String> metadata : bean.metadata()) {
        for (TopicPartition partition : metadata.getRecords().partitions()) {
            List<ConsumerRecord<String, String>> list = records.computeIfAbsent(partition, p -> new ArrayList<>());
            list.addAll(metadata.getRecords().records(partition));
        }
    }
    assertThat(records.keySet()).hasSize(3);
    assertThat(records.values()).flatMap(l -> l).hasSize(10).allSatisfy(r -> {
        assertThat(r.value()).startsWith("v-");
        assertThat(r.key()).startsWith("k");
    });
}
Also used : TopicPartition(org.apache.kafka.common.TopicPartition) Awaitility.await(org.awaitility.Awaitility.await) ProducerRecord(org.apache.kafka.clients.producer.ProducerRecord) KafkaCompanionTestBase(io.smallrye.reactive.messaging.kafka.base.KafkaCompanionTestBase) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) HashMap(java.util.HashMap) UUID(java.util.UUID) KafkaMapBasedConfig(io.smallrye.reactive.messaging.kafka.base.KafkaMapBasedConfig) ArrayList(java.util.ArrayList) Test(org.junit.jupiter.api.Test) List(java.util.List) Message(org.eclipse.microprofile.reactive.messaging.Message) CompletionStage(java.util.concurrent.CompletionStage) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) Map(java.util.Map) ApplicationScoped(javax.enterprise.context.ApplicationScoped) IncomingKafkaRecordBatchMetadata(io.smallrye.reactive.messaging.kafka.api.IncomingKafkaRecordBatchMetadata) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) Incoming(org.eclipse.microprofile.reactive.messaging.Incoming) HashMap(java.util.HashMap) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) TopicPartition(org.apache.kafka.common.TopicPartition) ArrayList(java.util.ArrayList) List(java.util.List) CopyOnWriteArrayList(java.util.concurrent.CopyOnWriteArrayList) KafkaMapBasedConfig(io.smallrye.reactive.messaging.kafka.base.KafkaMapBasedConfig) Test(org.junit.jupiter.api.Test)

Aggregations

IncomingKafkaRecordBatchMetadata (io.smallrye.reactive.messaging.kafka.api.IncomingKafkaRecordBatchMetadata)1 KafkaCompanionTestBase (io.smallrye.reactive.messaging.kafka.base.KafkaCompanionTestBase)1 KafkaMapBasedConfig (io.smallrye.reactive.messaging.kafka.base.KafkaMapBasedConfig)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 List (java.util.List)1 Map (java.util.Map)1 UUID (java.util.UUID)1 CompletionStage (java.util.concurrent.CompletionStage)1 CopyOnWriteArrayList (java.util.concurrent.CopyOnWriteArrayList)1 ApplicationScoped (javax.enterprise.context.ApplicationScoped)1 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)1 ProducerRecord (org.apache.kafka.clients.producer.ProducerRecord)1 TopicPartition (org.apache.kafka.common.TopicPartition)1 StringDeserializer (org.apache.kafka.common.serialization.StringDeserializer)1 Assertions.assertThat (org.assertj.core.api.Assertions.assertThat)1 Awaitility.await (org.awaitility.Awaitility.await)1 Incoming (org.eclipse.microprofile.reactive.messaging.Incoming)1 Message (org.eclipse.microprofile.reactive.messaging.Message)1 Test (org.junit.jupiter.api.Test)1