Search in sources :

Example 26 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamThreadTest method shouldOnlyShutdownOnce.

@Test
public void shouldOnlyShutdownOnce() {
    final Consumer<byte[], byte[]> consumer = EasyMock.createNiceMock(Consumer.class);
    final ConsumerGroupMetadata consumerGroupMetadata = mock(ConsumerGroupMetadata.class);
    expect(consumer.groupMetadata()).andStubReturn(consumerGroupMetadata);
    expect(consumerGroupMetadata.groupInstanceId()).andReturn(Optional.empty());
    EasyMock.replay(consumerGroupMetadata);
    final TaskManager taskManager = EasyMock.createNiceMock(TaskManager.class);
    expect(taskManager.producerClientIds()).andStubReturn(Collections.emptySet());
    taskManager.shutdown(true);
    EasyMock.expectLastCall();
    EasyMock.replay(taskManager, consumer);
    final StreamsMetricsImpl streamsMetrics = new StreamsMetricsImpl(metrics, CLIENT_ID, StreamsConfig.METRICS_LATEST, mockTime);
    final TopologyMetadata topologyMetadata = new TopologyMetadata(internalTopologyBuilder, config);
    topologyMetadata.buildAndRewriteTopology();
    final StreamThread thread = buildStreamThread(consumer, taskManager, config, topologyMetadata).updateThreadMetadata(getSharedAdminClientId(CLIENT_ID));
    thread.shutdown();
    // Execute the run method. Verification of the mock will check that shutdown was only done once
    thread.run();
    verify(taskManager);
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) StreamsMetricsImpl(org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl) Test(org.junit.Test)

Example 27 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldSendOffsetToTxOnEosCommit.

@Test
public void shouldSendOffsetToTxOnEosCommit() {
    eosAlphaStreamsProducer.commitTransaction(offsetsAndMetadata, new ConsumerGroupMetadata("appId"));
    assertThat(eosAlphaMockProducer.sentOffsets(), is(true));
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) Test(org.junit.Test)

Example 28 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldCommitTxOnEosCommit.

@Test
public void shouldCommitTxOnEosCommit() {
    eosAlphaStreamsProducer.send(record, null);
    assertThat(eosAlphaMockProducer.transactionInFlight(), is(true));
    eosAlphaStreamsProducer.commitTransaction(offsetsAndMetadata, new ConsumerGroupMetadata("appId"));
    assertThat(eosAlphaMockProducer.transactionInFlight(), is(false));
    assertThat(eosAlphaMockProducer.uncommittedRecords().isEmpty(), is(true));
    assertThat(eosAlphaMockProducer.uncommittedOffsets().isEmpty(), is(true));
    assertThat(eosAlphaMockProducer.history().size(), is(1));
    assertThat(eosAlphaMockProducer.history().get(0), is(record));
    assertThat(eosAlphaMockProducer.consumerGroupOffsetsHistory().size(), is(1));
    assertThat(eosAlphaMockProducer.consumerGroupOffsetsHistory().get(0).get("appId"), is(offsetsAndMetadata));
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) Test(org.junit.Test)

Example 29 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldCommitTxWithApplicationIdOnEosAlphaCommit.

@Test
public void shouldCommitTxWithApplicationIdOnEosAlphaCommit() {
    mockedProducer.initTransactions();
    expectLastCall();
    mockedProducer.beginTransaction();
    expectLastCall();
    expect(mockedProducer.send(record, null)).andReturn(null);
    mockedProducer.sendOffsetsToTransaction(null, new ConsumerGroupMetadata("appId"));
    expectLastCall();
    mockedProducer.commitTransaction();
    expectLastCall();
    replay(mockedProducer);
    eosAlphaStreamsProducerWithMock.initTransaction();
    // call `send()` to start a transaction
    eosAlphaStreamsProducerWithMock.send(record, null);
    eosAlphaStreamsProducerWithMock.commitTransaction(null, new ConsumerGroupMetadata("appId"));
    verify(mockedProducer);
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) Test(org.junit.Test)

Example 30 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldThrowStreamsExceptionOnEosSendOffsetError.

@Test
public void shouldThrowStreamsExceptionOnEosSendOffsetError() {
    eosAlphaMockProducer.sendOffsetsToTransactionException = new KafkaException("KABOOM!");
    final StreamsException thrown = assertThrows(StreamsException.class, // `sendOffsetsToTransaction()` would throw an NPE on `null` offsets
    () -> eosAlphaStreamsProducer.commitTransaction(null, new ConsumerGroupMetadata("appId")));
    assertThat(thrown.getCause(), is(eosAlphaMockProducer.sendOffsetsToTransactionException));
    assertThat(thrown.getMessage(), is("Error encountered trying to commit a transaction [test]"));
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) StreamsException(org.apache.kafka.streams.errors.StreamsException) KafkaException(org.apache.kafka.common.KafkaException) Test(org.junit.Test)

Aggregations

ConsumerGroupMetadata (org.apache.kafka.clients.consumer.ConsumerGroupMetadata)64 TopicPartition (org.apache.kafka.common.TopicPartition)34 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)29 Test (org.junit.Test)28 HashMap (java.util.HashMap)25 Test (org.junit.jupiter.api.Test)24 StreamsMetricsImpl (org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)14 TaskId (org.apache.kafka.streams.processor.TaskId)12 AtomicLong (java.util.concurrent.atomic.AtomicLong)9 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)8 LogContext (org.apache.kafka.common.utils.LogContext)8 Collection (java.util.Collection)7 LinkedList (java.util.LinkedList)7 Map (java.util.Map)7 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)7 KafkaException (org.apache.kafka.common.KafkaException)7 TimeoutException (org.apache.kafka.common.errors.TimeoutException)7 TaskCorruptedException (org.apache.kafka.streams.errors.TaskCorruptedException)7 Collections.emptySet (java.util.Collections.emptySet)6 HashSet (java.util.HashSet)6