Search in sources :

Example 61 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method testThrowTaskMigrateExceptionOnEosSendOffset.

private void testThrowTaskMigrateExceptionOnEosSendOffset(final RuntimeException exception) {
    // cannot use `eosMockProducer.fenceProducer()` because this would already trigger in `beginTransaction()`
    eosAlphaMockProducer.sendOffsetsToTransactionException = exception;
    final TaskMigratedException thrown = assertThrows(TaskMigratedException.class, // `sendOffsetsToTransaction()` would throw an NPE on `null` offsets
    () -> eosAlphaStreamsProducer.commitTransaction(null, new ConsumerGroupMetadata("appId")));
    assertThat(thrown.getCause(), is(eosAlphaMockProducer.sendOffsetsToTransactionException));
    assertThat(thrown.getMessage(), is("Producer got fenced trying to commit a transaction [test];" + " it means all tasks belonging to this thread should be migrated."));
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException)

Example 62 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldCommitTxWithConsumerGroupMetadataOnEosBetaCommit.

@Test
public void shouldCommitTxWithConsumerGroupMetadataOnEosBetaCommit() {
    mockedProducer.initTransactions();
    expectLastCall();
    mockedProducer.beginTransaction();
    expectLastCall();
    expect(mockedProducer.send(record, null)).andReturn(null);
    mockedProducer.sendOffsetsToTransaction(null, new ConsumerGroupMetadata("appId"));
    expectLastCall();
    mockedProducer.commitTransaction();
    expectLastCall();
    replay(mockedProducer);
    final StreamsProducer streamsProducer = new StreamsProducer(eosBetaConfig, "threadId-StreamThread-0", clientSupplier, null, UUID.randomUUID(), logContext, mockTime);
    streamsProducer.initTransaction();
    // call `send()` to start a transaction
    streamsProducer.send(record, null);
    streamsProducer.commitTransaction(null, new ConsumerGroupMetadata("appId"));
    verify(mockedProducer);
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) Test(org.junit.Test)

Example 63 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldThrowStreamsExceptionOnEosCommitTxError.

@Test
public void shouldThrowStreamsExceptionOnEosCommitTxError() {
    eosAlphaMockProducer.commitTransactionException = new KafkaException("KABOOM!");
    final StreamsException thrown = assertThrows(StreamsException.class, () -> eosAlphaStreamsProducer.commitTransaction(offsetsAndMetadata, new ConsumerGroupMetadata("appId")));
    assertThat(eosAlphaMockProducer.sentOffsets(), is(true));
    assertThat(thrown.getCause(), is(eosAlphaMockProducer.commitTransactionException));
    assertThat(thrown.getMessage(), is("Error encountered trying to commit a transaction [test]"));
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) StreamsException(org.apache.kafka.streams.errors.StreamsException) KafkaException(org.apache.kafka.common.KafkaException) Test(org.junit.Test)

Example 64 with ConsumerGroupMetadata

use of org.apache.kafka.clients.consumer.ConsumerGroupMetadata in project kafka by apache.

the class StreamsProducerTest method shouldBeginTxOnEosCommit.

@Test
public void shouldBeginTxOnEosCommit() {
    mockedProducer.initTransactions();
    mockedProducer.beginTransaction();
    mockedProducer.sendOffsetsToTransaction(offsetsAndMetadata, new ConsumerGroupMetadata("appId"));
    mockedProducer.commitTransaction();
    expectLastCall();
    replay(mockedProducer);
    eosAlphaStreamsProducerWithMock.initTransaction();
    eosAlphaStreamsProducerWithMock.commitTransaction(offsetsAndMetadata, new ConsumerGroupMetadata("appId"));
    verify(mockedProducer);
}
Also used : ConsumerGroupMetadata(org.apache.kafka.clients.consumer.ConsumerGroupMetadata) Test(org.junit.Test)

Aggregations

ConsumerGroupMetadata (org.apache.kafka.clients.consumer.ConsumerGroupMetadata)64 TopicPartition (org.apache.kafka.common.TopicPartition)34 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)29 Test (org.junit.Test)28 HashMap (java.util.HashMap)25 Test (org.junit.jupiter.api.Test)24 StreamsMetricsImpl (org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl)14 TaskId (org.apache.kafka.streams.processor.TaskId)12 AtomicLong (java.util.concurrent.atomic.AtomicLong)9 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)8 LogContext (org.apache.kafka.common.utils.LogContext)8 Collection (java.util.Collection)7 LinkedList (java.util.LinkedList)7 Map (java.util.Map)7 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)7 KafkaException (org.apache.kafka.common.KafkaException)7 TimeoutException (org.apache.kafka.common.errors.TimeoutException)7 TaskCorruptedException (org.apache.kafka.streams.errors.TaskCorruptedException)7 Collections.emptySet (java.util.Collections.emptySet)6 HashSet (java.util.HashSet)6