Search in sources :

Example 1 with TaskMigratedException

use of org.apache.kafka.streams.errors.TaskMigratedException in project apache-kafka-on-k8s by banzaicloud.

the class StreamThreadTest method shouldCloseTaskAsZombieAndRemoveFromActiveTasksIfProducerGotFencedAtBeginTransactionWhenTaskIsResumed.

@Test
public void shouldCloseTaskAsZombieAndRemoveFromActiveTasksIfProducerGotFencedAtBeginTransactionWhenTaskIsResumed() {
    internalTopologyBuilder.addSource(null, "name", null, null, null, topic1);
    internalTopologyBuilder.addSink("out", "output", null, null, null);
    final StreamThread thread = createStreamThread(clientId, new StreamsConfig(configProps(true)), true);
    thread.setState(StreamThread.State.RUNNING);
    thread.rebalanceListener.onPartitionsRevoked(null);
    final Map<TaskId, Set<TopicPartition>> activeTasks = new HashMap<>();
    final List<TopicPartition> assignedPartitions = new ArrayList<>();
    // assign single partition
    assignedPartitions.add(t1p1);
    activeTasks.put(task1, Collections.singleton(t1p1));
    thread.taskManager().setAssignmentMetadata(activeTasks, Collections.<TaskId, Set<TopicPartition>>emptyMap());
    final MockConsumer<byte[], byte[]> mockConsumer = (MockConsumer<byte[], byte[]>) thread.consumer;
    mockConsumer.assign(assignedPartitions);
    mockConsumer.updateBeginningOffsets(Collections.singletonMap(t1p1, 0L));
    thread.rebalanceListener.onPartitionsAssigned(assignedPartitions);
    thread.runOnce(-1);
    assertThat(thread.tasks().size(), equalTo(1));
    thread.rebalanceListener.onPartitionsRevoked(null);
    clientSupplier.producers.get(0).fenceProducer();
    thread.rebalanceListener.onPartitionsAssigned(assignedPartitions);
    try {
        thread.runOnce(-1);
        fail("Should have thrown TaskMigratedException");
    } catch (final TaskMigratedException expected) {
    /* ignore */
    }
    assertTrue(thread.tasks().isEmpty());
}
Also used : TaskId(org.apache.kafka.streams.processor.TaskId) Set(java.util.Set) HashSet(java.util.HashSet) HashMap(java.util.HashMap) TopicPartition(org.apache.kafka.common.TopicPartition) ArrayList(java.util.ArrayList) MockConsumer(org.apache.kafka.clients.consumer.MockConsumer) StreamsConfig(org.apache.kafka.streams.StreamsConfig) TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException) InternalStreamsBuilderTest(org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest) Test(org.junit.Test)

Example 2 with TaskMigratedException

use of org.apache.kafka.streams.errors.TaskMigratedException in project apache-kafka-on-k8s by banzaicloud.

the class AssignedStreamsTasksTest method shouldCloseTaskOnProcessesIfTaskMigratedException.

@Test
public void shouldCloseTaskOnProcessesIfTaskMigratedException() {
    mockTaskInitialization();
    t1.process();
    EasyMock.expectLastCall().andThrow(new TaskMigratedException(t1));
    t1.close(false, true);
    EasyMock.expectLastCall();
    EasyMock.replay(t1);
    addAndInitTask();
    try {
        assignedTasks.process();
        fail("Should have thrown TaskMigratedException.");
    } catch (final TaskMigratedException expected) {
    /* ignore */
    }
    assertThat(assignedTasks.runningTaskIds(), equalTo(Collections.EMPTY_SET));
    EasyMock.verify(t1);
}
Also used : TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException) Test(org.junit.Test)

Example 3 with TaskMigratedException

use of org.apache.kafka.streams.errors.TaskMigratedException in project apache-kafka-on-k8s by banzaicloud.

the class AssignedStreamsTasksTest method shouldCloseTaskOnMaybePunctuateSystemTimeIfTaskMigratedException.

@Test
public void shouldCloseTaskOnMaybePunctuateSystemTimeIfTaskMigratedException() {
    mockTaskInitialization();
    EasyMock.expect(t1.maybePunctuateStreamTime()).andReturn(true);
    t1.maybePunctuateSystemTime();
    EasyMock.expectLastCall().andThrow(new TaskMigratedException(t1));
    t1.close(false, true);
    EasyMock.expectLastCall();
    EasyMock.replay(t1);
    addAndInitTask();
    try {
        assignedTasks.punctuate();
        fail("Should have thrown TaskMigratedException.");
    } catch (final TaskMigratedException expected) {
    /* ignore */
    }
    EasyMock.verify(t1);
}
Also used : TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException) Test(org.junit.Test)

Example 4 with TaskMigratedException

use of org.apache.kafka.streams.errors.TaskMigratedException in project apache-kafka-on-k8s by banzaicloud.

the class AssignedStreamsTasksTest method shouldCloseTaskOnMaybeCommitIfTaskMigratedException.

@Test
public void shouldCloseTaskOnMaybeCommitIfTaskMigratedException() {
    mockTaskInitialization();
    EasyMock.expect(t1.commitNeeded()).andReturn(true);
    t1.commit();
    EasyMock.expectLastCall().andThrow(new TaskMigratedException(t1));
    t1.close(false, true);
    EasyMock.expectLastCall();
    EasyMock.replay(t1);
    addAndInitTask();
    try {
        assignedTasks.maybeCommit();
        fail("Should have thrown TaskMigratedException.");
    } catch (final TaskMigratedException expected) {
    /* ignore */
    }
    assertThat(assignedTasks.runningTaskIds(), equalTo(Collections.EMPTY_SET));
    EasyMock.verify(t1);
}
Also used : TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException) Test(org.junit.Test)

Example 5 with TaskMigratedException

use of org.apache.kafka.streams.errors.TaskMigratedException in project apache-kafka-on-k8s by banzaicloud.

the class AssignedStreamsTasksTest method shouldCloseTaskOnResumeSuspendedIfTaskMigratedException.

@Test
public void shouldCloseTaskOnResumeSuspendedIfTaskMigratedException() {
    mockRunningTaskSuspension();
    t1.resume();
    t1.initializeTopology();
    EasyMock.expectLastCall().andThrow(new TaskMigratedException(t1));
    t1.close(false, true);
    EasyMock.expectLastCall();
    EasyMock.replay(t1);
    assertThat(suspendTask(), nullValue());
    try {
        assignedTasks.maybeResumeSuspendedTask(taskId1, Collections.singleton(tp1));
        fail("Should have thrown TaskMigratedException.");
    } catch (final TaskMigratedException expected) {
    /* ignore */
    }
    assertThat(assignedTasks.runningTaskIds(), equalTo(Collections.EMPTY_SET));
    EasyMock.verify(t1);
}
Also used : TaskMigratedException(org.apache.kafka.streams.errors.TaskMigratedException) Test(org.junit.Test)

Aggregations

TaskMigratedException (org.apache.kafka.streams.errors.TaskMigratedException)16 Test (org.junit.Test)9 TopicPartition (org.apache.kafka.common.TopicPartition)6 HashMap (java.util.HashMap)4 ProducerFencedException (org.apache.kafka.common.errors.ProducerFencedException)4 ArrayList (java.util.ArrayList)3 HashSet (java.util.HashSet)3 Set (java.util.Set)3 StreamsException (org.apache.kafka.streams.errors.StreamsException)3 Map (java.util.Map)2 MockConsumer (org.apache.kafka.clients.consumer.MockConsumer)2 StreamsConfig (org.apache.kafka.streams.StreamsConfig)2 InternalStreamsBuilderTest (org.apache.kafka.streams.kstream.internals.InternalStreamsBuilderTest)2 TaskId (org.apache.kafka.streams.processor.TaskId)2 List (java.util.List)1 AtomicReference (java.util.concurrent.atomic.AtomicReference)1 CommitFailedException (org.apache.kafka.clients.consumer.CommitFailedException)1 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)1 InvalidOffsetException (org.apache.kafka.clients.consumer.InvalidOffsetException)1 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)1