Search in sources :

Example 1 with SyncGroupRequest

use of org.apache.kafka.common.requests.SyncGroupRequest in project kafka by apache.

the class WorkerCoordinatorTest method testJoinLeaderCannotAssign.

@Test
public void testJoinLeaderCannotAssign() {
    // If the selected leader can't get up to the maximum offset, it will fail to assign and we should immediately
    // need to retry the join.
    // When the first round fails, we'll take an updated config snapshot
    EasyMock.expect(configStorage.snapshot()).andReturn(configState1);
    EasyMock.expect(configStorage.snapshot()).andReturn(configState2);
    PowerMock.replayAll();
    final String memberId = "member";
    client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
    coordinator.ensureCoordinatorReady();
    // config mismatch results in assignment error
    client.prepareResponse(joinGroupFollowerResponse(1, memberId, "leader", Errors.NONE));
    MockClient.RequestMatcher matcher = new MockClient.RequestMatcher() {

        @Override
        public boolean matches(AbstractRequest body) {
            SyncGroupRequest sync = (SyncGroupRequest) body;
            return sync.memberId().equals(memberId) && sync.generationId() == 1 && sync.groupAssignment().isEmpty();
        }
    };
    client.prepareResponse(matcher, syncGroupResponse(ConnectProtocol.Assignment.CONFIG_MISMATCH, "leader", 10L, Collections.<String>emptyList(), Collections.<ConnectorTaskId>emptyList(), Errors.NONE));
    client.prepareResponse(joinGroupFollowerResponse(1, memberId, "leader", Errors.NONE));
    client.prepareResponse(matcher, syncGroupResponse(ConnectProtocol.Assignment.NO_ERROR, "leader", 1L, Collections.<String>emptyList(), Collections.singletonList(taskId1x0), Errors.NONE));
    coordinator.ensureActiveGroup();
    PowerMock.verifyAll();
}
Also used : ConnectorTaskId(org.apache.kafka.connect.util.ConnectorTaskId) SyncGroupRequest(org.apache.kafka.common.requests.SyncGroupRequest) AbstractRequest(org.apache.kafka.common.requests.AbstractRequest) MockClient(org.apache.kafka.clients.MockClient) Test(org.junit.Test)

Example 2 with SyncGroupRequest

use of org.apache.kafka.common.requests.SyncGroupRequest in project kafka by apache.

the class AbstractCoordinatorTest method testWakeupAfterSyncGroupSent.

@Test
public void testWakeupAfterSyncGroupSent() throws Exception {
    mockClient.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
    mockClient.prepareResponse(joinGroupFollowerResponse(1, "memberId", "leaderId", Errors.NONE));
    mockClient.prepareResponse(new MockClient.RequestMatcher() {

        private int invocations = 0;

        @Override
        public boolean matches(AbstractRequest body) {
            invocations++;
            boolean isSyncGroupRequest = body instanceof SyncGroupRequest;
            if (isSyncGroupRequest && invocations == 1)
                // simulate wakeup after the request sent
                throw new WakeupException();
            return isSyncGroupRequest;
        }
    }, syncGroupResponse(Errors.NONE));
    AtomicBoolean heartbeatReceived = prepareFirstHeartbeat();
    try {
        coordinator.ensureActiveGroup();
        fail("Should have woken up from ensureActiveGroup()");
    } catch (WakeupException e) {
    }
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(0, coordinator.onJoinCompleteInvokes);
    assertFalse(heartbeatReceived.get());
    coordinator.ensureActiveGroup();
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(1, coordinator.onJoinCompleteInvokes);
    awaitFirstHeartbeat(heartbeatReceived);
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SyncGroupRequest(org.apache.kafka.common.requests.SyncGroupRequest) AbstractRequest(org.apache.kafka.common.requests.AbstractRequest) WakeupException(org.apache.kafka.common.errors.WakeupException) MockClient(org.apache.kafka.clients.MockClient) Test(org.junit.Test)

Example 3 with SyncGroupRequest

use of org.apache.kafka.common.requests.SyncGroupRequest in project apache-kafka-on-k8s by banzaicloud.

the class WorkerCoordinatorTest method testJoinLeaderCannotAssign.

@Test
public void testJoinLeaderCannotAssign() {
    // If the selected leader can't get up to the maximum offset, it will fail to assign and we should immediately
    // need to retry the join.
    // When the first round fails, we'll take an updated config snapshot
    EasyMock.expect(configStorage.snapshot()).andReturn(configState1);
    EasyMock.expect(configStorage.snapshot()).andReturn(configState2);
    PowerMock.replayAll();
    final String memberId = "member";
    client.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
    coordinator.ensureCoordinatorReady();
    // config mismatch results in assignment error
    client.prepareResponse(joinGroupFollowerResponse(1, memberId, "leader", Errors.NONE));
    MockClient.RequestMatcher matcher = new MockClient.RequestMatcher() {

        @Override
        public boolean matches(AbstractRequest body) {
            SyncGroupRequest sync = (SyncGroupRequest) body;
            return sync.memberId().equals(memberId) && sync.generationId() == 1 && sync.groupAssignment().isEmpty();
        }
    };
    client.prepareResponse(matcher, syncGroupResponse(ConnectProtocol.Assignment.CONFIG_MISMATCH, "leader", 10L, Collections.<String>emptyList(), Collections.<ConnectorTaskId>emptyList(), Errors.NONE));
    client.prepareResponse(joinGroupFollowerResponse(1, memberId, "leader", Errors.NONE));
    client.prepareResponse(matcher, syncGroupResponse(ConnectProtocol.Assignment.NO_ERROR, "leader", 1L, Collections.<String>emptyList(), Collections.singletonList(taskId1x0), Errors.NONE));
    coordinator.ensureActiveGroup();
    PowerMock.verifyAll();
}
Also used : ConnectorTaskId(org.apache.kafka.connect.util.ConnectorTaskId) SyncGroupRequest(org.apache.kafka.common.requests.SyncGroupRequest) AbstractRequest(org.apache.kafka.common.requests.AbstractRequest) MockClient(org.apache.kafka.clients.MockClient) Test(org.junit.Test)

Example 4 with SyncGroupRequest

use of org.apache.kafka.common.requests.SyncGroupRequest in project apache-kafka-on-k8s by banzaicloud.

the class AbstractCoordinatorTest method testWakeupAfterSyncGroupReceived.

@Test
public void testWakeupAfterSyncGroupReceived() throws Exception {
    setupCoordinator(RETRY_BACKOFF_MS);
    mockClient.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
    mockClient.prepareResponse(joinGroupFollowerResponse(1, "memberId", "leaderId", Errors.NONE));
    mockClient.prepareResponse(new MockClient.RequestMatcher() {

        @Override
        public boolean matches(AbstractRequest body) {
            boolean isSyncGroupRequest = body instanceof SyncGroupRequest;
            if (isSyncGroupRequest)
                // wakeup after the request returns
                consumerClient.wakeup();
            return isSyncGroupRequest;
        }
    }, syncGroupResponse(Errors.NONE));
    AtomicBoolean heartbeatReceived = prepareFirstHeartbeat();
    try {
        coordinator.ensureActiveGroup();
        fail("Should have woken up from ensureActiveGroup()");
    } catch (WakeupException e) {
    }
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(0, coordinator.onJoinCompleteInvokes);
    assertFalse(heartbeatReceived.get());
    coordinator.ensureActiveGroup();
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(1, coordinator.onJoinCompleteInvokes);
    awaitFirstHeartbeat(heartbeatReceived);
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SyncGroupRequest(org.apache.kafka.common.requests.SyncGroupRequest) AbstractRequest(org.apache.kafka.common.requests.AbstractRequest) WakeupException(org.apache.kafka.common.errors.WakeupException) MockClient(org.apache.kafka.clients.MockClient) Test(org.junit.Test)

Example 5 with SyncGroupRequest

use of org.apache.kafka.common.requests.SyncGroupRequest in project apache-kafka-on-k8s by banzaicloud.

the class AbstractCoordinatorTest method testWakeupAfterSyncGroupSentExternalCompletion.

@Test
public void testWakeupAfterSyncGroupSentExternalCompletion() throws Exception {
    setupCoordinator(RETRY_BACKOFF_MS);
    mockClient.prepareResponse(groupCoordinatorResponse(node, Errors.NONE));
    mockClient.prepareResponse(joinGroupFollowerResponse(1, "memberId", "leaderId", Errors.NONE));
    mockClient.prepareResponse(new MockClient.RequestMatcher() {

        private int invocations = 0;

        @Override
        public boolean matches(AbstractRequest body) {
            invocations++;
            boolean isSyncGroupRequest = body instanceof SyncGroupRequest;
            if (isSyncGroupRequest && invocations == 1)
                // simulate wakeup after the request sent
                throw new WakeupException();
            return isSyncGroupRequest;
        }
    }, syncGroupResponse(Errors.NONE));
    AtomicBoolean heartbeatReceived = prepareFirstHeartbeat();
    try {
        coordinator.ensureActiveGroup();
        fail("Should have woken up from ensureActiveGroup()");
    } catch (WakeupException e) {
    }
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(0, coordinator.onJoinCompleteInvokes);
    assertFalse(heartbeatReceived.get());
    // the join group completes in this poll()
    consumerClient.poll(0);
    coordinator.ensureActiveGroup();
    assertEquals(1, coordinator.onJoinPrepareInvokes);
    assertEquals(1, coordinator.onJoinCompleteInvokes);
    awaitFirstHeartbeat(heartbeatReceived);
}
Also used : AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SyncGroupRequest(org.apache.kafka.common.requests.SyncGroupRequest) AbstractRequest(org.apache.kafka.common.requests.AbstractRequest) WakeupException(org.apache.kafka.common.errors.WakeupException) MockClient(org.apache.kafka.clients.MockClient) Test(org.junit.Test)

Aggregations

SyncGroupRequest (org.apache.kafka.common.requests.SyncGroupRequest)25 Test (org.junit.jupiter.api.Test)13 Test (org.junit.Test)10 MockClient (org.apache.kafka.clients.MockClient)9 AbstractRequest (org.apache.kafka.common.requests.AbstractRequest)9 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)8 WakeupException (org.apache.kafka.common.errors.WakeupException)8 TopicPartition (org.apache.kafka.common.TopicPartition)7 ArrayList (java.util.ArrayList)5 Collections.singletonList (java.util.Collections.singletonList)5 List (java.util.List)5 Collections.emptyList (java.util.Collections.emptyList)4 HashMap (java.util.HashMap)4 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)2 JoinGroupRequest (org.apache.kafka.common.requests.JoinGroupRequest)2 ConnectorTaskId (org.apache.kafka.connect.util.ConnectorTaskId)2 Collections.singletonMap (java.util.Collections.singletonMap)1 HashSet (java.util.HashSet)1 Map (java.util.Map)1 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)1