Search in sources :

Example 1 with ReadableRaftState

use of org.neo4j.causalclustering.core.consensus.state.ReadableRaftState in project neo4j by neo4j.

the class Appending method appendNewEntries.

static void appendNewEntries(ReadableRaftState ctx, Outcome outcome, List<ReplicatedContent> contents) throws IOException {
    long prevLogIndex = ctx.entryLog().appendIndex();
    long prevLogTerm = prevLogIndex == -1 ? -1 : prevLogIndex > ctx.lastLogIndexBeforeWeBecameLeader() ? ctx.term() : ctx.entryLog().readEntryTerm(prevLogIndex);
    RaftLogEntry[] raftLogEntries = contents.stream().map(content -> new RaftLogEntry(ctx.term(), content)).toArray(RaftLogEntry[]::new);
    outcome.addShipCommand(new ShipCommand.NewEntries(prevLogIndex, prevLogTerm, raftLogEntries));
    outcome.addLogCommand(new BatchAppendLogEntries(prevLogIndex + 1, 0, raftLogEntries));
}
Also used : Log(org.neo4j.logging.Log) Outcome(org.neo4j.causalclustering.core.consensus.outcome.Outcome) IOException(java.io.IOException) ReadableRaftState(org.neo4j.causalclustering.core.consensus.state.ReadableRaftState) String.format(java.lang.String.format) ShipCommand(org.neo4j.causalclustering.core.consensus.outcome.ShipCommand) List(java.util.List) RaftLogEntry(org.neo4j.causalclustering.core.consensus.log.RaftLogEntry) AppendLogEntry(org.neo4j.causalclustering.core.consensus.outcome.AppendLogEntry) ReplicatedContent(org.neo4j.causalclustering.core.replication.ReplicatedContent) BatchAppendLogEntries(org.neo4j.causalclustering.core.consensus.outcome.BatchAppendLogEntries) RaftMessages(org.neo4j.causalclustering.core.consensus.RaftMessages) TruncateLogCommand(org.neo4j.causalclustering.core.consensus.outcome.TruncateLogCommand) ShipCommand(org.neo4j.causalclustering.core.consensus.outcome.ShipCommand) BatchAppendLogEntries(org.neo4j.causalclustering.core.consensus.outcome.BatchAppendLogEntries) RaftLogEntry(org.neo4j.causalclustering.core.consensus.log.RaftLogEntry)

Example 2 with ReadableRaftState

use of org.neo4j.causalclustering.core.consensus.state.ReadableRaftState in project neo4j by neo4j.

the class LeaderTest method leaderShouldNotRespondToSuccessResponseThatIndicatesUpToDateFollower.

@Test
public void leaderShouldNotRespondToSuccessResponseThatIndicatesUpToDateFollower() throws Exception {
    // given
    /*
         * A leader who
         * - has an append index of 100
         * - knows about instance 2
         * - assumes that instance 2 is at an index less than 100 -say 84
         */
    Leader leader = new Leader();
    MemberId instance2 = member(2);
    FollowerState instance2State = createArtificialFollowerState(84);
    ReadableRaftState state = mock(ReadableRaftState.class);
    FollowerStates<MemberId> followerState = new FollowerStates<>();
    followerState = new FollowerStates<>(followerState, instance2, instance2State);
    ReadableRaftLog logMock = mock(ReadableRaftLog.class);
    when(logMock.appendIndex()).thenReturn(100L);
    when(state.commitIndex()).thenReturn(-1L);
    when(state.entryLog()).thenReturn(logMock);
    when(state.followerStates()).thenReturn(followerState);
    // both leader and follower are in the same term
    when(state.term()).thenReturn(4L);
    // when
    // that leader is asked to handle a response from that follower that says that the follower is up to date
    RaftMessages.AppendEntries.Response response = appendEntriesResponse().success().matchIndex(100).term(4).from(instance2).build();
    Outcome outcome = leader.handle(response, state, mock(Log.class));
    // then
    // The leader should not be trying to send any messages to that instance
    assertTrue(outcome.getOutgoingMessages().isEmpty());
    // And the follower state should be updated
    FollowerStates<MemberId> updatedFollowerStates = outcome.getFollowerStates();
    assertEquals(100, updatedFollowerStates.get(instance2).getMatchIndex());
}
Also used : MemberId(org.neo4j.causalclustering.identity.MemberId) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) Log(org.neo4j.logging.Log) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) RaftLog(org.neo4j.causalclustering.core.consensus.log.RaftLog) Outcome(org.neo4j.causalclustering.core.consensus.outcome.Outcome) FollowerStates(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerStates) AppendEntries(org.neo4j.causalclustering.core.consensus.RaftMessages.AppendEntries) FollowerState(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerState) ReadableRaftState(org.neo4j.causalclustering.core.consensus.state.ReadableRaftState) Test(org.junit.Test)

Example 3 with ReadableRaftState

use of org.neo4j.causalclustering.core.consensus.state.ReadableRaftState in project neo4j by neo4j.

the class LeaderTest method leaderShouldRespondToSuccessResponseThatIndicatesLaggingFollowerWithJustWhatItsMissing.

@Test
public void leaderShouldRespondToSuccessResponseThatIndicatesLaggingFollowerWithJustWhatItsMissing() throws Exception {
    // given
    /*
         * A leader who
         * - has an append index of 100
         * - knows about instance 2
         * - assumes that instance 2 is at an index less than 100 -say 50
         */
    Leader leader = new Leader();
    MemberId instance2 = member(2);
    FollowerState instance2State = createArtificialFollowerState(50);
    ReadableRaftState state = mock(ReadableRaftState.class);
    FollowerStates<MemberId> followerState = new FollowerStates<>();
    followerState = new FollowerStates<>(followerState, instance2, instance2State);
    ReadableRaftLog logMock = mock(ReadableRaftLog.class);
    when(logMock.appendIndex()).thenReturn(100L);
    // with commit requests in this test
    when(state.commitIndex()).thenReturn(-1L);
    when(state.entryLog()).thenReturn(logMock);
    when(state.followerStates()).thenReturn(followerState);
    // both leader and follower are in the same term
    when(state.term()).thenReturn(231L);
    // when that leader is asked to handle a response from that follower that says that the follower is still
    // missing things
    RaftMessages.AppendEntries.Response response = appendEntriesResponse().success().matchIndex(89).term(231).from(instance2).build();
    Outcome outcome = leader.handle(response, state, mock(Log.class));
    // then
    int matchCount = 0;
    for (ShipCommand shipCommand : outcome.getShipCommands()) {
        if (shipCommand instanceof ShipCommand.Match) {
            matchCount++;
        }
    }
    assertThat(matchCount, greaterThan(0));
}
Also used : ShipCommand(org.neo4j.causalclustering.core.consensus.outcome.ShipCommand) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) Log(org.neo4j.logging.Log) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) RaftLog(org.neo4j.causalclustering.core.consensus.log.RaftLog) AppendEntries(org.neo4j.causalclustering.core.consensus.RaftMessages.AppendEntries) ReadableRaftState(org.neo4j.causalclustering.core.consensus.state.ReadableRaftState) MemberId(org.neo4j.causalclustering.identity.MemberId) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) Outcome(org.neo4j.causalclustering.core.consensus.outcome.Outcome) FollowerStates(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerStates) FollowerState(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerState) Test(org.junit.Test)

Example 4 with ReadableRaftState

use of org.neo4j.causalclustering.core.consensus.state.ReadableRaftState in project neo4j by neo4j.

the class LeaderTest method leaderShouldSpawnMismatchCommandOnFailure.

// TODO: rethink this test, it does too much
@Test
public void leaderShouldSpawnMismatchCommandOnFailure() throws Exception {
    // given
    /*
         * A leader who
         * - has an append index of 100
         * - knows about instance 2
         * - assumes that instance 2 is fully caught up
         */
    Leader leader = new Leader();
    MemberId instance2 = member(2);
    FollowerState instance2State = createArtificialFollowerState(100);
    ReadableRaftState state = mock(ReadableRaftState.class);
    FollowerStates<MemberId> followerState = new FollowerStates<>();
    followerState = new FollowerStates<>(followerState, instance2, instance2State);
    RaftLog log = new InMemoryRaftLog();
    for (int i = 0; i <= 100; i++) {
        log.append(new RaftLogEntry(0, valueOf(i)));
    }
    when(state.commitIndex()).thenReturn(-1L);
    when(state.entryLog()).thenReturn(log);
    when(state.followerStates()).thenReturn(followerState);
    // both leader and follower are in the same term
    when(state.term()).thenReturn(4L);
    // when
    // that leader is asked to handle a response from that follower that says that the follower is still missing
    // things
    RaftMessages.AppendEntries.Response response = appendEntriesResponse().failure().appendIndex(0).matchIndex(-1).term(4).from(instance2).build();
    Outcome outcome = leader.handle(response, state, mock(Log.class));
    // then
    int mismatchCount = 0;
    for (ShipCommand shipCommand : outcome.getShipCommands()) {
        if (shipCommand instanceof ShipCommand.Mismatch) {
            mismatchCount++;
        }
    }
    assertThat(mismatchCount, greaterThan(0));
}
Also used : ShipCommand(org.neo4j.causalclustering.core.consensus.outcome.ShipCommand) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) Log(org.neo4j.logging.Log) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) RaftLog(org.neo4j.causalclustering.core.consensus.log.RaftLog) AppendEntries(org.neo4j.causalclustering.core.consensus.RaftMessages.AppendEntries) ReadableRaftState(org.neo4j.causalclustering.core.consensus.state.ReadableRaftState) RaftLogEntry(org.neo4j.causalclustering.core.consensus.log.RaftLogEntry) MemberId(org.neo4j.causalclustering.identity.MemberId) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) Outcome(org.neo4j.causalclustering.core.consensus.outcome.Outcome) FollowerStates(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerStates) FollowerState(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerState) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) RaftLog(org.neo4j.causalclustering.core.consensus.log.RaftLog) Test(org.junit.Test)

Example 5 with ReadableRaftState

use of org.neo4j.causalclustering.core.consensus.state.ReadableRaftState in project neo4j by neo4j.

the class LeaderTest method leaderShouldNotRespondToSuccessResponseFromFollowerThatWillSoonUpToDateViaInFlightMessages.

@Test
public void leaderShouldNotRespondToSuccessResponseFromFollowerThatWillSoonUpToDateViaInFlightMessages() throws Exception {
    // given
    /*
         * A leader who
         * - has an append index of 100
         * - knows about instance 2
         * - assumes that instance 2 is at an index less than 100 -say 84 but it has already been sent up to 100
         */
    Leader leader = new Leader();
    MemberId instance2 = member(2);
    FollowerState instance2State = createArtificialFollowerState(84);
    ReadableRaftState state = mock(ReadableRaftState.class);
    FollowerStates<MemberId> followerState = new FollowerStates<>();
    followerState = new FollowerStates<>(followerState, instance2, instance2State);
    ReadableRaftLog logMock = mock(ReadableRaftLog.class);
    when(logMock.appendIndex()).thenReturn(100L);
    when(state.commitIndex()).thenReturn(-1L);
    when(state.entryLog()).thenReturn(logMock);
    when(state.followerStates()).thenReturn(followerState);
    // both leader and follower are in the same term
    when(state.term()).thenReturn(4L);
    // when
    // that leader is asked to handle a response from that follower that says that the follower is up to date
    RaftMessages.AppendEntries.Response response = appendEntriesResponse().success().matchIndex(90).term(4).from(instance2).build();
    Outcome outcome = leader.handle(response, state, mock(Log.class));
    // then
    // The leader should not be trying to send any messages to that instance
    assertTrue(outcome.getOutgoingMessages().isEmpty());
    // And the follower state should be updated
    FollowerStates<MemberId> leadersViewOfFollowerStates = outcome.getFollowerStates();
    assertEquals(90, leadersViewOfFollowerStates.get(instance2).getMatchIndex());
}
Also used : MemberId(org.neo4j.causalclustering.identity.MemberId) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) InMemoryRaftLog(org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog) Log(org.neo4j.logging.Log) ReadableRaftLog(org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog) RaftLog(org.neo4j.causalclustering.core.consensus.log.RaftLog) Outcome(org.neo4j.causalclustering.core.consensus.outcome.Outcome) FollowerStates(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerStates) AppendEntries(org.neo4j.causalclustering.core.consensus.RaftMessages.AppendEntries) FollowerState(org.neo4j.causalclustering.core.consensus.roles.follower.FollowerState) ReadableRaftState(org.neo4j.causalclustering.core.consensus.state.ReadableRaftState) Test(org.junit.Test)

Aggregations

Outcome (org.neo4j.causalclustering.core.consensus.outcome.Outcome)10 ReadableRaftState (org.neo4j.causalclustering.core.consensus.state.ReadableRaftState)10 Test (org.junit.Test)9 ReadableRaftLog (org.neo4j.causalclustering.core.consensus.log.ReadableRaftLog)9 RaftLogEntry (org.neo4j.causalclustering.core.consensus.log.RaftLogEntry)6 Log (org.neo4j.logging.Log)6 AppendEntries (org.neo4j.causalclustering.core.consensus.RaftMessages.AppendEntries)5 InMemoryRaftLog (org.neo4j.causalclustering.core.consensus.log.InMemoryRaftLog)5 RaftLog (org.neo4j.causalclustering.core.consensus.log.RaftLog)5 FollowerState (org.neo4j.causalclustering.core.consensus.roles.follower.FollowerState)5 FollowerStates (org.neo4j.causalclustering.core.consensus.roles.follower.FollowerStates)5 MemberId (org.neo4j.causalclustering.identity.MemberId)5 ShipCommand (org.neo4j.causalclustering.core.consensus.outcome.ShipCommand)3 IOException (java.io.IOException)1 String.format (java.lang.String.format)1 List (java.util.List)1 RaftMessages (org.neo4j.causalclustering.core.consensus.RaftMessages)1 AppendLogEntry (org.neo4j.causalclustering.core.consensus.outcome.AppendLogEntry)1 BatchAppendLogEntries (org.neo4j.causalclustering.core.consensus.outcome.BatchAppendLogEntries)1 TruncateLogCommand (org.neo4j.causalclustering.core.consensus.outcome.TruncateLogCommand)1