Search in sources :

Example 41 with InstanceId

use of org.neo4j.cluster.InstanceId in project neo4j by neo4j.

the class HeartbeatStateTest method shouldLogFirstHeartbeatAfterTimeout.

@Test
public void shouldLogFirstHeartbeatAfterTimeout() throws Throwable {
    // given
    InstanceId instanceId = new InstanceId(1), otherInstance = new InstanceId(2);
    ClusterConfiguration configuration = new ClusterConfiguration("whatever", NullLogProvider.getInstance(), "cluster://1", "cluster://2");
    configuration.getMembers().put(otherInstance, URI.create("cluster://2"));
    AssertableLogProvider internalLog = new AssertableLogProvider(true);
    TimeoutStrategy timeoutStrategy = mock(TimeoutStrategy.class);
    Timeouts timeouts = new Timeouts(timeoutStrategy);
    Config config = mock(Config.class);
    when(config.get(ClusterSettings.max_acceptors)).thenReturn(10);
    MultiPaxosContext context = new MultiPaxosContext(instanceId, iterable(new ElectionRole("coordinator")), configuration, mock(Executor.class), internalLog, mock(ObjectInputStreamFactory.class), mock(ObjectOutputStreamFactory.class), mock(AcceptorInstanceStore.class), timeouts, mock(ElectionCredentialsProvider.class), config);
    StateMachines stateMachines = new StateMachines(internalLog, mock(StateMachines.Monitor.class), mock(MessageSource.class), mock(MessageSender.class), timeouts, mock(DelayedDirectExecutor.class), command -> command.run(), instanceId);
    stateMachines.addStateMachine(new StateMachine(context.getHeartbeatContext(), HeartbeatMessage.class, HeartbeatState.start, internalLog));
    timeouts.tick(0);
    when(timeoutStrategy.timeoutFor(any(Message.class))).thenReturn(5L);
    // when
    stateMachines.process(Message.internal(HeartbeatMessage.join));
    stateMachines.process(Message.internal(HeartbeatMessage.i_am_alive, new HeartbeatMessage.IAmAliveState(otherInstance)).setHeader(Message.CREATED_BY, otherInstance.toString()));
    for (int i = 1; i <= 15; i++) {
        timeouts.tick(i);
    }
    // then
    verify(timeoutStrategy, times(3)).timeoutTriggered(argThat(new MessageArgumentMatcher<>().onMessageType(HeartbeatMessage.timed_out)));
    internalLog.assertExactly(inLog(HeartbeatState.class).debug("Received timed out for server 2"), inLog(HeartbeatContext.class).info("1(me) is now suspecting 2"), inLog(HeartbeatState.class).debug("Received timed out for server 2"), inLog(HeartbeatState.class).debug("Received timed out for server 2"));
    internalLog.clear();
    // when
    stateMachines.process(Message.internal(HeartbeatMessage.i_am_alive, new HeartbeatMessage.IAmAliveState(otherInstance)).setHeader(Message.CREATED_BY, otherInstance.toString()));
    // then
    internalLog.assertExactly(inLog(HeartbeatState.class).debug("Received i_am_alive[2] after missing 3 (15ms)"));
}
Also used : LearnerMessage(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.LearnerMessage) Message(org.neo4j.cluster.com.message.Message) InstanceId(org.neo4j.cluster.InstanceId) Timeouts(org.neo4j.cluster.timeout.Timeouts) Config(org.neo4j.kernel.configuration.Config) MessageSender(org.neo4j.cluster.com.message.MessageSender) StateMachine(org.neo4j.cluster.statemachine.StateMachine) StateMachines(org.neo4j.cluster.StateMachines) TimeoutStrategy(org.neo4j.cluster.timeout.TimeoutStrategy) MessageSource(org.neo4j.cluster.com.message.MessageSource) ClusterConfiguration(org.neo4j.cluster.protocol.cluster.ClusterConfiguration) DelayedDirectExecutor(org.neo4j.cluster.DelayedDirectExecutor) ElectionRole(org.neo4j.cluster.protocol.election.ElectionRole) Executor(java.util.concurrent.Executor) DelayedDirectExecutor(org.neo4j.cluster.DelayedDirectExecutor) ElectionCredentialsProvider(org.neo4j.cluster.protocol.election.ElectionCredentialsProvider) MultiPaxosContext(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.context.MultiPaxosContext) ObjectInputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectInputStreamFactory) ObjectOutputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectOutputStreamFactory) AcceptorInstanceStore(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.AcceptorInstanceStore) AssertableLogProvider(org.neo4j.logging.AssertableLogProvider) Test(org.junit.Test)

Example 42 with InstanceId

use of org.neo4j.cluster.InstanceId in project neo4j by neo4j.

the class HeartbeatStateTest method shouldIgnoreSuspicionsForOurselves.

@Test
public void shouldIgnoreSuspicionsForOurselves() throws Throwable {
    // Given
    InstanceId instanceId = new InstanceId(1);
    HeartbeatState heartbeat = HeartbeatState.heartbeat;
    ClusterConfiguration configuration = new ClusterConfiguration("whatever", NullLogProvider.getInstance(), "cluster://1", "cluster://2");
    configuration.joined(instanceId, URI.create("cluster://1"));
    configuration.joined(new InstanceId(2), URI.create("cluster://2"));
    Config config = mock(Config.class);
    when(config.get(ClusterSettings.max_acceptors)).thenReturn(10);
    MultiPaxosContext context = new MultiPaxosContext(instanceId, iterable(new ElectionRole("coordinator")), configuration, Mockito.mock(Executor.class), NullLogProvider.getInstance(), Mockito.mock(ObjectInputStreamFactory.class), Mockito.mock(ObjectOutputStreamFactory.class), Mockito.mock(AcceptorInstanceStore.class), Mockito.mock(Timeouts.class), mock(ElectionCredentialsProvider.class), config);
    HeartbeatContext heartbeatContext = context.getHeartbeatContext();
    Message received = Message.internal(HeartbeatMessage.suspicions, new HeartbeatMessage.SuspicionsState(asSet(iterable(instanceId))));
    received.setHeader(Message.FROM, "cluster://2").setHeader(Message.INSTANCE_ID, "2");
    // When
    heartbeat.handle(heartbeatContext, received, mock(MessageHolder.class));
    // Then
    assertThat(heartbeatContext.getSuspicionsOf(instanceId).size(), equalTo(0));
}
Also used : LearnerMessage(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.LearnerMessage) Message(org.neo4j.cluster.com.message.Message) InstanceId(org.neo4j.cluster.InstanceId) Config(org.neo4j.kernel.configuration.Config) Timeouts(org.neo4j.cluster.timeout.Timeouts) ClusterConfiguration(org.neo4j.cluster.protocol.cluster.ClusterConfiguration) ElectionRole(org.neo4j.cluster.protocol.election.ElectionRole) MessageHolder(org.neo4j.cluster.com.message.MessageHolder) Executor(java.util.concurrent.Executor) DelayedDirectExecutor(org.neo4j.cluster.DelayedDirectExecutor) ElectionCredentialsProvider(org.neo4j.cluster.protocol.election.ElectionCredentialsProvider) MultiPaxosContext(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.context.MultiPaxosContext) ObjectInputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectInputStreamFactory) ObjectOutputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectOutputStreamFactory) AcceptorInstanceStore(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.AcceptorInstanceStore) Test(org.junit.Test)

Example 43 with InstanceId

use of org.neo4j.cluster.InstanceId in project neo4j by neo4j.

the class ClusterContextTest method testElectionVersionIsResetWhenElectorChangesFromMeToOther.

@Test
public void testElectionVersionIsResetWhenElectorChangesFromMeToOther() throws Exception {
    final String coordinatorRole = "coordinator";
    final InstanceId me = new InstanceId(1);
    final InstanceId winner = new InstanceId(2);
    final InstanceId elector = new InstanceId(2);
    HeartbeatContext heartbeatContext = mock(HeartbeatContext.class);
    when(heartbeatContext.getFailed()).thenReturn(Collections.<InstanceId>emptySet());
    Config config = mock(Config.class);
    when(config.get(ClusterSettings.max_acceptors)).thenReturn(10);
    MultiPaxosContext multiPaxosContext = new MultiPaxosContext(me, Iterables.<ElectionRole, ElectionRole>iterable(new ElectionRole(coordinatorRole)), mock(ClusterConfiguration.class), new Executor() {

        @Override
        public void execute(Runnable command) {
            command.run();
        }
    }, NullLogProvider.getInstance(), mock(ObjectInputStreamFactory.class), mock(ObjectOutputStreamFactory.class), mock(AcceptorInstanceStore.class), mock(Timeouts.class), mock(ElectionCredentialsProvider.class), config);
    ClusterContext context = multiPaxosContext.getClusterContext();
    ElectionContext electionContext = multiPaxosContext.getElectionContext();
    ClusterListener listener = mock(ClusterListener.class);
    context.setLastElectorVersion(5);
    context.setLastElector(me);
    context.addClusterListener(listener);
    long expectedVersion = electionContext.newConfigurationStateChange().getVersion();
    context.elected(coordinatorRole, winner, me, expectedVersion);
    verify(listener, times(1)).elected(coordinatorRole, winner, null);
    context.elected(coordinatorRole, winner, elector, 2);
    verify(listener, times(2)).elected(coordinatorRole, winner, null);
    context.elected(coordinatorRole, winner, elector, 3);
    verify(listener, times(3)).elected(coordinatorRole, winner, null);
    context.elected(coordinatorRole, winner, elector, 2);
    verifyNoMoreInteractions(listener);
}
Also used : ElectionContext(org.neo4j.cluster.protocol.election.ElectionContext) InstanceId(org.neo4j.cluster.InstanceId) Config(org.neo4j.kernel.configuration.Config) Timeouts(org.neo4j.cluster.timeout.Timeouts) ElectionRole(org.neo4j.cluster.protocol.election.ElectionRole) Executor(java.util.concurrent.Executor) ElectionCredentialsProvider(org.neo4j.cluster.protocol.election.ElectionCredentialsProvider) MultiPaxosContext(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.context.MultiPaxosContext) HeartbeatContext(org.neo4j.cluster.protocol.heartbeat.HeartbeatContext) ObjectInputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectInputStreamFactory) ObjectOutputStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectOutputStreamFactory) AcceptorInstanceStore(org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.AcceptorInstanceStore) Test(org.junit.Test)

Example 44 with InstanceId

use of org.neo4j.cluster.InstanceId in project neo4j by neo4j.

the class ClusterMockTest method verifyConfigurations.

public void verifyConfigurations(VerifyInstanceConfiguration[] toCheckAgainst) {
    logger.getLogger().fine("Verify configurations against given");
    List<URI> members;
    Map<String, InstanceId> roles;
    Set<InstanceId> failed;
    List<AssertionError> errors = new LinkedList<AssertionError>();
    List<TestProtocolServer> protocolServers = network.getServers();
    assertEquals("You must provide a configuration for all instances", protocolServers.size(), toCheckAgainst.length);
    for (int j = 0; j < protocolServers.size(); j++) {
        members = toCheckAgainst[j].members;
        roles = toCheckAgainst[j].roles;
        failed = toCheckAgainst[j].failed;
        StateMachines stateMachines = protocolServers.get(j).getServer().getStateMachines();
        State<?, ?> clusterState = stateMachines.getStateMachine(ClusterMessage.class).getState();
        if (!clusterState.equals(ClusterState.entered)) {
            logger.getLogger().warning("Instance " + (j + 1) + " is not in the cluster (" + clusterState + ")");
            continue;
        }
        ClusterContext context = (ClusterContext) stateMachines.getStateMachine(ClusterMessage.class).getContext();
        HeartbeatContext heartbeatContext = (HeartbeatContext) stateMachines.getStateMachine(HeartbeatMessage.class).getContext();
        ClusterConfiguration clusterConfiguration = context.getConfiguration();
        if (!clusterConfiguration.getMemberURIs().isEmpty()) {
            logger.getLogger().fine("   Server " + (j + 1) + ": Cluster:" + clusterConfiguration.getMemberURIs() + ", Roles:" + clusterConfiguration.getRoles() + ", Failed:" + heartbeatContext.getFailed());
            verifyConfigurations(stateMachines, members, roles, failed, errors);
        }
    }
    if (!errors.isEmpty()) {
        for (AssertionError error : errors) {
            logger.getLogger().severe(error.toString());
        }
        throw errors.get(0);
    }
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) StateMachines(org.neo4j.cluster.StateMachines) URI(java.net.URI) LinkedList(java.util.LinkedList) TestProtocolServer(org.neo4j.cluster.TestProtocolServer) HeartbeatContext(org.neo4j.cluster.protocol.heartbeat.HeartbeatContext)

Example 45 with InstanceId

use of org.neo4j.cluster.InstanceId in project neo4j by neo4j.

the class ClusterMockTest method testCluster.

protected void testCluster(int[] serverIds, VerifyInstanceConfiguration[] finalConfig, NetworkMock mock, ClusterTestScript script) throws ExecutionException, InterruptedException, URISyntaxException, TimeoutException {
    this.script = script;
    network = mock;
    servers.clear();
    out.clear();
    in.clear();
    for (int i = 0; i < serverIds.length; i++) {
        final URI uri = new URI("server" + (i + 1));
        members.put(serverIds[i], uri);
        TestProtocolServer server = network.addServer(serverIds[i], uri);
        final Cluster cluster = server.newClient(Cluster.class);
        clusterStateListener(uri, cluster);
        server.newClient(Heartbeat.class).addHeartbeatListener(new HeartbeatListener() {

            @Override
            public void failed(InstanceId server) {
                logger.getLogger().warning(uri + ": Failed:" + server);
            }

            @Override
            public void alive(InstanceId server) {
                logger.getLogger().fine(uri + ": Alive:" + server);
            }
        });
        server.newClient(AtomicBroadcast.class).addAtomicBroadcastListener(new AtomicBroadcastListener() {

            AtomicBroadcastSerializer serializer = new AtomicBroadcastSerializer(new ObjectStreamFactory(), new ObjectStreamFactory());

            @Override
            public void receive(Payload value) {
                try {
                    logger.getLogger().fine(uri + " received: " + serializer.receive(value));
                } catch (IOException e) {
                    e.printStackTrace();
                } catch (ClassNotFoundException e) {
                    e.printStackTrace();
                }
            }
        });
        servers.add(server);
        out.add(cluster);
    }
    // Run test
    for (int i = 0; i < script.rounds(); i++) {
        logger.getLogger().fine("Round " + i + ", time:" + network.getTime());
        script.tick(network.getTime());
        network.tick();
    }
    // Let messages settle
    network.tick(100);
    if (finalConfig == null) {
        verifyConfigurations("final config");
    } else {
        verifyConfigurations(finalConfig);
    }
    logger.getLogger().fine("All nodes leave");
    // All leave
    for (Cluster cluster : new ArrayList<Cluster>(in)) {
        logger.getLogger().fine("Leaving:" + cluster);
        cluster.leave();
        in.remove(cluster);
        network.tick(400);
    }
    if (finalConfig != null) {
        verifyConfigurations(finalConfig);
    } else {
        verifyConfigurations("test over");
    }
}
Also used : AtomicBroadcastSerializer(org.neo4j.cluster.protocol.atomicbroadcast.AtomicBroadcastSerializer) InstanceId(org.neo4j.cluster.InstanceId) AtomicBroadcast(org.neo4j.cluster.protocol.atomicbroadcast.AtomicBroadcast) ArrayList(java.util.ArrayList) IOException(java.io.IOException) URI(java.net.URI) AtomicBroadcastListener(org.neo4j.cluster.protocol.atomicbroadcast.AtomicBroadcastListener) ObjectStreamFactory(org.neo4j.cluster.protocol.atomicbroadcast.ObjectStreamFactory) HeartbeatListener(org.neo4j.cluster.protocol.heartbeat.HeartbeatListener) TestProtocolServer(org.neo4j.cluster.TestProtocolServer) Heartbeat(org.neo4j.cluster.protocol.heartbeat.Heartbeat) Payload(org.neo4j.cluster.protocol.atomicbroadcast.Payload)

Aggregations

InstanceId (org.neo4j.cluster.InstanceId)154 Test (org.junit.Test)129 URI (java.net.URI)47 Config (org.neo4j.kernel.configuration.Config)37 ClusterConfiguration (org.neo4j.cluster.protocol.cluster.ClusterConfiguration)36 Timeouts (org.neo4j.cluster.timeout.Timeouts)32 Executor (java.util.concurrent.Executor)28 ClusterContext (org.neo4j.cluster.protocol.cluster.ClusterContext)28 ObjectInputStreamFactory (org.neo4j.cluster.protocol.atomicbroadcast.ObjectInputStreamFactory)27 ObjectOutputStreamFactory (org.neo4j.cluster.protocol.atomicbroadcast.ObjectOutputStreamFactory)27 HeartbeatContext (org.neo4j.cluster.protocol.heartbeat.HeartbeatContext)27 MultiPaxosContext (org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.context.MultiPaxosContext)24 AcceptorInstanceStore (org.neo4j.cluster.protocol.atomicbroadcast.multipaxos.AcceptorInstanceStore)18 MessageHolder (org.neo4j.cluster.com.message.MessageHolder)16 Message (org.neo4j.cluster.com.message.Message)15 ArrayList (java.util.ArrayList)13 HashMap (java.util.HashMap)13 ClusterMemberAvailability (org.neo4j.cluster.member.ClusterMemberAvailability)13 ClusterMemberEvents (org.neo4j.cluster.member.ClusterMemberEvents)12 ClusterMemberListener (org.neo4j.cluster.member.ClusterMemberListener)12