Search in sources :

Example 1 with Election

use of org.neo4j.cluster.protocol.election.Election in project neo4j by neo4j.

the class HighAvailabilityModeSwitcherTest method shouldUseProperServerIdWhenDemotingFromMasterOnException.

@Test
public void shouldUseProperServerIdWhenDemotingFromMasterOnException() throws Throwable {
    /*
         * This a test that acts as a driver to prove a bug which had an instance send out a demote message
         * with instance id -1, since it used HAMS#getServerId(URI) with a URI coming from the NetworkReceiver binding
         * which did not contain the serverId URI argument. This has been fixed by explicitly adding the instanceid
         * as a constructor argument of the HAMS.
         */
    // Given
    SwitchToSlaveCopyThenBranch sts = mock(SwitchToSlaveCopyThenBranch.class);
    SwitchToMaster stm = mock(SwitchToMaster.class);
    // this is necessary to trigger a revert which uses the serverId from the HAMS#me field
    when(stm.switchToMaster(any(LifeSupport.class), any(URI.class))).thenThrow(new RuntimeException());
    Election election = mock(Election.class);
    ClusterMemberAvailability cma = mock(ClusterMemberAvailability.class);
    InstanceId instanceId = new InstanceId(14);
    HighAvailabilityModeSwitcher theSwitcher = new HighAvailabilityModeSwitcher(sts, stm, election, cma, mock(ClusterClient.class), storeSupplierMock(), instanceId, new ComponentSwitcherContainer(), neoStoreDataSourceSupplierMock(), NullLogService.getInstance());
    theSwitcher.init();
    theSwitcher.start();
    /*
         * This is the trick, kind of. NetworkReceiver creates this and passes it on to NetworkReceiver#getURI() and
         * that
         * is what HAMS uses as the HAMS#me field value. But we should not be using this to extract the instanceId.
         * Note the lack of a serverId argument
         */
    URI listeningAt = URI.create("ha://0.0.0.0:5001?name=someName");
    theSwitcher.listeningAt(listeningAt);
    // When
    try {
        // the instance fails to switch to master
        theSwitcher.masterIsElected(new HighAvailabilityMemberChangeEvent(HighAvailabilityMemberState.PENDING, HighAvailabilityMemberState.TO_MASTER, instanceId, listeningAt));
    } finally {
        theSwitcher.stop();
        theSwitcher.shutdown();
    }
    // Then
    // The demotion message must have used the proper instance id
    verify(election).demote(instanceId);
}
Also used : ClusterClient(org.neo4j.cluster.client.ClusterClient) SwitchToSlaveCopyThenBranch(org.neo4j.kernel.ha.cluster.SwitchToSlaveCopyThenBranch) InstanceId(org.neo4j.cluster.InstanceId) HighAvailabilityMemberChangeEvent(org.neo4j.kernel.ha.cluster.HighAvailabilityMemberChangeEvent) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) SwitchToMaster(org.neo4j.kernel.ha.cluster.SwitchToMaster) URI(java.net.URI) Election(org.neo4j.cluster.protocol.election.Election) Test(org.junit.Test)

Example 2 with Election

use of org.neo4j.cluster.protocol.election.Election in project neo4j by neo4j.

the class HighAvailabilityMemberStateMachineTest method whenHAModeSwitcherSwitchesToSlaveTheOtherModeSwitcherDoNotGetTheOldMasterClient.

@Test
public void whenHAModeSwitcherSwitchesToSlaveTheOtherModeSwitcherDoNotGetTheOldMasterClient() throws Throwable {
    InstanceId me = new InstanceId(1);
    StoreId storeId = newStoreIdForCurrentVersion();
    HighAvailabilityMemberContext context = mock(HighAvailabilityMemberContext.class);
    when(context.getMyId()).thenReturn(me);
    AvailabilityGuard guard = mock(AvailabilityGuard.class);
    ObservedClusterMembers members = mock(ObservedClusterMembers.class);
    ClusterMember masterMember = mock(ClusterMember.class);
    when(masterMember.getHARole()).thenReturn("master");
    when(masterMember.hasRole("master")).thenReturn(true);
    when(masterMember.getInstanceId()).thenReturn(new InstanceId(2));
    when(masterMember.getStoreId()).thenReturn(storeId);
    ClusterMember self = new ClusterMember(me);
    when(members.getMembers()).thenReturn(Arrays.asList(self, masterMember));
    when(members.getCurrentMember()).thenReturn(self);
    DependencyResolver dependencyResolver = mock(DependencyResolver.class);
    FileSystemAbstraction fs = mock(FileSystemAbstraction.class);
    when(fs.fileExists(any(File.class))).thenReturn(true);
    when(dependencyResolver.resolveDependency(FileSystemAbstraction.class)).thenReturn(fs);
    when(dependencyResolver.resolveDependency(Monitors.class)).thenReturn(new Monitors());
    NeoStoreDataSource dataSource = mock(NeoStoreDataSource.class);
    when(dataSource.getDependencyResolver()).thenReturn(dependencyResolver);
    when(dataSource.getStoreId()).thenReturn(storeId);
    when(dependencyResolver.resolveDependency(NeoStoreDataSource.class)).thenReturn(dataSource);
    when(dependencyResolver.resolveDependency(TransactionIdStore.class)).thenReturn(new DeadSimpleTransactionIdStore());
    when(dependencyResolver.resolveDependency(ObservedClusterMembers.class)).thenReturn(members);
    UpdatePuller updatePuller = mock(UpdatePuller.class);
    when(updatePuller.tryPullUpdates()).thenReturn(true);
    when(dependencyResolver.resolveDependency(UpdatePuller.class)).thenReturn(updatePuller);
    ClusterMemberAvailability clusterMemberAvailability = mock(ClusterMemberAvailability.class);
    final TriggerableClusterMemberEvents events = new TriggerableClusterMemberEvents();
    Election election = mock(Election.class);
    HighAvailabilityMemberStateMachine stateMachine = new HighAvailabilityMemberStateMachine(context, guard, members, events, election, NullLogProvider.getInstance());
    ClusterMembers clusterMembers = new ClusterMembers(members, stateMachine);
    when(dependencyResolver.resolveDependency(ClusterMembers.class)).thenReturn(clusterMembers);
    stateMachine.init();
    stateMachine.start();
    final DelegateInvocationHandler<Master> handler = new DelegateInvocationHandler<>(Master.class);
    MasterClientResolver masterClientResolver = mock(MasterClientResolver.class);
    MasterClient masterClient = mock(MasterClient.class);
    when(masterClient.getProtocolVersion()).thenReturn(MasterClient214.PROTOCOL_VERSION);
    when(masterClient.handshake(anyLong(), any(StoreId.class))).thenReturn(new Response<HandshakeResult>(new HandshakeResult(0, 42), storeId, mock(ResourceReleaser.class)) {

        @Override
        public void accept(Handler handler) throws IOException {
        }

        @Override
        public boolean hasTransactionsToBeApplied() {
            return false;
        }
    });
    when(masterClient.toString()).thenReturn("TheExpectedMasterClient!");
    when(masterClientResolver.instantiate(anyString(), anyInt(), anyString(), any(Monitors.class), any(StoreId.class), any(LifeSupport.class))).thenReturn(masterClient);
    final CountDownLatch latch = new CountDownLatch(2);
    final AtomicBoolean switchedSuccessfully = new AtomicBoolean();
    SwitchToSlave.Monitor monitor = new SwitchToSlave.Monitor() {

        @Override
        public void switchToSlaveCompleted(boolean wasSuccessful) {
            switchedSuccessfully.set(wasSuccessful);
            latch.countDown();
        }
    };
    Config config = Config.embeddedDefaults(Collections.singletonMap(ClusterSettings.server_id.name(), me.toString()));
    TransactionStats transactionCounters = mock(TransactionStats.class);
    when(transactionCounters.getNumberOfActiveTransactions()).thenReturn(0L);
    PageCache pageCacheMock = mock(PageCache.class);
    PagedFile pagedFileMock = mock(PagedFile.class);
    when(pagedFileMock.getLastPageId()).thenReturn(1L);
    when(pageCacheMock.map(any(File.class), anyInt())).thenReturn(pagedFileMock);
    TransactionIdStore transactionIdStoreMock = mock(TransactionIdStore.class);
    when(transactionIdStoreMock.getLastCommittedTransaction()).thenReturn(new TransactionId(0, 0, 0));
    SwitchToSlaveCopyThenBranch switchToSlave = new SwitchToSlaveCopyThenBranch(new File(""), NullLogService.getInstance(), mock(FileSystemAbstraction.class), config, dependencyResolver, mock(HaIdGeneratorFactory.class), handler, mock(ClusterMemberAvailability.class), mock(RequestContextFactory.class), mock(PullerFactory.class, RETURNS_MOCKS), Iterables.empty(), masterClientResolver, monitor, new StoreCopyClient.Monitor.Adapter(), Suppliers.singleton(dataSource), Suppliers.singleton(transactionIdStoreMock), slave -> {
        SlaveServer mock = mock(SlaveServer.class);
        when(mock.getSocketAddress()).thenReturn(new InetSocketAddress("localhost", 123));
        return mock;
    }, updatePuller, pageCacheMock, mock(Monitors.class), transactionCounters);
    ComponentSwitcherContainer switcherContainer = new ComponentSwitcherContainer();
    HighAvailabilityModeSwitcher haModeSwitcher = new HighAvailabilityModeSwitcher(switchToSlave, mock(SwitchToMaster.class), election, clusterMemberAvailability, mock(ClusterClient.class), storeSupplierMock(), me, switcherContainer, neoStoreDataSourceSupplierMock(), NullLogService.getInstance());
    haModeSwitcher.init();
    haModeSwitcher.start();
    haModeSwitcher.listeningAt(URI.create("http://localhost:12345"));
    stateMachine.addHighAvailabilityMemberListener(haModeSwitcher);
    final AtomicReference<Master> ref = new AtomicReference<>(null);
    //noinspection unchecked
    AbstractComponentSwitcher<Object> otherModeSwitcher = new AbstractComponentSwitcher<Object>(mock(DelegateInvocationHandler.class)) {

        @Override
        protected Object getSlaveImpl() {
            Master master = handler.cement();
            ref.set(master);
            latch.countDown();
            return null;
        }

        @Override
        protected Object getMasterImpl() {
            return null;
        }
    };
    switcherContainer.add(otherModeSwitcher);
    // When
    events.switchToSlave(me);
    // Then
    latch.await();
    assertTrue("mode switch failed", switchedSuccessfully.get());
    Master actual = ref.get();
    // let's test the toString()s since there are too many wrappers of proxies
    assertEquals(masterClient.toString(), actual.toString());
    stateMachine.stop();
    stateMachine.shutdown();
    haModeSwitcher.stop();
    haModeSwitcher.shutdown();
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) InetSocketAddress(java.net.InetSocketAddress) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) AvailabilityGuard(org.neo4j.kernel.AvailabilityGuard) ObservedClusterMembers(org.neo4j.kernel.ha.cluster.member.ObservedClusterMembers) RequestContextFactory(org.neo4j.kernel.ha.com.RequestContextFactory) PageCache(org.neo4j.io.pagecache.PageCache) HighAvailabilityModeSwitcher(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) InstanceId(org.neo4j.cluster.InstanceId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) DependencyResolver(org.neo4j.graphdb.DependencyResolver) TransactionId(org.neo4j.kernel.impl.store.TransactionId) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Monitors(org.neo4j.kernel.monitoring.Monitors) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File) HaIdGeneratorFactory(org.neo4j.kernel.ha.id.HaIdGeneratorFactory) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) Config(org.neo4j.kernel.configuration.Config) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) ClusterMembers(org.neo4j.kernel.ha.cluster.member.ClusterMembers) ObservedClusterMembers(org.neo4j.kernel.ha.cluster.member.ObservedClusterMembers) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) ClusterMember(org.neo4j.kernel.ha.cluster.member.ClusterMember) ClusterClient(org.neo4j.cluster.client.ClusterClient) StoreId(org.neo4j.kernel.impl.store.StoreId) AbstractComponentSwitcher(org.neo4j.kernel.ha.cluster.modeswitch.AbstractComponentSwitcher) UpdatePuller(org.neo4j.kernel.ha.UpdatePuller) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) PullerFactory(org.neo4j.kernel.ha.PullerFactory) PagedFile(org.neo4j.io.pagecache.PagedFile) ComponentSwitcherContainer(org.neo4j.kernel.ha.cluster.modeswitch.ComponentSwitcherContainer) AtomicReference(java.util.concurrent.atomic.AtomicReference) IOException(java.io.IOException) CountDownLatch(java.util.concurrent.CountDownLatch) Election(org.neo4j.cluster.protocol.election.Election) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) Master(org.neo4j.kernel.ha.com.master.Master) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) Test(org.junit.Test)

Example 3 with Election

use of org.neo4j.cluster.protocol.election.Election in project neo4j by neo4j.

the class HAStateMachineIllegalTransitionsTest method setup.

@Before
public void setup() throws Throwable {
    HighAvailabilityMemberContext context = new SimpleHighAvailabilityMemberContext(me, false);
    ClusterMemberEvents events = mock(ClusterMemberEvents.class);
    HighAvailabilityMemberStateMachineTest.ClusterMemberListenerContainer memberListenerContainer = mockAddClusterMemberListener(events);
    election = mock(Election.class);
    stateMachine = buildMockedStateMachine(context, events, election);
    stateMachine.init();
    memberListener = memberListenerContainer.get();
    HighAvailabilityMemberStateMachineTest.HAStateChangeListener probe = new HighAvailabilityMemberStateMachineTest.HAStateChangeListener();
    stateMachine.addHighAvailabilityMemberListener(probe);
}
Also used : ClusterMemberEvents(org.neo4j.cluster.member.ClusterMemberEvents) Election(org.neo4j.cluster.protocol.election.Election) Before(org.junit.Before)

Example 4 with Election

use of org.neo4j.cluster.protocol.election.Election in project neo4j by neo4j.

the class HighAvailabilityModeSwitcherTest method shouldAllowForcedElectionsAfterModeSwitch.

@Test
public void shouldAllowForcedElectionsAfterModeSwitch() throws Throwable {
    // Given
    SwitchToSlaveCopyThenBranch switchToSlave = mock(SwitchToSlaveCopyThenBranch.class);
    when(switchToSlave.switchToSlave(any(LifeSupport.class), any(URI.class), any(URI.class), any(CancellationRequest.class))).thenReturn(URI.create("http://localhost"));
    ClusterMemberAvailability memberAvailability = mock(ClusterMemberAvailability.class);
    Election election = mock(Election.class);
    final CountDownLatch modeSwitchHappened = new CountDownLatch(1);
    HighAvailabilityModeSwitcher modeSwitcher = new HighAvailabilityModeSwitcher(switchToSlave, mock(SwitchToMaster.class), election, memberAvailability, mock(ClusterClient.class), storeSupplierMock(), mock(InstanceId.class), new ComponentSwitcherContainer(), neoStoreDataSourceSupplierMock(), NullLogService.getInstance()) {

        @Override
        ScheduledExecutorService createExecutor() {
            ScheduledExecutorService executor = mock(ScheduledExecutorService.class);
            doAnswer(invocation -> {
                ((Runnable) invocation.getArguments()[0]).run();
                modeSwitchHappened.countDown();
                return mock(Future.class);
            }).when(executor).submit(any(Runnable.class));
            return executor;
        }
    };
    modeSwitcher.init();
    modeSwitcher.start();
    modeSwitcher.forceElections();
    reset(memberAvailability, election);
    // When
    modeSwitcher.masterIsAvailable(new HighAvailabilityMemberChangeEvent(PENDING, TO_SLAVE, mock(InstanceId.class), URI.create("http://localhost:9090?serverId=42")));
    modeSwitchHappened.await();
    modeSwitcher.forceElections();
    // Then
    InOrder inOrder = inOrder(memberAvailability, election);
    inOrder.verify(memberAvailability).memberIsUnavailable(HighAvailabilityModeSwitcher.SLAVE);
    inOrder.verify(election).performRoleElections();
    inOrder.verifyNoMoreInteractions();
}
Also used : ScheduledExecutorService(java.util.concurrent.ScheduledExecutorService) InOrder(org.mockito.InOrder) InstanceId(org.neo4j.cluster.InstanceId) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) CountDownLatch(java.util.concurrent.CountDownLatch) URI(java.net.URI) Election(org.neo4j.cluster.protocol.election.Election) ClusterClient(org.neo4j.cluster.client.ClusterClient) SwitchToSlaveCopyThenBranch(org.neo4j.kernel.ha.cluster.SwitchToSlaveCopyThenBranch) HighAvailabilityMemberChangeEvent(org.neo4j.kernel.ha.cluster.HighAvailabilityMemberChangeEvent) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) SwitchToMaster(org.neo4j.kernel.ha.cluster.SwitchToMaster) CancellationRequest(org.neo4j.helpers.CancellationRequest) Test(org.junit.Test)

Example 5 with Election

use of org.neo4j.cluster.protocol.election.Election in project neo4j by neo4j.

the class HighAvailabilityModeSwitcherTest method shouldSwitchToSlaveForNullMasterAndBeSilentWhenMovingToDetached.

@Test
public void shouldSwitchToSlaveForNullMasterAndBeSilentWhenMovingToDetached() throws Throwable {
    // Given
    SwitchToSlaveCopyThenBranch sts = mock(SwitchToSlaveCopyThenBranch.class);
    SwitchToMaster stm = mock(SwitchToMaster.class);
    Election election = mock(Election.class);
    ClusterMemberAvailability cma = mock(ClusterMemberAvailability.class);
    InstanceId instanceId = new InstanceId(14);
    ComponentSwitcher componentSwitcher = mock(ComponentSwitcher.class);
    HighAvailabilityModeSwitcher theSwitcher = new HighAvailabilityModeSwitcher(sts, stm, election, cma, mock(ClusterClient.class), storeSupplierMock(), instanceId, componentSwitcher, neoStoreDataSourceSupplierMock(), NullLogService.getInstance());
    // When
    theSwitcher.init();
    theSwitcher.start();
    theSwitcher.instanceDetached(new HighAvailabilityMemberChangeEvent(HighAvailabilityMemberState.MASTER, HighAvailabilityMemberState.PENDING, null, null));
    // Then
    verify(componentSwitcher).switchToSlave();
    verifyZeroInteractions(cma);
}
Also used : ClusterClient(org.neo4j.cluster.client.ClusterClient) SwitchToSlaveCopyThenBranch(org.neo4j.kernel.ha.cluster.SwitchToSlaveCopyThenBranch) InstanceId(org.neo4j.cluster.InstanceId) HighAvailabilityMemberChangeEvent(org.neo4j.kernel.ha.cluster.HighAvailabilityMemberChangeEvent) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) SwitchToMaster(org.neo4j.kernel.ha.cluster.SwitchToMaster) Election(org.neo4j.cluster.protocol.election.Election) Test(org.junit.Test)

Aggregations

Election (org.neo4j.cluster.protocol.election.Election)7 Test (org.junit.Test)6 InstanceId (org.neo4j.cluster.InstanceId)6 ClusterClient (org.neo4j.cluster.client.ClusterClient)6 ClusterMemberAvailability (org.neo4j.cluster.member.ClusterMemberAvailability)6 SwitchToMaster (org.neo4j.kernel.ha.cluster.SwitchToMaster)5 SwitchToSlaveCopyThenBranch (org.neo4j.kernel.ha.cluster.SwitchToSlaveCopyThenBranch)5 InOrder (org.mockito.InOrder)3 HighAvailabilityMemberChangeEvent (org.neo4j.kernel.ha.cluster.HighAvailabilityMemberChangeEvent)3 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)3 URI (java.net.URI)2 CountDownLatch (java.util.concurrent.CountDownLatch)2 File (java.io.File)1 IOException (java.io.IOException)1 InetSocketAddress (java.net.InetSocketAddress)1 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)1 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)1 AtomicReference (java.util.concurrent.atomic.AtomicReference)1 Before (org.junit.Before)1 ClusterMemberEvents (org.neo4j.cluster.member.ClusterMemberEvents)1