Search in sources :

Example 1 with CancellationRequest

use of org.neo4j.helpers.CancellationRequest in project neo4j by neo4j.

the class SwitchToSlaveCopyThenBranchTest method shouldNotBranchStoreUnlessWeHaveCopiedDownAReplacement.

@Test
public void shouldNotBranchStoreUnlessWeHaveCopiedDownAReplacement() throws Throwable {
    // Given
    StoreCopyClient storeCopyClient = mock(StoreCopyClient.class);
    doAnswer(invocation -> {
        MoveAfterCopy moveAfterCopy = invocation.getArgumentAt(2, MoveAfterCopy.class);
        moveAfterCopy.move(Stream.empty(), new File(""), new File(""));
        return null;
    }).when(storeCopyClient).copyStore(any(StoreCopyClient.StoreCopyRequester.class), any(CancellationRequest.class), any(MoveAfterCopy.class));
    PageCache pageCacheMock = mock(PageCache.class);
    PagedFile pagedFileMock = mock(PagedFile.class);
    when(pagedFileMock.getLastPageId()).thenReturn(1L);
    when(pageCacheMock.map(any(File.class), anyInt())).thenReturn(pagedFileMock);
    SwitchToSlaveCopyThenBranch switchToSlave = newSwitchToSlaveSpy(pageCacheMock, storeCopyClient);
    URI masterUri = new URI("cluster://localhost?serverId=1");
    URI me = new URI("cluster://localhost?serverId=2");
    CancellationRequest cancellationRequest = CancellationRequest.NEVER_CANCELLED;
    MasterClient masterClient = mock(MasterClient.class);
    when(masterClient.handshake(anyLong(), any(StoreId.class))).thenThrow(new BranchedDataException(""));
    TransactionIdStore transactionIdStore = mock(TransactionIdStore.class);
    when(transactionIdStore.getLastCommittedTransaction()).thenReturn(new TransactionId(42, 42, 42));
    when(transactionIdStore.getLastCommittedTransactionId()).thenReturn(TransactionIdStore.BASE_TX_ID);
    // When
    BranchedDataPolicy branchPolicy = mock(BranchedDataPolicy.class);
    switchToSlave.stopServicesAndHandleBranchedStore(branchPolicy, masterUri, me, cancellationRequest);
    // Then
    InOrder inOrder = Mockito.inOrder(storeCopyClient, branchPolicy);
    inOrder.verify(storeCopyClient).copyStore(any(StoreCopyClient.StoreCopyRequester.class), any(CancellationRequest.class), any(MoveAfterCopy.class));
    inOrder.verify(branchPolicy).handle(new File(""), pageCacheMock, NullLogService.getInstance());
}
Also used : PagedFile(org.neo4j.io.pagecache.PagedFile) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) InOrder(org.mockito.InOrder) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) URI(java.net.URI) TransactionId(org.neo4j.kernel.impl.store.TransactionId) StoreId(org.neo4j.kernel.impl.store.StoreId) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) BranchedDataPolicy(org.neo4j.kernel.ha.BranchedDataPolicy) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File) BranchedDataException(org.neo4j.kernel.ha.BranchedDataException) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy) CancellationRequest(org.neo4j.helpers.CancellationRequest) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 2 with CancellationRequest

use of org.neo4j.helpers.CancellationRequest in project neo4j by neo4j.

the class SwitchToSlaveCopyThenBranch method stopServicesAndHandleBranchedStore.

void stopServicesAndHandleBranchedStore(BranchedDataPolicy branchPolicy, URI masterUri, URI me, CancellationRequest cancellationRequest) throws Throwable {
    MoveAfterCopy moveWithCopyThenBranch = (moves, fromDirectory, toDirectory) -> {
        stopServices();
        msgLog.debug("Branching store: " + storeDir);
        branchPolicy.handle(storeDir, pageCache, logService);
        msgLog.debug("Moving downloaded store from " + fromDirectory + " to " + toDirectory);
        MoveAfterCopy.moveReplaceExisting().move(moves, fromDirectory, toDirectory);
        msgLog.debug("Moved downloaded store from " + fromDirectory + " to " + toDirectory);
    };
    copyStore(masterUri, me, cancellationRequest, moveWithCopyThenBranch);
}
Also used : BranchedDataPolicy(org.neo4j.kernel.ha.BranchedDataPolicy) StoreId(org.neo4j.kernel.impl.store.StoreId) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) Monitors(org.neo4j.kernel.monitoring.Monitors) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) StoreUnableToParticipateInClusterException(org.neo4j.kernel.ha.StoreUnableToParticipateInClusterException) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Function(java.util.function.Function) Supplier(java.util.function.Supplier) PullerFactory(org.neo4j.kernel.ha.PullerFactory) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) HaIdGeneratorFactory(org.neo4j.kernel.ha.id.HaIdGeneratorFactory) CancellationRequest(org.neo4j.helpers.CancellationRequest) URI(java.net.URI) MismatchingStoreIdException(org.neo4j.kernel.impl.store.MismatchingStoreIdException) BASE_TX_ID(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_ID) PageCache(org.neo4j.io.pagecache.PageCache) Config(org.neo4j.kernel.configuration.Config) IOException(java.io.IOException) LogService(org.neo4j.kernel.impl.logging.LogService) HaSettings(org.neo4j.kernel.ha.HaSettings) File(java.io.File) UpdatePuller(org.neo4j.kernel.ha.UpdatePuller) ForeignStoreException(org.neo4j.kernel.ha.store.ForeignStoreException) UnableToCopyStoreFromOldMasterException(org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) DependencyResolver(org.neo4j.graphdb.DependencyResolver) Slave(org.neo4j.kernel.ha.com.master.Slave) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) RequestContextFactory(org.neo4j.kernel.ha.com.RequestContextFactory) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Master(org.neo4j.kernel.ha.com.master.Master) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy)

Example 3 with CancellationRequest

use of org.neo4j.helpers.CancellationRequest in project neo4j by neo4j.

the class StoreCopyClientTest method shouldDeleteTempCopyFolderOnFailures.

@Test
public void shouldDeleteTempCopyFolderOnFailures() throws Exception {
    // GIVEN
    File initialStore = testDir.directory("initialStore");
    File backupStore = testDir.directory("backupStore");
    PageCache pageCache = pageCacheRule.getPageCache(fileSystem);
    GraphDatabaseService initialDatabase = createInitialDatabase(initialStore);
    StoreCopyClient copier = new StoreCopyClient(backupStore, Config.empty(), loadKernelExtensions(), NullLogProvider.getInstance(), fileSystem, pageCache, new StoreCopyClient.Monitor.Adapter(), false);
    CancellationRequest falseCancellationRequest = () -> false;
    RuntimeException exception = new RuntimeException("Boom!");
    StoreCopyClient.StoreCopyRequester storeCopyRequest = new LocalStoreCopyRequester((GraphDatabaseAPI) initialDatabase, initialStore, fileSystem) {

        @Override
        public void done() {
            throw exception;
        }
    };
    // WHEN
    try {
        copier.copyStore(storeCopyRequest, falseCancellationRequest, MoveAfterCopy.moveReplaceExisting());
        fail("should have thrown ");
    } catch (RuntimeException ex) {
        assertEquals(exception, ex);
    }
    // THEN
    assertFalse(new File(backupStore, TEMP_COPY_DIRECTORY_NAME).exists());
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) CancellationRequest(org.neo4j.helpers.CancellationRequest) Test(org.junit.Test)

Example 4 with CancellationRequest

use of org.neo4j.helpers.CancellationRequest in project neo4j by neo4j.

the class StoreCopyClientTest method shouldResetNeoStoreLastTransactionOffsetForNonForensicCopy.

@Test
public void shouldResetNeoStoreLastTransactionOffsetForNonForensicCopy() throws Exception {
    // GIVEN
    File initialStore = testDir.directory("initialStore");
    File backupStore = testDir.directory("backupStore");
    PageCache pageCache = pageCacheRule.getPageCache(fileSystem);
    createInitialDatabase(initialStore);
    long originalTransactionOffset = MetaDataStore.getRecord(pageCache, new File(initialStore, MetaDataStore.DEFAULT_NAME), MetaDataStore.Position.LAST_CLOSED_TRANSACTION_LOG_BYTE_OFFSET);
    GraphDatabaseService initialDatabase = startDatabase(initialStore);
    StoreCopyClient copier = new StoreCopyClient(backupStore, Config.empty(), loadKernelExtensions(), NullLogProvider.getInstance(), fileSystem, pageCache, new StoreCopyClient.Monitor.Adapter(), false);
    CancellationRequest falseCancellationRequest = () -> false;
    StoreCopyClient.StoreCopyRequester storeCopyRequest = new LocalStoreCopyRequester((GraphDatabaseAPI) initialDatabase, initialStore, fileSystem);
    // WHEN
    copier.copyStore(storeCopyRequest, falseCancellationRequest, MoveAfterCopy.moveReplaceExisting());
    // THEN
    long updatedTransactionOffset = MetaDataStore.getRecord(pageCache, new File(backupStore, MetaDataStore.DEFAULT_NAME), MetaDataStore.Position.LAST_CLOSED_TRANSACTION_LOG_BYTE_OFFSET);
    assertNotEquals(originalTransactionOffset, updatedTransactionOffset);
    assertEquals(LogHeader.LOG_HEADER_SIZE, updatedTransactionOffset);
    assertFalse(new File(backupStore, TEMP_COPY_DIRECTORY_NAME).exists());
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) File(java.io.File) PageCache(org.neo4j.io.pagecache.PageCache) CancellationRequest(org.neo4j.helpers.CancellationRequest) Test(org.junit.Test)

Example 5 with CancellationRequest

use of org.neo4j.helpers.CancellationRequest in project neo4j by neo4j.

the class HighAvailabilityModeSwitcherTest method shouldReswitchToSlaveIfNewMasterBecameElectedAndAvailableDuringSwitch.

@Test
public void shouldReswitchToSlaveIfNewMasterBecameElectedAndAvailableDuringSwitch() throws Throwable {
    // Given
    final CountDownLatch switching = new CountDownLatch(1);
    final CountDownLatch slaveAvailable = new CountDownLatch(2);
    final AtomicBoolean firstSwitch = new AtomicBoolean(true);
    ClusterMemberAvailability availability = mock(ClusterMemberAvailability.class);
    SwitchToSlaveCopyThenBranch switchToSlave = mock(SwitchToSlaveCopyThenBranch.class);
    @SuppressWarnings("resource") SwitchToMaster switchToMaster = mock(SwitchToMaster.class);
    when(switchToSlave.switchToSlave(any(LifeSupport.class), any(URI.class), any(URI.class), any(CancellationRequest.class))).thenAnswer(invocationOnMock -> {
        switching.countDown();
        CancellationRequest cancel = (CancellationRequest) invocationOnMock.getArguments()[3];
        if (firstSwitch.get()) {
            while (!cancel.cancellationRequested()) {
                Thread.sleep(1);
            }
            firstSwitch.set(false);
        }
        slaveAvailable.countDown();
        return URI.create("ha://slave");
    });
    HighAvailabilityModeSwitcher toTest = new HighAvailabilityModeSwitcher(switchToSlave, switchToMaster, mock(Election.class), availability, mock(ClusterClient.class), storeSupplierMock(), mock(InstanceId.class), new ComponentSwitcherContainer(), neoStoreDataSourceSupplierMock(), NullLogService.getInstance());
    toTest.init();
    toTest.start();
    toTest.listeningAt(URI.create("ha://server3?serverId=3"));
    // When
    // This will start a switch to slave
    toTest.masterIsAvailable(new HighAvailabilityMemberChangeEvent(PENDING, TO_SLAVE, mock(InstanceId.class), URI.create("ha://server1")));
    // Wait until it starts and blocks on the cancellation request
    switching.await();
    // change the elected master, moving to pending, cancelling the previous change. This will block until the
    // previous switch is aborted
    toTest.masterIsElected(new HighAvailabilityMemberChangeEvent(TO_SLAVE, PENDING, new InstanceId(2), URI.create("ha://server2")));
    // Now move to the new master by switching to TO_SLAVE
    toTest.masterIsAvailable(new HighAvailabilityMemberChangeEvent(PENDING, TO_SLAVE, new InstanceId(2), URI.create("ha://server2")));
    // Then
    // The second switch must happen and this test won't block
    slaveAvailable.await();
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) CountDownLatch(java.util.concurrent.CountDownLatch) URI(java.net.URI) Election(org.neo4j.cluster.protocol.election.Election) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) ClusterClient(org.neo4j.cluster.client.ClusterClient) SwitchToSlaveCopyThenBranch(org.neo4j.kernel.ha.cluster.SwitchToSlaveCopyThenBranch) HighAvailabilityMemberChangeEvent(org.neo4j.kernel.ha.cluster.HighAvailabilityMemberChangeEvent) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) SwitchToMaster(org.neo4j.kernel.ha.cluster.SwitchToMaster) CancellationRequest(org.neo4j.helpers.CancellationRequest) Test(org.junit.Test)

Aggregations

CancellationRequest (org.neo4j.helpers.CancellationRequest)6 File (java.io.File)5 PageCache (org.neo4j.io.pagecache.PageCache)5 URI (java.net.URI)4 Test (org.junit.Test)4 ClusterMemberAvailability (org.neo4j.cluster.member.ClusterMemberAvailability)3 MoveAfterCopy (org.neo4j.com.storecopy.MoveAfterCopy)3 StoreCopyClient (org.neo4j.com.storecopy.StoreCopyClient)3 IOException (java.io.IOException)2 Function (java.util.function.Function)2 Supplier (java.util.function.Supplier)2 InstanceId (org.neo4j.cluster.InstanceId)2 DependencyResolver (org.neo4j.graphdb.DependencyResolver)2 GraphDatabaseService (org.neo4j.graphdb.GraphDatabaseService)2 NeoStoreDataSource (org.neo4j.kernel.NeoStoreDataSource)2 Config (org.neo4j.kernel.configuration.Config)2 BranchedDataException (org.neo4j.kernel.ha.BranchedDataException)2 BranchedDataPolicy (org.neo4j.kernel.ha.BranchedDataPolicy)2 DelegateInvocationHandler (org.neo4j.kernel.ha.DelegateInvocationHandler)2 HaSettings (org.neo4j.kernel.ha.HaSettings)2