Search in sources :

Example 6 with NeoStoreDataSource

use of org.neo4j.kernel.NeoStoreDataSource in project neo4j by neo4j.

the class TransactionLogCatchUpWriterTest method simulateStoreCopy.

private org.neo4j.kernel.impl.store.StoreId simulateStoreCopy() throws IOException {
    // create an empty store
    org.neo4j.kernel.impl.store.StoreId storeId;
    NeoStoreDataSource ds = dsRule.getDataSource(storeDir, fs, pageCache, emptyMap());
    try (Lifespan ignored = new Lifespan(ds)) {
        storeId = ds.getStoreId();
    }
    // we don't have log files after a store copy
    PhysicalLogFiles logFiles = new PhysicalLogFiles(storeDir, fsRule.get());
    logFiles.accept((file, version) -> file.delete());
    return storeId;
}
Also used : NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)

Example 7 with NeoStoreDataSource

use of org.neo4j.kernel.NeoStoreDataSource in project neo4j by neo4j.

the class DefaultMasterImplSPITest method flushStoreFilesWithCorrectCheckpointTriggerName.

@Test
public void flushStoreFilesWithCorrectCheckpointTriggerName() throws IOException {
    CheckPointer checkPointer = mock(CheckPointer.class);
    NeoStoreDataSource dataSource = mock(NeoStoreDataSource.class);
    when(dataSource.listStoreFiles(anyBoolean())).thenReturn(Iterators.emptyIterator());
    DefaultMasterImplSPI master = new DefaultMasterImplSPI(mock(GraphDatabaseAPI.class, RETURNS_MOCKS), mock(FileSystemAbstraction.class), new Monitors(), mock(LabelTokenHolder.class), mock(PropertyKeyTokenHolder.class), mock(RelationshipTypeTokenHolder.class), mock(IdGeneratorFactory.class), mock(TransactionCommitProcess.class), checkPointer, mock(TransactionIdStore.class), mock(LogicalTransactionStore.class), dataSource, mock(PageCache.class), new StoreCopyCheckPointMutex());
    master.flushStoresAndStreamStoreFiles(mock(StoreWriter.class));
    TriggerInfo expectedTriggerInfo = new SimpleTriggerInfo(DefaultMasterImplSPI.STORE_COPY_CHECKPOINT_TRIGGER);
    verify(checkPointer).tryCheckPoint(expectedTriggerInfo);
}
Also used : FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) RelationshipTypeTokenHolder(org.neo4j.kernel.impl.core.RelationshipTypeTokenHolder) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) TransactionCommitProcess(org.neo4j.kernel.impl.api.TransactionCommitProcess) TriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.TriggerInfo) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) IdGeneratorFactory(org.neo4j.kernel.impl.store.id.IdGeneratorFactory) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) SimpleTriggerInfo(org.neo4j.kernel.impl.transaction.log.checkpoint.SimpleTriggerInfo) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) PropertyKeyTokenHolder(org.neo4j.kernel.impl.core.PropertyKeyTokenHolder) StoreWriter(org.neo4j.com.storecopy.StoreWriter) CheckPointer(org.neo4j.kernel.impl.transaction.log.checkpoint.CheckPointer) Monitors(org.neo4j.kernel.monitoring.Monitors) StoreCopyCheckPointMutex(org.neo4j.kernel.impl.transaction.log.checkpoint.StoreCopyCheckPointMutex) LabelTokenHolder(org.neo4j.kernel.impl.core.LabelTokenHolder) PageCache(org.neo4j.io.pagecache.PageCache) Test(org.junit.Test)

Example 8 with NeoStoreDataSource

use of org.neo4j.kernel.NeoStoreDataSource in project neo4j by neo4j.

the class SwitchToSlaveBranchThenCopyTest method newSwitchToSlaveSpy.

@SuppressWarnings("unchecked")
private SwitchToSlaveBranchThenCopy newSwitchToSlaveSpy(PageCache pageCacheMock, StoreCopyClient storeCopyClient) throws IOException {
    ClusterMembers clusterMembers = mock(ClusterMembers.class);
    ClusterMember master = mock(ClusterMember.class);
    when(master.getStoreId()).thenReturn(storeId);
    when(master.getHARole()).thenReturn(HighAvailabilityModeSwitcher.MASTER);
    when(master.hasRole(eq(HighAvailabilityModeSwitcher.MASTER))).thenReturn(true);
    when(master.getInstanceId()).thenReturn(new InstanceId(1));
    when(clusterMembers.getMembers()).thenReturn(singletonList(master));
    Dependencies resolver = new Dependencies();
    resolver.satisfyDependencies(requestContextFactory, clusterMembers, mock(TransactionObligationFulfiller.class), mock(OnlineBackupKernelExtension.class), mock(IndexConfigStore.class), mock(TransactionCommittingResponseUnpacker.class), mock(DataSourceManager.class), mock(StoreLockerLifecycleAdapter.class), mock(FileSystemWatcherService.class));
    NeoStoreDataSource dataSource = mock(NeoStoreDataSource.class);
    when(dataSource.getStoreId()).thenReturn(storeId);
    TransactionStats transactionCounters = mock(TransactionStats.class);
    when(transactionCounters.getNumberOfActiveTransactions()).thenReturn(0L);
    Response<HandshakeResult> response = mock(Response.class);
    when(response.response()).thenReturn(new HandshakeResult(42, 2));
    when(masterClient.handshake(anyLong(), any(StoreId.class))).thenReturn(response);
    when(masterClient.getProtocolVersion()).thenReturn(MasterClient320.PROTOCOL_VERSION);
    TransactionIdStore transactionIdStoreMock = mock(TransactionIdStore.class);
    // note that the checksum (the second member of the array) is the same as the one in the handshake mock above
    when(transactionIdStoreMock.getLastCommittedTransaction()).thenReturn(new TransactionId(42, 42, 42));
    MasterClientResolver masterClientResolver = mock(MasterClientResolver.class);
    when(masterClientResolver.instantiate(anyString(), anyInt(), anyString(), any(Monitors.class), any(StoreId.class), any(LifeSupport.class))).thenReturn(masterClient);
    return spy(new SwitchToSlaveBranchThenCopy(new File(""), NullLogService.getInstance(), configMock(), resolver, mock(HaIdGeneratorFactory.class), mock(DelegateInvocationHandler.class), mock(ClusterMemberAvailability.class), requestContextFactory, pullerFactory, masterClientResolver, mock(SwitchToSlave.Monitor.class), storeCopyClient, Suppliers.singleton(dataSource), Suppliers.singleton(transactionIdStoreMock), slave -> {
        SlaveServer server = mock(SlaveServer.class);
        InetSocketAddress inetSocketAddress = InetSocketAddress.createUnresolved("localhost", 42);
        when(server.getSocketAddress()).thenReturn(inetSocketAddress);
        return server;
    }, updatePuller, pageCacheMock, mock(Monitors.class), transactionCounters));
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) StoreId(org.neo4j.kernel.impl.store.StoreId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) Suppliers(org.neo4j.function.Suppliers) URISyntaxException(java.net.URISyntaxException) OnlineBackupKernelExtension(org.neo4j.backup.OnlineBackupKernelExtension) DelegateInvocationHandler(org.neo4j.kernel.ha.DelegateInvocationHandler) Dependencies(org.neo4j.kernel.impl.util.Dependencies) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) NullLogProvider(org.neo4j.logging.NullLogProvider) MasterClient(org.neo4j.kernel.ha.com.slave.MasterClient) Collections.singletonList(java.util.Collections.singletonList) StoreIdTestFactory.newStoreIdForCurrentVersion(org.neo4j.com.StoreIdTestFactory.newStoreIdForCurrentVersion) HighAvailabilityModeSwitcher(org.neo4j.kernel.ha.cluster.modeswitch.HighAvailabilityModeSwitcher) Mockito.doThrow(org.mockito.Mockito.doThrow) TransactionObligationFulfiller(org.neo4j.com.storecopy.TransactionObligationFulfiller) Matchers.eq(org.mockito.Matchers.eq) Mockito.doAnswer(org.mockito.Mockito.doAnswer) Matchers.anyInt(org.mockito.Matchers.anyInt) Assert.fail(org.junit.Assert.fail) URI(java.net.URI) MismatchingStoreIdException(org.neo4j.kernel.impl.store.MismatchingStoreIdException) Response(org.neo4j.com.Response) PageCache(org.neo4j.io.pagecache.PageCache) InetSocketAddress(java.net.InetSocketAddress) TransactionCommittingResponseUnpacker(org.neo4j.com.storecopy.TransactionCommittingResponseUnpacker) UpdatePuller(org.neo4j.kernel.ha.UpdatePuller) UpdatePullerScheduler(org.neo4j.kernel.ha.UpdatePullerScheduler) Matchers.any(org.mockito.Matchers.any) Stream(java.util.stream.Stream) ClusterMemberAvailability(org.neo4j.cluster.member.ClusterMemberAvailability) Mockito.withSettings(org.mockito.Mockito.withSettings) RequestContextFactory(org.neo4j.kernel.ha.com.RequestContextFactory) SlaveUpdatePuller(org.neo4j.kernel.ha.SlaveUpdatePuller) Mockito.mock(org.mockito.Mockito.mock) BranchedDataException(org.neo4j.kernel.ha.BranchedDataException) BranchedDataPolicy(org.neo4j.kernel.ha.BranchedDataPolicy) MoveAfterCopy(org.neo4j.com.storecopy.MoveAfterCopy) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) PagedFile(org.neo4j.io.pagecache.PagedFile) Monitors(org.neo4j.kernel.monitoring.Monitors) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) Mockito.spy(org.mockito.Mockito.spy) PullerFactory(org.neo4j.kernel.ha.PullerFactory) Matchers.anyString(org.mockito.Matchers.anyString) TransactionId(org.neo4j.kernel.impl.store.TransactionId) ClusterMembers(org.neo4j.kernel.ha.cluster.member.ClusterMembers) StoreLockerLifecycleAdapter(org.neo4j.kernel.internal.StoreLockerLifecycleAdapter) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) HaIdGeneratorFactory(org.neo4j.kernel.ha.id.HaIdGeneratorFactory) CancellationRequest(org.neo4j.helpers.CancellationRequest) Matchers.anyLong(org.mockito.Matchers.anyLong) MapUtil.stringMap(org.neo4j.helpers.collection.MapUtil.stringMap) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) Config(org.neo4j.kernel.configuration.Config) Test(org.junit.Test) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) DataSourceManager(org.neo4j.kernel.impl.transaction.state.DataSourceManager) File(java.io.File) Mockito.verify(org.mockito.Mockito.verify) TimeUnit(java.util.concurrent.TimeUnit) NullLogService(org.neo4j.kernel.impl.logging.NullLogService) Mockito.never(org.mockito.Mockito.never) IndexConfigStore(org.neo4j.kernel.impl.index.IndexConfigStore) Assert.assertNull(org.junit.Assert.assertNull) ClusterSettings(org.neo4j.cluster.ClusterSettings) StoreCopyClient(org.neo4j.com.storecopy.StoreCopyClient) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) ClusterMember(org.neo4j.kernel.ha.cluster.member.ClusterMember) FileSystemWatcherService(org.neo4j.kernel.impl.util.watcher.FileSystemWatcherService) MasterClient320(org.neo4j.kernel.ha.MasterClient320) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) HandshakeResult(org.neo4j.kernel.ha.com.master.HandshakeResult) TransactionObligationFulfiller(org.neo4j.com.storecopy.TransactionObligationFulfiller) InetSocketAddress(java.net.InetSocketAddress) OnlineBackupKernelExtension(org.neo4j.backup.OnlineBackupKernelExtension) ClusterMembers(org.neo4j.kernel.ha.cluster.member.ClusterMembers) DataSourceManager(org.neo4j.kernel.impl.transaction.state.DataSourceManager) SlaveServer(org.neo4j.kernel.ha.com.slave.SlaveServer) ClusterMember(org.neo4j.kernel.ha.cluster.member.ClusterMember) StoreId(org.neo4j.kernel.impl.store.StoreId) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Dependencies(org.neo4j.kernel.impl.util.Dependencies) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) InstanceId(org.neo4j.cluster.InstanceId) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) IndexConfigStore(org.neo4j.kernel.impl.index.IndexConfigStore) TransactionCommittingResponseUnpacker(org.neo4j.com.storecopy.TransactionCommittingResponseUnpacker) TransactionId(org.neo4j.kernel.impl.store.TransactionId) MasterClientResolver(org.neo4j.kernel.ha.com.slave.MasterClientResolver) TransactionStats(org.neo4j.kernel.impl.transaction.TransactionStats) FileSystemWatcherService(org.neo4j.kernel.impl.util.watcher.FileSystemWatcherService) Monitors(org.neo4j.kernel.monitoring.Monitors) StoreLockerLifecycleAdapter(org.neo4j.kernel.internal.StoreLockerLifecycleAdapter) PagedFile(org.neo4j.io.pagecache.PagedFile) File(java.io.File)

Example 9 with NeoStoreDataSource

use of org.neo4j.kernel.NeoStoreDataSource in project neo4j by neo4j.

the class EditionModule method doAfterRecoveryAndStartup.

protected void doAfterRecoveryAndStartup(DatabaseInfo databaseInfo, DependencyResolver dependencyResolver) {
    DiagnosticsManager diagnosticsManager = dependencyResolver.resolveDependency(DiagnosticsManager.class);
    NeoStoreDataSource neoStoreDataSource = dependencyResolver.resolveDependency(NeoStoreDataSource.class);
    diagnosticsManager.prependProvider(new KernelDiagnostics.Versions(databaseInfo, neoStoreDataSource.getStoreId()));
    neoStoreDataSource.registerDiagnosticsWith(diagnosticsManager);
    diagnosticsManager.appendProvider(new KernelDiagnostics.StoreFiles(neoStoreDataSource.getStoreDir()));
}
Also used : KernelDiagnostics(org.neo4j.kernel.internal.KernelDiagnostics) DiagnosticsManager(org.neo4j.kernel.info.DiagnosticsManager) NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource)

Example 10 with NeoStoreDataSource

use of org.neo4j.kernel.NeoStoreDataSource in project neo4j by neo4j.

the class GraphDatabaseFacadeFactory method initFacade.

/**
     * Instantiate a graph database given configuration, dependencies, and a custom implementation of {@link org
     * .neo4j.kernel.impl.factory.GraphDatabaseFacade}.
     *
     * @param storeDir the directory where the Neo4j data store is located
     * @param config configuration
     * @param dependencies the dependencies required to construct the {@link GraphDatabaseFacade}
     * @param graphDatabaseFacade the already created facade which needs initialisation
     * @return the initialised {@link GraphDatabaseFacade}
     */
public GraphDatabaseFacade initFacade(File storeDir, Config config, final Dependencies dependencies, final GraphDatabaseFacade graphDatabaseFacade) {
    PlatformModule platform = createPlatform(storeDir, config, dependencies, graphDatabaseFacade);
    EditionModule edition = editionFactory.apply(platform);
    AtomicReference<QueryExecutionEngine> queryEngine = new AtomicReference<>(noEngine());
    final DataSourceModule dataSource = createDataSource(platform, edition, queryEngine::get);
    Logger msgLog = platform.logging.getInternalLog(getClass()).infoLogger();
    CoreAPIAvailabilityGuard coreAPIAvailabilityGuard = edition.coreAPIAvailabilityGuard;
    ClassicCoreSPI spi = new ClassicCoreSPI(platform, dataSource, msgLog, coreAPIAvailabilityGuard);
    graphDatabaseFacade.init(spi, dataSource.guard, dataSource.threadToTransactionBridge, platform.config);
    // Start it
    platform.dataSourceManager.addListener(new DataSourceManager.Listener() {

        private QueryExecutionEngine engine;

        @Override
        public void registered(NeoStoreDataSource dataSource) {
            if (engine == null) {
                engine = QueryEngineProvider.initialize(platform.dependencies, platform.graphDatabaseFacade, dependencies.executionEngines());
            }
            queryEngine.set(engine);
        }

        @Override
        public void unregistered(NeoStoreDataSource dataSource) {
            queryEngine.set(noEngine());
        }
    });
    Throwable error = null;
    try {
        // Done after create to avoid a redundant
        // "database is now unavailable"
        enableAvailabilityLogging(platform.availabilityGuard, msgLog);
        platform.life.start();
    } catch (final Throwable throwable) {
        error = new RuntimeException("Error starting " + getClass().getName() + ", " + platform.storeDir, throwable);
    } finally {
        if (error != null) {
            try {
                graphDatabaseFacade.shutdown();
            } catch (Throwable shutdownError) {
                error = Exceptions.withSuppressed(shutdownError, error);
            }
        }
    }
    if (error != null) {
        msgLog.log("Failed to start database", error);
        throw Exceptions.launderedException(error);
    }
    return graphDatabaseFacade;
}
Also used : NeoStoreDataSource(org.neo4j.kernel.NeoStoreDataSource) AtomicReference(java.util.concurrent.atomic.AtomicReference) Logger(org.neo4j.logging.Logger) DataSourceManager(org.neo4j.kernel.impl.transaction.state.DataSourceManager) CoreAPIAvailabilityGuard(org.neo4j.kernel.impl.coreapi.CoreAPIAvailabilityGuard) QueryExecutionEngine(org.neo4j.kernel.impl.query.QueryExecutionEngine)

Aggregations

NeoStoreDataSource (org.neo4j.kernel.NeoStoreDataSource)16 Test (org.junit.Test)9 Monitors (org.neo4j.kernel.monitoring.Monitors)5 File (java.io.File)4 URI (java.net.URI)4 InstanceId (org.neo4j.cluster.InstanceId)4 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)4 PageCache (org.neo4j.io.pagecache.PageCache)4 StoreId (org.neo4j.kernel.impl.store.StoreId)4 TransactionIdStore (org.neo4j.kernel.impl.transaction.log.TransactionIdStore)4 IOException (java.io.IOException)3 InetSocketAddress (java.net.InetSocketAddress)3 ClusterMemberAvailability (org.neo4j.cluster.member.ClusterMemberAvailability)3 DataSourceManager (org.neo4j.kernel.impl.transaction.state.DataSourceManager)3 URISyntaxException (java.net.URISyntaxException)2 TimeUnit (java.util.concurrent.TimeUnit)2 AtomicReference (java.util.concurrent.atomic.AtomicReference)2 DependencyResolver (org.neo4j.graphdb.DependencyResolver)2 PagedFile (org.neo4j.io.pagecache.PagedFile)2 AvailabilityGuard (org.neo4j.kernel.AvailabilityGuard)2