Search in sources :

Example 11 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class BackupServiceIT method checkLastCommittedTxIdInLogAndNeoStore.

private void checkLastCommittedTxIdInLogAndNeoStore(long txId, long txIdFromOrigin) throws Exception {
    // Assert last committed transaction can be found in tx log and is the last tx in the log
    LifeSupport life = new LifeSupport();
    PageCache pageCache = pageCacheRule.getPageCache(fileSystem);
    LogicalTransactionStore transactionStore = life.add(new ReadOnlyTransactionStore(pageCache, fileSystem, backupDir, monitors));
    life.start();
    try (IOCursor<CommittedTransactionRepresentation> cursor = transactionStore.getTransactions(txId)) {
        assertTrue(cursor.next());
        assertEquals(txId, cursor.get().getCommitEntry().getTxId());
        assertFalse(cursor.next());
    } finally {
        life.shutdown();
    }
    // Assert last committed transaction is correct in neostore
    assertEquals(txId, txIdFromOrigin);
}
Also used : CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) PageCache(org.neo4j.io.pagecache.PageCache) ReadOnlyTransactionStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionStore)

Example 12 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class ConsistencyCheckService method runFullConsistencyCheck.

public Result runFullConsistencyCheck(final File storeDir, Config config, ProgressMonitorFactory progressFactory, final LogProvider logProvider, final FileSystemAbstraction fileSystem, final PageCache pageCache, final boolean verbose, File reportDir, CheckConsistencyConfig checkConsistencyConfig) throws ConsistencyCheckIncompleteException {
    Log log = logProvider.getLog(getClass());
    config = config.with(stringMap(GraphDatabaseSettings.read_only.name(), TRUE, GraphDatabaseSettings.label_index.name(), LabelIndex.AUTO.name()));
    StoreFactory factory = new StoreFactory(storeDir, config, new DefaultIdGeneratorFactory(fileSystem), pageCache, fileSystem, logProvider);
    ConsistencySummaryStatistics summary;
    final File reportFile = chooseReportPath(reportDir);
    Log reportLog = new ConsistencyReportLog(Suppliers.lazySingleton(() -> {
        try {
            return new PrintWriter(createOrOpenAsOuputStream(fileSystem, reportFile, true));
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }));
    // Bootstrap kernel extensions
    LifeSupport life = new LifeSupport();
    try (NeoStores neoStores = factory.openAllNeoStores()) {
        IndexStoreView indexStoreView = new NeoStoreIndexStoreView(LockService.NO_LOCK_SERVICE, neoStores);
        Dependencies dependencies = new Dependencies();
        dependencies.satisfyDependencies(config, fileSystem, new SimpleLogService(logProvider, logProvider), indexStoreView, pageCache);
        KernelContext kernelContext = new SimpleKernelContext(storeDir, UNKNOWN, dependencies);
        KernelExtensions extensions = life.add(new KernelExtensions(kernelContext, (Iterable) load(KernelExtensionFactory.class), dependencies, ignore()));
        life.start();
        LabelScanStore labelScanStore = life.add(extensions.resolveDependency(LabelScanStoreProvider.class, new NamedLabelScanStoreSelectionStrategy(config)).getLabelScanStore());
        SchemaIndexProvider indexes = life.add(extensions.resolveDependency(SchemaIndexProvider.class, HighestSelectionStrategy.getInstance()));
        int numberOfThreads = defaultConsistencyCheckThreadsNumber();
        Statistics statistics;
        StoreAccess storeAccess;
        AccessStatistics stats = new AccessStatistics();
        if (verbose) {
            statistics = new VerboseStatistics(stats, new DefaultCounts(numberOfThreads), log);
            storeAccess = new AccessStatsKeepingStoreAccess(neoStores, stats);
        } else {
            statistics = Statistics.NONE;
            storeAccess = new StoreAccess(neoStores);
        }
        storeAccess.initialize();
        DirectStoreAccess stores = new DirectStoreAccess(storeAccess, labelScanStore, indexes);
        FullCheck check = new FullCheck(progressFactory, statistics, numberOfThreads, checkConsistencyConfig);
        summary = check.execute(stores, new DuplicatingLog(log, reportLog));
    } finally {
        life.shutdown();
    }
    if (!summary.isConsistent()) {
        log.warn("See '%s' for a detailed consistency report.", reportFile.getPath());
        return Result.failure(reportFile);
    }
    return Result.success(reportFile);
}
Also used : LabelScanStore(org.neo4j.kernel.api.labelscan.LabelScanStore) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) StoreAccess(org.neo4j.kernel.impl.store.StoreAccess) SimpleLogService(org.neo4j.kernel.impl.logging.SimpleLogService) DirectStoreAccess(org.neo4j.kernel.api.direct.DirectStoreAccess) StoreFactory(org.neo4j.kernel.impl.store.StoreFactory) DefaultCounts(org.neo4j.consistency.statistics.DefaultCounts) AccessStatistics(org.neo4j.consistency.statistics.AccessStatistics) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) DuplicatingLog(org.neo4j.logging.DuplicatingLog) VerboseStatistics(org.neo4j.consistency.statistics.VerboseStatistics) Dependencies(org.neo4j.kernel.impl.util.Dependencies) NamedLabelScanStoreSelectionStrategy(org.neo4j.kernel.extension.dependency.NamedLabelScanStoreSelectionStrategy) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) PrintWriter(java.io.PrintWriter) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) KernelContext(org.neo4j.kernel.impl.spi.KernelContext) SchemaIndexProvider(org.neo4j.kernel.api.index.SchemaIndexProvider) Log(org.neo4j.logging.Log) DuplicatingLog(org.neo4j.logging.DuplicatingLog) DefaultIdGeneratorFactory(org.neo4j.kernel.impl.store.id.DefaultIdGeneratorFactory) AccessStatsKeepingStoreAccess(org.neo4j.consistency.statistics.AccessStatsKeepingStoreAccess) IOException(java.io.IOException) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) Statistics(org.neo4j.consistency.statistics.Statistics) VerboseStatistics(org.neo4j.consistency.statistics.VerboseStatistics) ConsistencySummaryStatistics(org.neo4j.consistency.report.ConsistencySummaryStatistics) AccessStatistics(org.neo4j.consistency.statistics.AccessStatistics) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) FullCheck(org.neo4j.consistency.checking.full.FullCheck) KernelExtensions(org.neo4j.kernel.extension.KernelExtensions) NeoStores(org.neo4j.kernel.impl.store.NeoStores) IndexStoreView(org.neo4j.kernel.impl.api.index.IndexStoreView) NeoStoreIndexStoreView(org.neo4j.kernel.impl.transaction.state.storeview.NeoStoreIndexStoreView) File(java.io.File)

Example 13 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class KernelExtensionsTest method shouldConsultUnsatisfiedDependencyHandler.

@Test
public void shouldConsultUnsatisfiedDependencyHandler() throws Exception {
    // GIVEN
    KernelContext context = mock(KernelContext.class);
    UnsatisfiedDependencyStrategy handler = mock(UnsatisfiedDependencyStrategy.class);
    // that hasn't got anything.
    Dependencies dependencies = new Dependencies();
    TestingExtensionFactory extensionFactory = new TestingExtensionFactory();
    KernelExtensions extensions = new KernelExtensions(context, extensions(extensionFactory), dependencies, handler);
    // WHEN
    LifeSupport life = new LifeSupport();
    life.add(extensions);
    try {
        life.start();
        // THEN
        verify(handler).handle(eq(extensionFactory), any(UnsatisfiedDependencyException.class));
    } finally {
        life.shutdown();
    }
}
Also used : UnsatisfiedDependencyException(org.neo4j.kernel.impl.util.UnsatisfiedDependencyException) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Dependencies(org.neo4j.kernel.impl.util.Dependencies) KernelContext(org.neo4j.kernel.impl.spi.KernelContext) Test(org.junit.Test)

Example 14 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class EditionModule method setupSecurityModule.

protected static void setupSecurityModule(PlatformModule platformModule, Log log, Procedures procedures, String key) {
    for (SecurityModule candidate : Service.load(SecurityModule.class)) {
        if (candidate.matches(key)) {
            try {
                candidate.setup(new SecurityModule.Dependencies() {

                    @Override
                    public LogService logService() {
                        return platformModule.logging;
                    }

                    @Override
                    public Config config() {
                        return platformModule.config;
                    }

                    @Override
                    public Procedures procedures() {
                        return procedures;
                    }

                    @Override
                    public JobScheduler scheduler() {
                        return platformModule.jobScheduler;
                    }

                    @Override
                    public FileSystemAbstraction fileSystem() {
                        return platformModule.fileSystem;
                    }

                    @Override
                    public LifeSupport lifeSupport() {
                        return platformModule.life;
                    }

                    @Override
                    public DependencySatisfier dependencySatisfier() {
                        return platformModule.dependencies;
                    }
                });
                return;
            } catch (Exception e) {
                String errorMessage = "Failed to load security module.";
                log.error(errorMessage);
                throw new RuntimeException(errorMessage, e);
            }
        }
    }
    String errorMessage = "Failed to load security module with key '" + key + "'.";
    log.error(errorMessage);
    throw new IllegalArgumentException(errorMessage);
}
Also used : JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) Procedures(org.neo4j.kernel.impl.proc.Procedures) KernelException(org.neo4j.kernel.api.exceptions.KernelException) SecurityModule(org.neo4j.kernel.api.security.SecurityModule) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) DependencySatisfier(org.neo4j.kernel.impl.util.DependencySatisfier) LogService(org.neo4j.kernel.impl.logging.LogService)

Example 15 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class BoltKernelExtension method newInstance.

@Override
public Lifecycle newInstance(KernelContext context, Dependencies dependencies) throws Throwable {
    Config config = dependencies.config();
    GraphDatabaseService gdb = dependencies.db();
    GraphDatabaseAPI api = (GraphDatabaseAPI) gdb;
    LogService logService = dependencies.logService();
    Clock clock = dependencies.clock();
    Log log = logService.getInternalLog(WorkerFactory.class);
    LifeSupport life = new LifeSupport();
    JobScheduler scheduler = dependencies.scheduler();
    InternalLoggerFactory.setDefaultFactory(new Netty4LoggerFactory(logService.getInternalLogProvider()));
    Authentication authentication = authentication(dependencies.authManager(), dependencies.userManagerSupplier());
    BoltFactory boltFactory = life.add(new BoltFactoryImpl(api, dependencies.usageData(), logService, dependencies.txBridge(), authentication, dependencies.sessionTracker(), config));
    WorkerFactory workerFactory = createWorkerFactory(boltFactory, scheduler, dependencies, logService, clock);
    List<ProtocolInitializer> connectors = config.enabledBoltConnectors().stream().map((connConfig) -> {
        ListenSocketAddress listenAddress = config.get(connConfig.listen_address);
        AdvertisedSocketAddress advertisedAddress = config.get(connConfig.advertised_address);
        SslContext sslCtx;
        boolean requireEncryption;
        final BoltConnector.EncryptionLevel encryptionLevel = config.get(connConfig.encryption_level);
        switch(encryptionLevel) {
            case REQUIRED:
                // Encrypted connections are mandatory, a self-signed certificate may be generated.
                requireEncryption = true;
                sslCtx = createSslContext(config, log, advertisedAddress);
                break;
            case OPTIONAL:
                // Encrypted connections are optional, a self-signed certificate may be generated.
                requireEncryption = false;
                sslCtx = createSslContext(config, log, advertisedAddress);
                break;
            case DISABLED:
                // Encryption is turned off, no self-signed certificate will be generated.
                requireEncryption = false;
                sslCtx = null;
                break;
            default:
                // In the unlikely event that we happen to fall through to the default option here,
                // there is a mismatch between the BoltConnector.EncryptionLevel enum and the options
                // handled in this switch statement. In this case, we'll log a warning and default to
                // disabling encryption, since this mirrors the functionality introduced in 3.0.
                log.warn(format("Unhandled encryption level %s - assuming DISABLED.", encryptionLevel.name()));
                requireEncryption = false;
                sslCtx = null;
                break;
        }
        final Map<Long, BiFunction<Channel, Boolean, BoltProtocol>> versions = newVersions(logService, workerFactory);
        return new SocketTransport(listenAddress, sslCtx, requireEncryption, logService.getInternalLogProvider(), versions);
    }).collect(toList());
    if (connectors.size() > 0 && !config.get(GraphDatabaseSettings.disconnected)) {
        life.add(new NettyServer(scheduler.threadFactory(boltNetworkIO), connectors));
        log.info("Bolt Server extension loaded.");
        for (ProtocolInitializer connector : connectors) {
            logService.getUserLog(WorkerFactory.class).info("Bolt enabled on %s.", connector.address());
        }
    }
    return life;
}
Also used : Service(org.neo4j.helpers.Service) UsageData(org.neo4j.udc.UsageData) Log(org.neo4j.logging.Log) Authentication(org.neo4j.bolt.security.auth.Authentication) BiFunction(java.util.function.BiFunction) OperatorCreationException(org.bouncycastle.operator.OperatorCreationException) KernelContext(org.neo4j.kernel.impl.spi.KernelContext) SocketTransport(org.neo4j.bolt.transport.SocketTransport) BoltProtocol(org.neo4j.bolt.transport.BoltProtocol) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) KeyStoreFactory(org.neo4j.bolt.security.ssl.KeyStoreFactory) AdvertisedSocketAddress(org.neo4j.helpers.AdvertisedSocketAddress) GeneralSecurityException(java.security.GeneralSecurityException) ProtocolInitializer(org.neo4j.bolt.transport.NettyServer.ProtocolInitializer) Map(java.util.Map) KeyStoreInformation(org.neo4j.bolt.security.ssl.KeyStoreInformation) Groups.boltNetworkIO(org.neo4j.kernel.impl.util.JobScheduler.Groups.boltNetworkIO) BoltConnectionDescriptor(org.neo4j.bolt.v1.runtime.BoltConnectionDescriptor) BoltFactory(org.neo4j.bolt.v1.runtime.BoltFactory) BoltConnector(org.neo4j.kernel.configuration.BoltConnector) ThreadToStatementContextBridge(org.neo4j.kernel.impl.core.ThreadToStatementContextBridge) LogService(org.neo4j.kernel.impl.logging.LogService) String.format(java.lang.String.format) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Netty4LoggerFactory(org.neo4j.bolt.transport.Netty4LoggerFactory) Settings.derivedSetting(org.neo4j.kernel.configuration.Settings.derivedSetting) List(java.util.List) Description(org.neo4j.configuration.Description) BoltFactoryImpl(org.neo4j.bolt.v1.runtime.BoltFactoryImpl) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) WorkerFactory(org.neo4j.bolt.v1.runtime.WorkerFactory) BasicAuthentication(org.neo4j.bolt.security.auth.BasicAuthentication) GraphDatabaseSettings(org.neo4j.graphdb.factory.GraphDatabaseSettings) Settings.pathSetting(org.neo4j.kernel.configuration.Settings.pathSetting) Internal(org.neo4j.configuration.Internal) Monitors(org.neo4j.kernel.monitoring.Monitors) HashMap(java.util.HashMap) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) Configuration(org.neo4j.graphdb.config.Configuration) ListenSocketAddress(org.neo4j.helpers.ListenSocketAddress) BoltProtocolV1(org.neo4j.bolt.v1.transport.BoltProtocolV1) Certificates(org.neo4j.bolt.security.ssl.Certificates) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) NettyServer(org.neo4j.bolt.transport.NettyServer) ThreadedWorkerFactory(org.neo4j.bolt.v1.runtime.concurrent.ThreadedWorkerFactory) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) Config(org.neo4j.kernel.configuration.Config) SslContext(io.netty.handler.ssl.SslContext) BoltConnectionTracker(org.neo4j.kernel.api.bolt.BoltConnectionTracker) Setting(org.neo4j.graphdb.config.Setting) IOException(java.io.IOException) PATH(org.neo4j.kernel.configuration.Settings.PATH) File(java.io.File) Channel(io.netty.channel.Channel) UserManagerSupplier(org.neo4j.kernel.api.security.UserManagerSupplier) BoltWorker(org.neo4j.bolt.v1.runtime.BoltWorker) Collectors.toList(java.util.stream.Collectors.toList) SslContextBuilder(io.netty.handler.ssl.SslContextBuilder) Clock(java.time.Clock) InternalLoggerFactory(io.netty.util.internal.logging.InternalLoggerFactory) MonitoredWorkerFactory(org.neo4j.bolt.v1.runtime.MonitoredWorkerFactory) AuthManager(org.neo4j.kernel.api.security.AuthManager) Config(org.neo4j.kernel.configuration.Config) AdvertisedSocketAddress(org.neo4j.helpers.AdvertisedSocketAddress) Clock(java.time.Clock) BoltFactory(org.neo4j.bolt.v1.runtime.BoltFactory) WorkerFactory(org.neo4j.bolt.v1.runtime.WorkerFactory) ThreadedWorkerFactory(org.neo4j.bolt.v1.runtime.concurrent.ThreadedWorkerFactory) MonitoredWorkerFactory(org.neo4j.bolt.v1.runtime.MonitoredWorkerFactory) NettyServer(org.neo4j.bolt.transport.NettyServer) BoltFactoryImpl(org.neo4j.bolt.v1.runtime.BoltFactoryImpl) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) SslContext(io.netty.handler.ssl.SslContext) ProtocolInitializer(org.neo4j.bolt.transport.NettyServer.ProtocolInitializer) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) Log(org.neo4j.logging.Log) SocketTransport(org.neo4j.bolt.transport.SocketTransport) Channel(io.netty.channel.Channel) BoltProtocol(org.neo4j.bolt.transport.BoltProtocol) Authentication(org.neo4j.bolt.security.auth.Authentication) BasicAuthentication(org.neo4j.bolt.security.auth.BasicAuthentication) ListenSocketAddress(org.neo4j.helpers.ListenSocketAddress) Netty4LoggerFactory(org.neo4j.bolt.transport.Netty4LoggerFactory) Map(java.util.Map) HashMap(java.util.HashMap) LogService(org.neo4j.kernel.impl.logging.LogService)

Aggregations

LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)51 Test (org.junit.Test)22 IOException (java.io.IOException)14 File (java.io.File)12 Monitors (org.neo4j.kernel.monitoring.Monitors)12 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)8 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)8 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)8 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)6 JobScheduler (org.neo4j.kernel.impl.util.JobScheduler)6 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)5 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)5 Monitor (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.Monitor)5 Dependencies (org.neo4j.kernel.impl.util.Dependencies)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 URI (java.net.URI)4 Matchers.anyString (org.mockito.Matchers.anyString)4 Config (org.neo4j.kernel.configuration.Config)4 UnableToCopyStoreFromOldMasterException (org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException)4 KernelContext (org.neo4j.kernel.impl.spi.KernelContext)4