Search in sources :

Example 1 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class PhysicalLogicalTransactionStoreTest method shouldOpenCleanStore.

@Test
public void shouldOpenCleanStore() throws Exception {
    // GIVEN
    TransactionIdStore transactionIdStore = new DeadSimpleTransactionIdStore();
    TransactionMetadataCache positionCache = new TransactionMetadataCache(1000);
    LogHeaderCache logHeaderCache = new LogHeaderCache(10);
    LifeSupport life = new LifeSupport();
    PhysicalLogFiles logFiles = new PhysicalLogFiles(testDir, DEFAULT_NAME, fileSystemRule.get());
    Monitor monitor = new Monitors().newMonitor(PhysicalLogFile.Monitor.class);
    LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 1000, transactionIdStore::getLastCommittedTransactionId, mock(LogVersionRepository.class), monitor, logHeaderCache));
    life.add(new BatchingTransactionAppender(logFile, NO_ROTATION, positionCache, transactionIdStore, BYPASS, DATABASE_HEALTH));
    try {
        // WHEN
        life.start();
    } finally {
        life.shutdown();
    }
}
Also used : DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) Monitor(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.Monitor) Monitors(org.neo4j.kernel.monitoring.Monitors) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Test(org.junit.Test)

Example 2 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class EditionModule method setupSecurityModule.

protected static void setupSecurityModule(PlatformModule platformModule, Log log, Procedures procedures, String key) {
    for (SecurityModule candidate : Service.load(SecurityModule.class)) {
        if (candidate.matches(key)) {
            try {
                candidate.setup(new SecurityModule.Dependencies() {

                    @Override
                    public LogService logService() {
                        return platformModule.logging;
                    }

                    @Override
                    public Config config() {
                        return platformModule.config;
                    }

                    @Override
                    public Procedures procedures() {
                        return procedures;
                    }

                    @Override
                    public JobScheduler scheduler() {
                        return platformModule.jobScheduler;
                    }

                    @Override
                    public FileSystemAbstraction fileSystem() {
                        return platformModule.fileSystem;
                    }

                    @Override
                    public LifeSupport lifeSupport() {
                        return platformModule.life;
                    }

                    @Override
                    public DependencySatisfier dependencySatisfier() {
                        return platformModule.dependencies;
                    }
                });
                return;
            } catch (Exception e) {
                String errorMessage = "Failed to load security module.";
                log.error(errorMessage);
                throw new RuntimeException(errorMessage, e);
            }
        }
    }
    String errorMessage = "Failed to load security module with key '" + key + "'.";
    log.error(errorMessage);
    throw new IllegalArgumentException(errorMessage);
}
Also used : JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) Procedures(org.neo4j.kernel.impl.proc.Procedures) KernelException(org.neo4j.kernel.api.exceptions.KernelException) SecurityModule(org.neo4j.kernel.api.security.SecurityModule) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) DependencySatisfier(org.neo4j.kernel.impl.util.DependencySatisfier) LogService(org.neo4j.kernel.impl.logging.LogService)

Example 3 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class BoltKernelExtension method newInstance.

@Override
public Lifecycle newInstance(KernelContext context, Dependencies dependencies) throws Throwable {
    Config config = dependencies.config();
    GraphDatabaseService gdb = dependencies.db();
    GraphDatabaseAPI api = (GraphDatabaseAPI) gdb;
    LogService logService = dependencies.logService();
    Clock clock = dependencies.clock();
    Log log = logService.getInternalLog(WorkerFactory.class);
    LifeSupport life = new LifeSupport();
    JobScheduler scheduler = dependencies.scheduler();
    InternalLoggerFactory.setDefaultFactory(new Netty4LoggerFactory(logService.getInternalLogProvider()));
    Authentication authentication = authentication(dependencies.authManager(), dependencies.userManagerSupplier());
    BoltFactory boltFactory = life.add(new BoltFactoryImpl(api, dependencies.usageData(), logService, dependencies.txBridge(), authentication, dependencies.sessionTracker(), config));
    WorkerFactory workerFactory = createWorkerFactory(boltFactory, scheduler, dependencies, logService, clock);
    List<ProtocolInitializer> connectors = config.enabledBoltConnectors().stream().map((connConfig) -> {
        ListenSocketAddress listenAddress = config.get(connConfig.listen_address);
        AdvertisedSocketAddress advertisedAddress = config.get(connConfig.advertised_address);
        SslContext sslCtx;
        boolean requireEncryption;
        final BoltConnector.EncryptionLevel encryptionLevel = config.get(connConfig.encryption_level);
        switch(encryptionLevel) {
            case REQUIRED:
                // Encrypted connections are mandatory, a self-signed certificate may be generated.
                requireEncryption = true;
                sslCtx = createSslContext(config, log, advertisedAddress);
                break;
            case OPTIONAL:
                // Encrypted connections are optional, a self-signed certificate may be generated.
                requireEncryption = false;
                sslCtx = createSslContext(config, log, advertisedAddress);
                break;
            case DISABLED:
                // Encryption is turned off, no self-signed certificate will be generated.
                requireEncryption = false;
                sslCtx = null;
                break;
            default:
                // In the unlikely event that we happen to fall through to the default option here,
                // there is a mismatch between the BoltConnector.EncryptionLevel enum and the options
                // handled in this switch statement. In this case, we'll log a warning and default to
                // disabling encryption, since this mirrors the functionality introduced in 3.0.
                log.warn(format("Unhandled encryption level %s - assuming DISABLED.", encryptionLevel.name()));
                requireEncryption = false;
                sslCtx = null;
                break;
        }
        final Map<Long, BiFunction<Channel, Boolean, BoltProtocol>> versions = newVersions(logService, workerFactory);
        return new SocketTransport(listenAddress, sslCtx, requireEncryption, logService.getInternalLogProvider(), versions);
    }).collect(toList());
    if (connectors.size() > 0 && !config.get(GraphDatabaseSettings.disconnected)) {
        life.add(new NettyServer(scheduler.threadFactory(boltNetworkIO), connectors));
        log.info("Bolt Server extension loaded.");
        for (ProtocolInitializer connector : connectors) {
            logService.getUserLog(WorkerFactory.class).info("Bolt enabled on %s.", connector.address());
        }
    }
    return life;
}
Also used : Service(org.neo4j.helpers.Service) UsageData(org.neo4j.udc.UsageData) Log(org.neo4j.logging.Log) Authentication(org.neo4j.bolt.security.auth.Authentication) BiFunction(java.util.function.BiFunction) OperatorCreationException(org.bouncycastle.operator.OperatorCreationException) KernelContext(org.neo4j.kernel.impl.spi.KernelContext) SocketTransport(org.neo4j.bolt.transport.SocketTransport) BoltProtocol(org.neo4j.bolt.transport.BoltProtocol) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) KeyStoreFactory(org.neo4j.bolt.security.ssl.KeyStoreFactory) AdvertisedSocketAddress(org.neo4j.helpers.AdvertisedSocketAddress) GeneralSecurityException(java.security.GeneralSecurityException) ProtocolInitializer(org.neo4j.bolt.transport.NettyServer.ProtocolInitializer) Map(java.util.Map) KeyStoreInformation(org.neo4j.bolt.security.ssl.KeyStoreInformation) Groups.boltNetworkIO(org.neo4j.kernel.impl.util.JobScheduler.Groups.boltNetworkIO) BoltConnectionDescriptor(org.neo4j.bolt.v1.runtime.BoltConnectionDescriptor) BoltFactory(org.neo4j.bolt.v1.runtime.BoltFactory) BoltConnector(org.neo4j.kernel.configuration.BoltConnector) ThreadToStatementContextBridge(org.neo4j.kernel.impl.core.ThreadToStatementContextBridge) LogService(org.neo4j.kernel.impl.logging.LogService) String.format(java.lang.String.format) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Netty4LoggerFactory(org.neo4j.bolt.transport.Netty4LoggerFactory) Settings.derivedSetting(org.neo4j.kernel.configuration.Settings.derivedSetting) List(java.util.List) Description(org.neo4j.configuration.Description) BoltFactoryImpl(org.neo4j.bolt.v1.runtime.BoltFactoryImpl) KernelExtensionFactory(org.neo4j.kernel.extension.KernelExtensionFactory) WorkerFactory(org.neo4j.bolt.v1.runtime.WorkerFactory) BasicAuthentication(org.neo4j.bolt.security.auth.BasicAuthentication) GraphDatabaseSettings(org.neo4j.graphdb.factory.GraphDatabaseSettings) Settings.pathSetting(org.neo4j.kernel.configuration.Settings.pathSetting) Internal(org.neo4j.configuration.Internal) Monitors(org.neo4j.kernel.monitoring.Monitors) HashMap(java.util.HashMap) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) Configuration(org.neo4j.graphdb.config.Configuration) ListenSocketAddress(org.neo4j.helpers.ListenSocketAddress) BoltProtocolV1(org.neo4j.bolt.v1.transport.BoltProtocolV1) Certificates(org.neo4j.bolt.security.ssl.Certificates) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) NettyServer(org.neo4j.bolt.transport.NettyServer) ThreadedWorkerFactory(org.neo4j.bolt.v1.runtime.concurrent.ThreadedWorkerFactory) Lifecycle(org.neo4j.kernel.lifecycle.Lifecycle) Config(org.neo4j.kernel.configuration.Config) SslContext(io.netty.handler.ssl.SslContext) BoltConnectionTracker(org.neo4j.kernel.api.bolt.BoltConnectionTracker) Setting(org.neo4j.graphdb.config.Setting) IOException(java.io.IOException) PATH(org.neo4j.kernel.configuration.Settings.PATH) File(java.io.File) Channel(io.netty.channel.Channel) UserManagerSupplier(org.neo4j.kernel.api.security.UserManagerSupplier) BoltWorker(org.neo4j.bolt.v1.runtime.BoltWorker) Collectors.toList(java.util.stream.Collectors.toList) SslContextBuilder(io.netty.handler.ssl.SslContextBuilder) Clock(java.time.Clock) InternalLoggerFactory(io.netty.util.internal.logging.InternalLoggerFactory) MonitoredWorkerFactory(org.neo4j.bolt.v1.runtime.MonitoredWorkerFactory) AuthManager(org.neo4j.kernel.api.security.AuthManager) Config(org.neo4j.kernel.configuration.Config) AdvertisedSocketAddress(org.neo4j.helpers.AdvertisedSocketAddress) Clock(java.time.Clock) BoltFactory(org.neo4j.bolt.v1.runtime.BoltFactory) WorkerFactory(org.neo4j.bolt.v1.runtime.WorkerFactory) ThreadedWorkerFactory(org.neo4j.bolt.v1.runtime.concurrent.ThreadedWorkerFactory) MonitoredWorkerFactory(org.neo4j.bolt.v1.runtime.MonitoredWorkerFactory) NettyServer(org.neo4j.bolt.transport.NettyServer) BoltFactoryImpl(org.neo4j.bolt.v1.runtime.BoltFactoryImpl) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) SslContext(io.netty.handler.ssl.SslContext) ProtocolInitializer(org.neo4j.bolt.transport.NettyServer.ProtocolInitializer) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) Log(org.neo4j.logging.Log) SocketTransport(org.neo4j.bolt.transport.SocketTransport) Channel(io.netty.channel.Channel) BoltProtocol(org.neo4j.bolt.transport.BoltProtocol) Authentication(org.neo4j.bolt.security.auth.Authentication) BasicAuthentication(org.neo4j.bolt.security.auth.BasicAuthentication) ListenSocketAddress(org.neo4j.helpers.ListenSocketAddress) Netty4LoggerFactory(org.neo4j.bolt.transport.Netty4LoggerFactory) Map(java.util.Map) HashMap(java.util.HashMap) LogService(org.neo4j.kernel.impl.logging.LogService)

Example 4 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class RecoveryTest method recover.

private boolean recover(PhysicalLogFiles logFiles) {
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    final AtomicBoolean recoveryRequired = new AtomicBoolean();
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                recoveryRequired.set(true);
                return super.startRecovery();
            }
        }, monitor));
        life.start();
    } finally {
        life.shutdown();
    }
    return recoveryRequired.get();
}
Also used : PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) IOException(java.io.IOException) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)

Example 5 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class RecoveryTest method shouldSeeThatACleanDatabaseShouldNotRequireRecovery.

@Test
public void shouldSeeThatACleanDatabaseShouldNotRequireRecovery() throws Exception {
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            LogPositionMarker marker = new LogPositionMarker();
            // last committed tx
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            writer.writeCommitEntry(4L, 5L);
            // check point
            consumer.accept(marker);
            writer.writeCheckPointEntry(marker.newPosition());
            return true;
        }
    });
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                fail("Recovery should not be required");
                // <-- to satisfy the compiler
                return null;
            }
        }, monitor));
        life.start();
        verifyZeroInteractions(monitor);
    } finally {
        life.shutdown();
    }
}
Also used : DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Consumer(java.util.function.Consumer) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) Pair(org.neo4j.helpers.collection.Pair) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) IOException(java.io.IOException) IOException(java.io.IOException) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Test(org.junit.Test)

Aggregations

LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)51 Test (org.junit.Test)22 IOException (java.io.IOException)14 File (java.io.File)12 Monitors (org.neo4j.kernel.monitoring.Monitors)12 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)8 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)8 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)8 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)6 JobScheduler (org.neo4j.kernel.impl.util.JobScheduler)6 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)5 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)5 Monitor (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.Monitor)5 Dependencies (org.neo4j.kernel.impl.util.Dependencies)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 URI (java.net.URI)4 Matchers.anyString (org.mockito.Matchers.anyString)4 Config (org.neo4j.kernel.configuration.Config)4 UnableToCopyStoreFromOldMasterException (org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException)4 KernelContext (org.neo4j.kernel.impl.spi.KernelContext)4