Search in sources :

Example 6 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class StoreMigration method run.

public void run(final FileSystemAbstraction fs, final File storeDirectory, Config config, LogProvider userLogProvider) throws IOException {
    StoreLogService logService = StoreLogService.withUserLogProvider(userLogProvider).inLogsDirectory(fs, storeDirectory);
    VisibleMigrationProgressMonitor progressMonitor = new VisibleMigrationProgressMonitor(logService.getUserLog(StoreMigration.class));
    LifeSupport life = new LifeSupport();
    life.add(logService);
    // Add participants from kernel extensions...
    LegacyIndexProvider legacyIndexProvider = new LegacyIndexProvider();
    Log log = userLogProvider.getLog(StoreMigration.class);
    try (PageCache pageCache = createPageCache(fs, config)) {
        Dependencies deps = new Dependencies();
        deps.satisfyDependencies(fs, config, legacyIndexProvider, pageCache, logService);
        KernelContext kernelContext = new SimpleKernelContext(storeDirectory, DatabaseInfo.UNKNOWN, deps);
        KernelExtensions kernelExtensions = life.add(new KernelExtensions(kernelContext, GraphDatabaseDependencies.newDependencies().kernelExtensions(), deps, ignore()));
        // Add the kernel store migrator
        life.start();
        SchemaIndexProvider schemaIndexProvider = kernelExtensions.resolveDependency(SchemaIndexProvider.class, HighestSelectionStrategy.getInstance());
        LabelScanStoreProvider labelScanStoreProvider = kernelExtensions.resolveDependency(LabelScanStoreProvider.class, new NamedLabelScanStoreSelectionStrategy(config));
        long startTime = System.currentTimeMillis();
        DatabaseMigrator migrator = new DatabaseMigrator(progressMonitor, fs, config, logService, schemaIndexProvider, labelScanStoreProvider, legacyIndexProvider.getIndexProviders(), pageCache, RecordFormatSelector.selectForConfig(config, userLogProvider));
        migrator.migrate(storeDirectory);
        long duration = System.currentTimeMillis() - startTime;
        log.info(format("Migration completed in %d s%n", duration / 1000));
    } catch (Exception e) {
        throw new StoreUpgrader.UnableToUpgradeException("Failure during upgrade", e);
    } finally {
        life.shutdown();
    }
}
Also used : SchemaIndexProvider(org.neo4j.kernel.api.index.SchemaIndexProvider) LabelScanStoreProvider(org.neo4j.kernel.impl.api.scan.LabelScanStoreProvider) VisibleMigrationProgressMonitor(org.neo4j.kernel.impl.storemigration.monitoring.VisibleMigrationProgressMonitor) StoreLogService(org.neo4j.kernel.impl.logging.StoreLogService) Log(org.neo4j.logging.Log) IOException(java.io.IOException) DatabaseMigrator(org.neo4j.kernel.impl.storemigration.DatabaseMigrator) KernelExtensions(org.neo4j.kernel.extension.KernelExtensions) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Dependencies(org.neo4j.kernel.impl.util.Dependencies) GraphDatabaseDependencies(org.neo4j.kernel.GraphDatabaseDependencies) StoreUpgrader(org.neo4j.kernel.impl.storemigration.StoreUpgrader) NamedLabelScanStoreSelectionStrategy(org.neo4j.kernel.extension.dependency.NamedLabelScanStoreSelectionStrategy) PageCache(org.neo4j.io.pagecache.PageCache) ConfigurableStandalonePageCacheFactory.createPageCache(org.neo4j.kernel.impl.pagecache.ConfigurableStandalonePageCacheFactory.createPageCache) SimpleKernelContext(org.neo4j.kernel.impl.spi.SimpleKernelContext) KernelContext(org.neo4j.kernel.impl.spi.KernelContext)

Example 7 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class ApplyTransactionsCommand method applyTransactions.

private long applyTransactions(File fromPath, GraphDatabaseAPI toDb, long fromTxExclusive, long toTxInclusive, PrintStream out) throws IOException, TransactionFailureException {
    DependencyResolver resolver = toDb.getDependencyResolver();
    TransactionRepresentationCommitProcess commitProcess = new TransactionRepresentationCommitProcess(resolver.resolveDependency(TransactionAppender.class), resolver.resolveDependency(StorageEngine.class));
    LifeSupport life = new LifeSupport();
    try (DefaultFileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction();
        PageCache pageCache = StandalonePageCacheFactory.createPageCache(fileSystem)) {
        LogicalTransactionStore source = life.add(new ReadOnlyTransactionStore(pageCache, fileSystem, fromPath, new Monitors()));
        life.start();
        long lastAppliedTx = fromTxExclusive;
        // Some progress if there are more than a couple of transactions to apply
        ProgressListener progress = toTxInclusive - fromTxExclusive >= 100 ? textual(out).singlePart("Application progress", toTxInclusive - fromTxExclusive) : ProgressListener.NONE;
        try (IOCursor<CommittedTransactionRepresentation> cursor = source.getTransactions(fromTxExclusive + 1)) {
            while (cursor.next()) {
                CommittedTransactionRepresentation transaction = cursor.get();
                TransactionRepresentation transactionRepresentation = transaction.getTransactionRepresentation();
                try {
                    commitProcess.commit(new TransactionToApply(transactionRepresentation), NULL, EXTERNAL);
                    progress.add(1);
                } catch (final Throwable e) {
                    System.err.println("ERROR applying transaction " + transaction.getCommitEntry().getTxId());
                    throw e;
                }
                lastAppliedTx = transaction.getCommitEntry().getTxId();
                if (lastAppliedTx == toTxInclusive) {
                    break;
                }
            }
        }
        return lastAppliedTx;
    } finally {
        life.shutdown();
    }
}
Also used : TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) TransactionAppender(org.neo4j.kernel.impl.transaction.log.TransactionAppender) TransactionRepresentation(org.neo4j.kernel.impl.transaction.TransactionRepresentation) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) ReadOnlyTransactionStore(org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionStore) DependencyResolver(org.neo4j.graphdb.DependencyResolver) ProgressListener(org.neo4j.helpers.progress.ProgressListener) TransactionRepresentationCommitProcess(org.neo4j.kernel.impl.api.TransactionRepresentationCommitProcess) Monitors(org.neo4j.kernel.monitoring.Monitors) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) PageCache(org.neo4j.io.pagecache.PageCache)

Example 8 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class DatabaseRebuildTool method run.

public void run(String... arguments) throws Exception {
    if (arguments.length == 0) {
        System.err.println("Tool for rebuilding database from transaction logs onto a new store");
        System.err.println("Example: dbrebuild --from path/to/some.db --to path/to/new.db apply next");
        System.err.println("         dbrebuild --from path/to/some.db --to path/to/new.db -i");
        System.err.println("          --from : which db to use as source for reading transactions");
        System.err.println("            --to : where to build the new db");
        System.err.println("  --overwrite-to : always starts from empty 'to' db");
        System.err.println("              -i : interactive mode (enter a shell)");
        return;
    }
    Args args = Args.withFlags("i", "overwrite-to").parse(arguments);
    File fromPath = getFrom(args);
    File toPath = getTo(args);
    GraphDatabaseBuilder dbBuilder = newDbBuilder(toPath, args);
    boolean interactive = args.getBoolean("i");
    if (interactive && !args.orphans().isEmpty()) {
        throw new IllegalArgumentException("No additional commands allowed in interactive mode");
    }
    @SuppressWarnings("resource") InputStream input = interactive ? in : oneCommand(args.orphansAsArray());
    LifeSupport life = new LifeSupport();
    ConsoleInput consoleInput = console(fromPath, dbBuilder, input, interactive ? staticPrompt("# ") : NO_PROMPT, life);
    life.start();
    try {
        consoleInput.waitFor();
    } finally {
        life.shutdown();
    }
}
Also used : Args(org.neo4j.helpers.Args) InputStream(java.io.InputStream) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) ConsoleInput(org.neo4j.tools.console.input.ConsoleInput) File(java.io.File) GraphDatabaseBuilder(org.neo4j.graphdb.factory.GraphDatabaseBuilder)

Example 9 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class ClusterTopologyChangesIT method failedInstanceShouldReceiveCorrectCoordinatorIdUponRejoiningCluster.

@Test
public void failedInstanceShouldReceiveCorrectCoordinatorIdUponRejoiningCluster() throws Throwable {
    // Given
    HighlyAvailableGraphDatabase initialMaster = cluster.getMaster();
    // When
    cluster.shutdown(initialMaster);
    cluster.await(masterAvailable(initialMaster));
    cluster.await(masterSeesSlavesAsAvailable(1));
    // create node on new master to ensure that it has the greatest tx id
    createNodeOn(cluster.getMaster());
    cluster.sync();
    LifeSupport life = new LifeSupport();
    ClusterClientModule clusterClient = newClusterClient(life, new InstanceId(1));
    cleanup.add(life);
    final AtomicReference<InstanceId> coordinatorIdWhenReJoined = new AtomicReference<>();
    final CountDownLatch latch = new CountDownLatch(1);
    clusterClient.clusterClient.addClusterListener(new ClusterListener.Adapter() {

        @Override
        public void enteredCluster(ClusterConfiguration clusterConfiguration) {
            coordinatorIdWhenReJoined.set(clusterConfiguration.getElected(COORDINATOR));
            latch.countDown();
        }
    });
    life.start();
    // Then
    assertTrue(latch.await(20, SECONDS));
    assertEquals(new InstanceId(2), coordinatorIdWhenReJoined.get());
}
Also used : InstanceId(org.neo4j.cluster.InstanceId) ClusterListener(org.neo4j.cluster.protocol.cluster.ClusterListener) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) AtomicReference(java.util.concurrent.atomic.AtomicReference) ClusterConfiguration(org.neo4j.cluster.protocol.cluster.ClusterConfiguration) CountDownLatch(java.util.concurrent.CountDownLatch) ClusterClientModule(org.neo4j.cluster.client.ClusterClientModule) Test(org.junit.Test)

Example 10 with LifeSupport

use of org.neo4j.kernel.lifecycle.LifeSupport in project neo4j by neo4j.

the class BackupProtocolTest method shouldGatherForensicsInFullBackupRequest.

private void shouldGatherForensicsInFullBackupRequest(boolean forensics) throws Exception {
    // GIVEN
    Response<Void> response = Response.EMPTY;
    StoreId storeId = response.getStoreId();
    String host = "localhost";
    int port = BackupServer.DEFAULT_PORT;
    LifeSupport life = new LifeSupport();
    LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
    BackupClient client = life.add(new BackupClient(host, port, null, NullLogProvider.getInstance(), storeId, 10_000, mock(ResponseUnpacker.class), mock(ByteCounterMonitor.class), mock(RequestMonitor.class), reader));
    ControlledBackupInterface backup = new ControlledBackupInterface();
    life.add(new BackupServer(backup, new HostnamePort(host, port), NullLogProvider.getInstance(), mock(ByteCounterMonitor.class), mock(RequestMonitor.class)));
    life.start();
    try {
        // WHEN
        StoreWriter writer = mock(StoreWriter.class);
        client.fullBackup(writer, forensics);
        // THEN
        assertEquals(forensics, backup.receivedForensics);
    } finally {
        life.shutdown();
    }
}
Also used : HostnamePort(org.neo4j.helpers.HostnamePort) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) StoreId(org.neo4j.kernel.impl.store.StoreId) StoreWriter(org.neo4j.com.storecopy.StoreWriter) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)

Aggregations

LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)51 Test (org.junit.Test)22 IOException (java.io.IOException)14 File (java.io.File)12 Monitors (org.neo4j.kernel.monitoring.Monitors)12 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)8 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)8 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)8 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)6 JobScheduler (org.neo4j.kernel.impl.util.JobScheduler)6 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)5 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)5 Monitor (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.Monitor)5 Dependencies (org.neo4j.kernel.impl.util.Dependencies)5 StorageEngine (org.neo4j.storageengine.api.StorageEngine)5 URI (java.net.URI)4 Matchers.anyString (org.mockito.Matchers.anyString)4 Config (org.neo4j.kernel.configuration.Config)4 UnableToCopyStoreFromOldMasterException (org.neo4j.kernel.ha.store.UnableToCopyStoreFromOldMasterException)4 KernelContext (org.neo4j.kernel.impl.spi.KernelContext)4