Search in sources :

Example 1 with DefaultRecoverySPI

use of org.neo4j.kernel.recovery.DefaultRecoverySPI in project neo4j by neo4j.

the class RecoveryTest method recover.

private boolean recover(PhysicalLogFiles logFiles) {
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    final AtomicBoolean recoveryRequired = new AtomicBoolean();
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                recoveryRequired.set(true);
                return super.startRecovery();
            }
        }, monitor));
        life.start();
    } finally {
        life.shutdown();
    }
    return recoveryRequired.get();
}
Also used : PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) IOException(java.io.IOException) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)

Example 2 with DefaultRecoverySPI

use of org.neo4j.kernel.recovery.DefaultRecoverySPI in project neo4j by neo4j.

the class RecoveryTest method shouldSeeThatACleanDatabaseShouldNotRequireRecovery.

@Test
public void shouldSeeThatACleanDatabaseShouldNotRequireRecovery() throws Exception {
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            LogPositionMarker marker = new LogPositionMarker();
            // last committed tx
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            writer.writeCommitEntry(4L, 5L);
            // check point
            consumer.accept(marker);
            writer.writeCheckPointEntry(marker.newPosition());
            return true;
        }
    });
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                fail("Recovery should not be required");
                // <-- to satisfy the compiler
                return null;
            }
        }, monitor));
        life.start();
        verifyZeroInteractions(monitor);
    } finally {
        life.shutdown();
    }
}
Also used : DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Consumer(java.util.function.Consumer) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) Pair(org.neo4j.helpers.collection.Pair) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) IOException(java.io.IOException) IOException(java.io.IOException) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Test(org.junit.Test)

Example 3 with DefaultRecoverySPI

use of org.neo4j.kernel.recovery.DefaultRecoverySPI in project neo4j by neo4j.

the class RecoveryTest method shouldRecoverExistingData.

@Test
public void shouldRecoverExistingData() throws Exception {
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            LogPositionMarker marker = new LogPositionMarker();
            // last committed tx
            consumer.accept(marker);
            LogPosition lastCommittedTxPosition = marker.newPosition();
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            lastCommittedTxStartEntry = new LogEntryStart(0, 1, 2L, 3L, new byte[0], lastCommittedTxPosition);
            writer.writeCommitEntry(4L, 5L);
            lastCommittedTxCommitEntry = new OnePhaseCommit(4L, 5L);
            // check point pointing to the previously committed transaction
            writer.writeCheckPointEntry(lastCommittedTxPosition);
            expectedCheckPointEntry = new CheckPoint(lastCommittedTxPosition);
            // tx committed after checkpoint
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 6L, 4L, new byte[0]);
            expectedStartEntry = new LogEntryStart(0, 1, 6L, 4L, new byte[0], marker.newPosition());
            writer.writeCommitEntry(5L, 7L);
            expectedCommitEntry = new OnePhaseCommit(5L, 7L);
            return true;
        }
    });
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    final AtomicBoolean recoveryRequired = new AtomicBoolean();
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            private int nr = 0;

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                recoveryRequired.set(true);
                final Visitor<CommittedTransactionRepresentation, Exception> actual = super.startRecovery();
                return new Visitor<CommittedTransactionRepresentation, Exception>() {

                    @Override
                    public boolean visit(CommittedTransactionRepresentation tx) throws Exception {
                        actual.visit(tx);
                        switch(nr++) {
                            case 0:
                                assertEquals(lastCommittedTxStartEntry, tx.getStartEntry());
                                assertEquals(lastCommittedTxCommitEntry, tx.getCommitEntry());
                                break;
                            case 1:
                                assertEquals(expectedStartEntry, tx.getStartEntry());
                                assertEquals(expectedCommitEntry, tx.getCommitEntry());
                                break;
                            default:
                                fail("Too many recovered transactions");
                        }
                        return false;
                    }
                };
            }
        }, monitor));
        life.start();
        InOrder order = inOrder(monitor);
        order.verify(monitor, times(1)).recoveryRequired(any(LogPosition.class));
        order.verify(monitor, times(1)).recoveryCompleted(2);
        assertTrue(recoveryRequired.get());
    } finally {
        life.shutdown();
    }
}
Also used : DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) Visitor(org.neo4j.helpers.collection.Visitor) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Consumer(java.util.function.Consumer) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) Pair(org.neo4j.helpers.collection.Pair) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) InOrder(org.mockito.InOrder) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) IOException(java.io.IOException) IOException(java.io.IOException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Test(org.junit.Test)

Example 4 with DefaultRecoverySPI

use of org.neo4j.kernel.recovery.DefaultRecoverySPI in project neo4j by neo4j.

the class NeoStoreDataSource method buildRecovery.

private void buildRecovery(final FileSystemAbstraction fileSystemAbstraction, TransactionIdStore transactionIdStore, LogVersionRepository logVersionRepository, Recovery.Monitor recoveryMonitor, PositionToRecoverFrom.Monitor positionMonitor, final PhysicalLogFiles logFiles, final StartupStatisticsProvider startupStatistics, StorageEngine storageEngine, LogEntryReader<ReadableClosablePositionAwareChannel> logEntryReader, LogicalTransactionStore logicalTransactionStore) {
    final LatestCheckPointFinder checkPointFinder = new LatestCheckPointFinder(logFiles, fileSystemAbstraction, logEntryReader);
    Recovery.SPI spi = new DefaultRecoverySPI(storageEngine, logFiles, fileSystemAbstraction, logVersionRepository, checkPointFinder, transactionIdStore, logicalTransactionStore, positionMonitor);
    Recovery recovery = new Recovery(spi, recoveryMonitor);
    monitors.addMonitorListener(new Recovery.Monitor() {

        @Override
        public void recoveryCompleted(int numberOfRecoveredTransactions) {
            startupStatistics.setNumberOfRecoveredTransactions(numberOfRecoveredTransactions);
        }
    });
    life.add(recovery);
}
Also used : LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) Recovery(org.neo4j.kernel.recovery.Recovery)

Aggregations

DefaultRecoverySPI (org.neo4j.kernel.recovery.DefaultRecoverySPI)4 LatestCheckPointFinder (org.neo4j.kernel.recovery.LatestCheckPointFinder)4 Recovery (org.neo4j.kernel.recovery.Recovery)4 IOException (java.io.IOException)3 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)3 LogFile (org.neo4j.kernel.impl.transaction.log.LogFile)3 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)3 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)3 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)3 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)3 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)3 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)3 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)3 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)3 StorageEngine (org.neo4j.storageengine.api.StorageEngine)3 File (java.io.File)2 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)2 Consumer (java.util.function.Consumer)2 Test (org.junit.Test)2 Pair (org.neo4j.helpers.collection.Pair)2