Search in sources :

Example 1 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class RecoveryTest method shouldSeeThatACleanDatabaseShouldNotRequireRecovery.

@Test
public void shouldSeeThatACleanDatabaseShouldNotRequireRecovery() throws Exception {
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            LogPositionMarker marker = new LogPositionMarker();
            // last committed tx
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            writer.writeCommitEntry(4L, 5L);
            // check point
            consumer.accept(marker);
            writer.writeCheckPointEntry(marker.newPosition());
            return true;
        }
    });
    LifeSupport life = new LifeSupport();
    Recovery.Monitor monitor = mock(Recovery.Monitor.class);
    try {
        StorageEngine storageEngine = mock(StorageEngine.class);
        final LogEntryReader<ReadableClosablePositionAwareChannel> reader = new VersionAwareLogEntryReader<>();
        LatestCheckPointFinder finder = new LatestCheckPointFinder(logFiles, fileSystemRule.get(), reader);
        TransactionMetadataCache metadataCache = new TransactionMetadataCache(100);
        LogHeaderCache logHeaderCache = new LogHeaderCache(10);
        LogFile logFile = life.add(new PhysicalLogFile(fileSystemRule.get(), logFiles, 50, () -> transactionIdStore.getLastCommittedTransactionId(), logVersionRepository, mock(PhysicalLogFile.Monitor.class), logHeaderCache));
        LogicalTransactionStore txStore = new PhysicalLogicalTransactionStore(logFile, metadataCache, reader);
        life.add(new Recovery(new DefaultRecoverySPI(storageEngine, logFiles, fileSystemRule.get(), logVersionRepository, finder, transactionIdStore, txStore, NO_MONITOR) {

            @Override
            public Visitor<CommittedTransactionRepresentation, Exception> startRecovery() {
                fail("Recovery should not be required");
                // <-- to satisfy the compiler
                return null;
            }
        }, monitor));
        life.start();
        verifyZeroInteractions(monitor);
    } finally {
        life.shutdown();
    }
}
Also used : DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) StorageEngine(org.neo4j.storageengine.api.StorageEngine) Recovery(org.neo4j.kernel.recovery.Recovery) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Consumer(java.util.function.Consumer) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) Pair(org.neo4j.helpers.collection.Pair) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) IOException(java.io.IOException) IOException(java.io.IOException) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Test(org.junit.Test)

Example 2 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class VersionAwareLogEntryReader method readLogEntry.

@Override
public LogEntry readLogEntry(SOURCE channel) throws IOException {
    try {
        LogPositionMarker positionMarker = new LogPositionMarker();
        channel.getCurrentPosition(positionMarker);
        while (true) {
            LogEntryVersion version = null;
            LogEntryParser<LogEntry> entryReader;
            try {
                /*
                     * if the read type is negative than it is actually the log entry version
                     * so we need to read an extra byte which will contain the type
                     */
                byte typeCode = channel.get();
                byte versionCode = 0;
                if (typeCode < 0) {
                    versionCode = typeCode;
                    typeCode = channel.get();
                }
                version = byVersion(versionCode);
                entryReader = version.entryParser(typeCode);
            } catch (ReadPastEndException e) {
                // Make these exceptions slip by straight out to the outer handler
                throw e;
            } catch (Exception e) {
                // Tag all other exceptions with log position and other useful information
                LogPosition position = positionMarker.newPosition();
                e = withMessage(e, e.getMessage() + ". At position " + position + " and entry version " + version);
                throw launderedException(IOException.class, e);
            }
            LogEntry entry = entryReader.parse(version, channel, positionMarker, commandReaderFactory);
            if (!entryReader.skip()) {
                return entry;
            }
        }
    } catch (ReadPastEndException e) {
        return null;
    }
}
Also used : IOException(java.io.IOException) ReadPastEndException(org.neo4j.storageengine.api.ReadPastEndException) ReadPastEndException(org.neo4j.storageengine.api.ReadPastEndException) Exceptions.launderedException(org.neo4j.helpers.Exceptions.launderedException) IOException(java.io.IOException) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 3 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class BaseCommandReader method unknownCommandType.

protected IOException unknownCommandType(byte commandType, ReadableChannel channel) throws IOException {
    String message = "Unknown command type[" + commandType + "]";
    if (channel instanceof PositionAwareChannel) {
        PositionAwareChannel logChannel = (PositionAwareChannel) channel;
        LogPositionMarker position = new LogPositionMarker();
        logChannel.getCurrentPosition(position);
        message += " near " + position.newPosition();
    }
    return new IOException(message);
}
Also used : PositionAwareChannel(org.neo4j.kernel.impl.transaction.log.PositionAwareChannel) IOException(java.io.IOException) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker)

Example 4 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class VersionAwareLogEntryReaderIT method correctlyResetPositionOfObservedZeroWhenChannelSwitchOnExactlyCheckedByte.

@Test
void correctlyResetPositionOfObservedZeroWhenChannelSwitchOnExactlyCheckedByte() throws IOException {
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fs).withLogEntryReader(entryReader).withLogVersionRepository(new SimpleLogVersionRepository()).withTransactionIdStore(new SimpleTransactionIdStore()).withStoreId(StoreId.UNKNOWN).withKernelVersionProvider(() -> KernelVersion.V4_0).build();
    try (Lifespan lifespan = new Lifespan(logFiles)) {
        LogPositionMarker positionMarker = new LogPositionMarker();
        LogFile logFile = logFiles.getLogFile();
        long initialPosition = getLastReadablePosition(logFiles);
        long checkpointsEndDataOffset = DEFAULT_READ_AHEAD_SIZE + initialPosition;
        TransactionLogWriter logWriter = logFile.getTransactionLogWriter();
        do {
            logWriter.legacyCheckPoint(new LogPosition(4, 5));
            logWriter.getCurrentPosition(positionMarker);
        } while (positionMarker.getByteOffset() <= checkpointsEndDataOffset);
        logFile.flush();
        logFiles.getLogFile().rotate();
        fs.truncate(logFiles.getLogFile().getLogFileForVersion(0), checkpointsEndDataOffset);
        try (StoreChannel storeChannel = fs.write(logFiles.getLogFile().getLogFileForVersion(1))) {
            storeChannel.position(CURRENT_FORMAT_LOG_HEADER_SIZE);
            storeChannel.writeAll(ByteBuffer.wrap(new byte[] { 0 }));
        }
        try (ReadableLogChannel logChannel = logFiles.getLogFile().getReader(new LogPosition(0, initialPosition))) {
            LogEntry logEntry = entryReader.readLogEntry(logChannel);
            // we reading expected checkpoint records
            assertThat(logEntry).isInstanceOf(LogEntryInlinedCheckPoint.class);
            LogEntryInlinedCheckPoint checkPoint = (LogEntryInlinedCheckPoint) logEntry;
            LogPosition logPosition = checkPoint.getLogPosition();
            assertEquals(4, logPosition.getLogVersion());
            assertEquals(5, logPosition.getByteOffset());
            // set position to the end of the buffer to cause channel switch on next byte read
            ((PositionableChannel) logChannel).setCurrentPosition(checkpointsEndDataOffset);
            while (entryReader.readLogEntry(logChannel) != null) {
            // read to the end
            }
            // channel should be switched now and position should be just after the header
            LogPosition position = entryReader.lastPosition();
            assertEquals(CURRENT_FORMAT_LOG_HEADER_SIZE, position.getByteOffset());
            assertEquals(1, position.getLogVersion());
        }
    }
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) StoreChannel(org.neo4j.io.fs.StoreChannel) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) PositionableChannel(org.neo4j.io.fs.PositionableChannel) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) TransactionLogWriter(org.neo4j.kernel.impl.transaction.log.TransactionLogWriter) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.jupiter.api.Test)

Example 5 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class LogPositionMarkerTest method shouldReturnUnspecifiedWhenAskedTo.

@Test
void shouldReturnUnspecifiedWhenAskedTo() {
    // given
    final LogPositionMarker marker = new LogPositionMarker();
    // when
    marker.mark(1, 2);
    marker.unspecified();
    final LogPosition logPosition = marker.newPosition();
    // given
    assertEquals(LogPosition.UNSPECIFIED, logPosition);
}
Also used : LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.jupiter.api.Test)

Aggregations

LogPositionMarker (org.neo4j.kernel.impl.transaction.log.LogPositionMarker)22 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)16 Test (org.junit.jupiter.api.Test)12 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)12 IOException (java.io.IOException)10 Path (java.nio.file.Path)9 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)9 File (java.io.File)7 Consumer (java.util.function.Consumer)7 Test (org.junit.Test)7 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)7 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)7 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)7 Pair (org.neo4j.helpers.collection.Pair)6 LogFile (org.neo4j.kernel.impl.transaction.log.LogFile)6 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)6 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)6 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)6 StorageEngine (org.neo4j.storageengine.api.StorageEngine)6 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)5