Search in sources :

Example 26 with LogHeader

use of org.neo4j.kernel.impl.transaction.log.entry.LogHeader in project neo4j by neo4j.

the class LegacyCheckpointLogFile method reachableCheckpoints.

public List<CheckpointInfo> reachableCheckpoints() throws IOException {
    var logFile = logFiles.getLogFile();
    long highestVersion = logFile.getHighestLogVersion();
    if (highestVersion < 0) {
        return emptyList();
    }
    long lowestVersion = logFile.getLowestLogVersion();
    long currentVersion = highestVersion;
    var checkpoints = new ArrayList<CheckpointInfo>();
    while (currentVersion >= lowestVersion) {
        try (var channel = logFile.openForVersion(currentVersion);
            var reader = new ReadAheadLogChannel(channel, NO_MORE_CHANNELS, context.getMemoryTracker());
            var logEntryCursor = new LogEntryCursor(context.getLogEntryReader(), reader)) {
            LogHeader logHeader = logFile.extractHeader(currentVersion);
            var storeId = logHeader.getStoreId();
            LogPosition lastLocation = reader.getCurrentPosition();
            while (logEntryCursor.next()) {
                LogEntry logEntry = logEntryCursor.get();
                // Collect data about latest checkpoint
                if (logEntry instanceof LogEntryInlinedCheckPoint) {
                    checkpoints.add(new CheckpointInfo((LogEntryInlinedCheckPoint) logEntry, storeId, lastLocation));
                }
                lastLocation = reader.getCurrentPosition();
            }
            currentVersion--;
        }
    }
    return checkpoints;
}
Also used : ArrayList(java.util.ArrayList) LogEntryInlinedCheckPoint(org.neo4j.kernel.impl.transaction.log.entry.LogEntryInlinedCheckPoint) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 27 with LogHeader

use of org.neo4j.kernel.impl.transaction.log.entry.LogHeader in project neo4j by neo4j.

the class TransactionLogChannelAllocatorIT method rawChannelDoesNotTryToAdviseOnFileContent.

@Test
void rawChannelDoesNotTryToAdviseOnFileContent() throws IOException {
    Path path = fileHelper.getLogFileForVersion(1);
    try (var storeChannel = fileSystem.write(path)) {
        writeLogHeader(storeChannel, new LogHeader(CURRENT_LOG_FORMAT_VERSION, 1, 1, 1), INSTANCE);
    }
    var logHeaderCache = new LogHeaderCache(10);
    var logFileContext = createLogFileContext();
    var nativeChannelAccessor = new AdviseCountingChannelNativeAccessor();
    var channelAllocator = new TransactionLogChannelAllocator(logFileContext, fileHelper, logHeaderCache, nativeChannelAccessor);
    try (var channel = channelAllocator.openLogChannel(1, true)) {
        assertEquals(0, nativeChannelAccessor.getCallCounter());
    }
}
Also used : Path(java.nio.file.Path) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) Test(org.junit.jupiter.api.Test)

Example 28 with LogHeader

use of org.neo4j.kernel.impl.transaction.log.entry.LogHeader in project neo4j by neo4j.

the class TransactionLogChannelAllocatorIT method defaultChannelTryToAdviseOnFileContent.

@Test
void defaultChannelTryToAdviseOnFileContent() throws IOException {
    Path path = fileHelper.getLogFileForVersion(1);
    try (StoreChannel storeChannel = fileSystem.write(path)) {
        writeLogHeader(storeChannel, new LogHeader(CURRENT_LOG_FORMAT_VERSION, 1, 1, 1), INSTANCE);
    }
    var logHeaderCache = new LogHeaderCache(10);
    var logFileContext = createLogFileContext();
    var nativeChannelAccessor = new AdviseCountingChannelNativeAccessor();
    var channelAllocator = new TransactionLogChannelAllocator(logFileContext, fileHelper, logHeaderCache, nativeChannelAccessor);
    try (var channel = channelAllocator.openLogChannel(1)) {
        assertEquals(1, nativeChannelAccessor.getCallCounter());
    }
}
Also used : Path(java.nio.file.Path) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) Test(org.junit.jupiter.api.Test)

Example 29 with LogHeader

use of org.neo4j.kernel.impl.transaction.log.entry.LogHeader in project neo4j by neo4j.

the class LogTestUtils method filterTransactionLogFile.

private static void filterTransactionLogFile(FileSystemAbstraction fileSystem, Path file, final LogHook<LogEntry> filter, ChannelNativeAccessor channelNativeAccessor) throws IOException {
    filter.file(file);
    try (StoreChannel in = fileSystem.read(file)) {
        LogHeader logHeader = readLogHeader(ByteBuffers.allocate(CURRENT_FORMAT_LOG_HEADER_SIZE, INSTANCE), in, true, file);
        assert logHeader != null : "Looks like we tried to read a log header of an empty pre-allocated file.";
        PhysicalLogVersionedStoreChannel inChannel = new PhysicalLogVersionedStoreChannel(in, logHeader.getLogVersion(), logHeader.getLogFormatVersion(), file, channelNativeAccessor);
        ReadableLogChannel inBuffer = new ReadAheadLogChannel(inChannel, INSTANCE);
        LogEntryReader entryReader = new VersionAwareLogEntryReader(new TestCommandReaderFactory());
        LogEntry entry;
        while ((entry = entryReader.readLogEntry(inBuffer)) != null) {
            filter.test(entry);
        }
    }
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 30 with LogHeader

use of org.neo4j.kernel.impl.transaction.log.entry.LogHeader in project neo4j by neo4j.

the class LegacyLogs method getTransactionInformation.

public Optional<TransactionId> getTransactionInformation(File storeDir, long transactionId) throws IOException {
    List<File> logFiles = Arrays.asList(fs.listFiles(storeDir, versionedLegacyLogFilesFilter));
    logFiles.sort(NEWEST_FIRST);
    for (File file : logFiles) {
        Pair<LogHeader, IOCursor<LogEntry>> pair = reader.openReadableChannel(file);
        boolean hadAnyTransactions = false;
        try (IOCursor<LogEntry> cursor = pair.other()) {
            // The log entries will come sorted from this cursor, so no need to keep track of identifiers and such.
            LogEntryStart startEntry = null;
            while (cursor.next()) {
                LogEntry logEntry = cursor.get();
                if (logEntry instanceof LogEntryStart) {
                    startEntry = (LogEntryStart) logEntry;
                } else if (logEntry instanceof LogEntryCommit) {
                    hadAnyTransactions = true;
                    LogEntryCommit commitEntry = logEntry.as();
                    if (commitEntry.getTxId() == transactionId) {
                        return Optional.of(new TransactionId(transactionId, startEntry.checksum(), commitEntry.getTimeWritten()));
                    }
                }
            }
        }
        if (hadAnyTransactions) {
            // No need to go further back than this. We're looking for the last transaction
            break;
        }
    }
    return Optional.empty();
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) IOCursor(org.neo4j.cursor.IOCursor) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) File(java.io.File) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) TransactionId(org.neo4j.kernel.impl.store.TransactionId)

Aggregations

LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)45 LogHeaderReader.readLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader)18 Test (org.junit.jupiter.api.Test)13 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)12 File (java.io.File)11 StoreChannel (org.neo4j.io.fs.StoreChannel)11 PhysicalLogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)11 LogHeaderWriter.writeLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader)10 ReadAheadLogChannel (org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel)9 Path (java.nio.file.Path)7 Test (org.junit.Test)6 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)6 LogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel)6 IOException (java.io.IOException)5 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)5 ReadableLogChannel (org.neo4j.kernel.impl.transaction.log.ReadableLogChannel)5 LogHeaderCache (org.neo4j.kernel.impl.transaction.log.LogHeaderCache)4 LogEntryStart (org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart)4 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)4 ByteBuffer (java.nio.ByteBuffer)3