Search in sources :

Example 6 with ReadAheadLogChannel

use of org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel in project neo4j by neo4j.

the class LogMatchers method logEntries.

public static List<LogEntry> logEntries(FileSystemAbstraction fileSystem, String logPath) throws IOException {
    File logFile = new File(logPath);
    StoreChannel fileChannel = fileSystem.open(logFile, "r");
    // Always a header
    LogHeader header = readLogHeader(ByteBuffer.allocateDirect(LOG_HEADER_SIZE), fileChannel, true, logFile);
    // Read all log entries
    PhysicalLogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(fileChannel, header.logVersion, header.logFormatVersion);
    ReadableLogChannel logChannel = new ReadAheadLogChannel(versionedStoreChannel, NO_MORE_CHANNELS);
    LogEntryCursor logEntryCursor = new LogEntryCursor(new VersionAwareLogEntryReader<>(), logChannel);
    return Iterables.asList(new IOCursorAsResourceIterable<>(logEntryCursor));
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) File(java.io.File) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader)

Example 7 with ReadAheadLogChannel

use of org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel in project neo4j by neo4j.

the class LegacyLogEntryReader method openReadableChannel.

public Pair<LogHeader, IOCursor<LogEntry>> openReadableChannel(File logFile) throws IOException {
    final StoreChannel rawChannel = fs.open(logFile, "r");
    final LogHeader header = readLogHeader(ByteBuffer.allocate(LOG_HEADER_SIZE), rawChannel, false, logFile);
    LogEntryReader<ReadableLogChannel> reader = readerFactory.apply(header);
    // this ensures that the last committed txId field in the header is initialized properly
    long lastCommittedTxId = Math.max(BASE_TX_ID, header.lastCommittedTxId);
    final PhysicalLogVersionedStoreChannel channel = new PhysicalLogVersionedStoreChannel(rawChannel, header.logVersion, header.logFormatVersion);
    final ReadableLogChannel readableChannel = new ReadAheadLogChannel(channel, NO_MORE_CHANNELS);
    final IOCursor<LogEntry> cursor = new LogEntrySortingCursor(reader, readableChannel);
    return Pair.of(new LogHeader(CURRENT_LOG_VERSION, header.logVersion, lastCommittedTxId), cursor);
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 8 with ReadAheadLogChannel

use of org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel in project neo4j by neo4j.

the class LatestCheckPointFinder method find.

public LatestCheckPoint find(long fromVersionBackwards) throws IOException {
    long version = fromVersionBackwards;
    long versionToSearchForCommits = fromVersionBackwards;
    LogEntryStart latestStartEntry = null;
    LogEntryStart oldestStartEntry = null;
    long oldestVersionFound = -1;
    while (version >= INITIAL_LOG_VERSION) {
        LogVersionedStoreChannel channel = PhysicalLogFile.tryOpenForVersion(logFiles, fileSystem, version, false);
        if (channel == null) {
            break;
        }
        oldestVersionFound = version;
        CheckPoint latestCheckPoint = null;
        ReadableLogChannel recoveredDataChannel = new ReadAheadLogChannel(channel, NO_MORE_CHANNELS);
        boolean firstStartEntry = true;
        try (LogEntryCursor cursor = new LogEntryCursor(logEntryReader, recoveredDataChannel)) {
            LogEntry entry;
            while (cursor.next()) {
                entry = cursor.get();
                if (entry instanceof CheckPoint) {
                    latestCheckPoint = entry.as();
                }
                if (entry instanceof LogEntryStart) {
                    LogEntryStart startEntry = entry.as();
                    if (version == versionToSearchForCommits) {
                        latestStartEntry = startEntry;
                    }
                    // Oldest start entry will be the first in the last log version scanned.
                    if (firstStartEntry) {
                        oldestStartEntry = startEntry;
                        firstStartEntry = false;
                    }
                }
            }
        }
        if (latestCheckPoint != null) {
            return latestCheckPoint(fromVersionBackwards, version, latestStartEntry, oldestVersionFound, latestCheckPoint);
        }
        version--;
        // if we have found no commits in the latest log, keep searching in the next one
        if (latestStartEntry == null) {
            versionToSearchForCommits--;
        }
    }
    boolean commitsAfterCheckPoint = oldestStartEntry != null;
    long firstTxAfterPosition = commitsAfterCheckPoint ? extractFirstTxIdAfterPosition(oldestStartEntry.getStartPosition(), fromVersionBackwards) : LatestCheckPoint.NO_TRANSACTION_ID;
    return new LatestCheckPoint(null, commitsAfterCheckPoint, firstTxAfterPosition, oldestVersionFound);
}
Also used : LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint)

Example 9 with ReadAheadLogChannel

use of org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel in project neo4j by neo4j.

the class LatestCheckPointFinder method extractFirstTxIdAfterPosition.

/**
     * Extracts txId from first commit entry, when starting reading at the given {@code position}.
     * If no commit entry found in the version, the reader will continue into next version(s) up till
     * {@code maxLogVersion} until finding one.
     *
     * @param initialPosition {@link LogPosition} to start scan from.
     * @param maxLogVersion max log version to scan.
     * @return txId of closes commit entry to {@code initialPosition}, or {@link LatestCheckPoint#NO_TRANSACTION_ID}
     * if not found.
     * @throws IOException on I/O error.
     */
private long extractFirstTxIdAfterPosition(LogPosition initialPosition, long maxLogVersion) throws IOException {
    LogPosition currentPosition = initialPosition;
    while (currentPosition.getLogVersion() <= maxLogVersion) {
        LogVersionedStoreChannel storeChannel = PhysicalLogFile.tryOpenForVersion(logFiles, fileSystem, currentPosition.getLogVersion(), false);
        if (storeChannel != null) {
            try {
                storeChannel.position(currentPosition.getByteOffset());
                try (ReadAheadLogChannel logChannel = new ReadAheadLogChannel(storeChannel, NO_MORE_CHANNELS);
                    LogEntryCursor cursor = new LogEntryCursor(logEntryReader, logChannel)) {
                    while (cursor.next()) {
                        LogEntry entry = cursor.get();
                        if (entry instanceof LogEntryCommit) {
                            return ((LogEntryCommit) entry).getTxId();
                        }
                    }
                }
            } finally {
                storeChannel.close();
            }
        }
        currentPosition = LogPosition.start(currentPosition.getLogVersion() + 1);
    }
    return LatestCheckPoint.NO_TRANSACTION_ID;
}
Also used : LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogEntryCursor(org.neo4j.kernel.impl.transaction.log.LogEntryCursor) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 10 with ReadAheadLogChannel

use of org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel in project neo4j by neo4j.

the class TransactionLogCatchUpWriterTest method verifyTransactionsInLog.

private void verifyTransactionsInLog(long fromTxId, long endTxId) throws IOException {
    long expectedTxId = fromTxId;
    PhysicalLogFiles logFiles = new PhysicalLogFiles(storeDir, fs);
    LogVersionedStoreChannel versionedStoreChannel = PhysicalLogFile.openForVersion(logFiles, fs, 0, false);
    try (ReadableLogChannel channel = new ReadAheadLogChannel(versionedStoreChannel, LogVersionBridge.NO_MORE_CHANNELS, 1024)) {
        try (PhysicalTransactionCursor<ReadableLogChannel> txCursor = new PhysicalTransactionCursor<>(channel, new VersionAwareLogEntryReader<>())) {
            while (txCursor.next()) {
                CommittedTransactionRepresentation tx = txCursor.get();
                long txId = tx.getCommitEntry().getTxId();
                assertThat(expectedTxId, lessThanOrEqualTo(endTxId));
                assertEquals(expectedTxId, txId);
                expectedTxId++;
            }
        }
    }
}
Also used : PhysicalTransactionCursor(org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor) ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)

Aggregations

ReadAheadLogChannel (org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel)12 StoreChannel (org.neo4j.io.fs.StoreChannel)8 PhysicalLogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel)8 ReadableLogChannel (org.neo4j.kernel.impl.transaction.log.ReadableLogChannel)8 LogEntryCursor (org.neo4j.kernel.impl.transaction.log.LogEntryCursor)6 LogVersionedStoreChannel (org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel)6 LogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeader)6 LogHeaderReader.readLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader)6 File (java.io.File)5 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)5 IOException (java.io.IOException)4 ByteBuffer (java.nio.ByteBuffer)4 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)4 ReadableClosablePositionAwareChannel (org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel)3 Test (org.junit.Test)2 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)2 LogVersionBridge (org.neo4j.kernel.impl.transaction.log.LogVersionBridge)2 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)2 PhysicalTransactionCursor (org.neo4j.kernel.impl.transaction.log.PhysicalTransactionCursor)2 LogHeaderWriter.writeLogHeader (org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader)2