Search in sources :

Example 56 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class LogEntryParserDispatcherV6Test method failToParseCheckpointWithLatestFormat.

@Test
void failToParseCheckpointWithLatestFormat() {
    var checkPoint = new LogEntryInlinedCheckPoint(version, new LogPosition(43, 44));
    var channel = new InMemoryClosableChannel();
    channel.putLong(checkPoint.getLogPosition().getLogVersion());
    channel.putLong(checkPoint.getLogPosition().getByteOffset());
    channel.putChecksum();
    channel.getCurrentPosition(marker);
    assertThrows(Exception.class, () -> parserSet(LATEST).select(LEGACY_CHECK_POINT));
}
Also used : InMemoryClosableChannel(org.neo4j.kernel.impl.transaction.log.InMemoryClosableChannel) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.jupiter.api.Test)

Example 57 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class VersionAwareLogEntryReader method readLogEntry.

@Override
public LogEntry readLogEntry(ReadableClosablePositionAwareChecksumChannel channel) throws IOException {
    try {
        while (true) {
            channel.getCurrentPosition(positionMarker);
            byte versionCode = channel.get();
            if (versionCode == 0) {
                // and we report that we reach end of record stream from our point of view
                if (channel instanceof PositionableChannel) {
                    resetChannelPosition(channel);
                } else {
                    throw new IllegalStateException("Log reader expects positionable channel to be able to reset offset. Current channel: " + channel);
                }
                return null;
            }
            if (parserSet == null || parserSet.getIntroductionVersion().version() != versionCode) {
                try {
                    parserSet = LogEntryParserSets.parserSet(KernelVersion.getForVersion(versionCode));
                } catch (IllegalArgumentException e) {
                    throw new UnsupportedLogVersionException(String.format("Log file contains entries with prefix %d, and the lowest supported prefix is %s. This " + "indicates that the log files originates from an older version of neo4j, which we don't support " + "migrations from.", versionCode, KernelVersion.V2_3));
                }
                // a new checksum segment if we change version parser.
                if (channel instanceof PositionableChannel) {
                    resetChannelPosition(channel);
                    channel.beginChecksum();
                    channel.get();
                }
            }
            byte typeCode = channel.get();
            LogEntryParser entryReader;
            LogEntry entry;
            try {
                entryReader = parserSet.select(typeCode);
                entry = entryReader.parse(parserSet.getIntroductionVersion(), channel, positionMarker, commandReaderFactory);
            } catch (ReadPastEndException e) {
                // Make these exceptions slip by straight out to the outer handler
                throw e;
            } catch (Exception e) {
                // Tag all other exceptions with log position and other useful information
                LogPosition position = positionMarker.newPosition();
                withMessage(e, e.getMessage() + ". At position " + position + " and entry version " + versionCode);
                throwIfInstanceOf(e, UnsupportedLogVersionException.class);
                throw new IOException(e);
            }
            verifyChecksumChain(entry);
            return entry;
        }
    } catch (ReadPastEndException e) {
        return null;
    }
}
Also used : PositionableChannel(org.neo4j.io.fs.PositionableChannel) IOException(java.io.IOException) ReadPastEndException(org.neo4j.io.fs.ReadPastEndException) IOException(java.io.IOException) ReadPastEndException(org.neo4j.io.fs.ReadPastEndException) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 58 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class DetachedCheckpointLogEntryWriterTest method writeCheckpoint.

private static void writeCheckpoint(DetachedCheckpointLogEntryWriter checkpointLogEntryWriter, String reason) throws IOException {
    var storeId = new StoreId(3, 4, 5, 6, 7);
    LogPosition logPosition = new LogPosition(1, 2);
    checkpointLogEntryWriter.writeCheckPointEntry(logPosition, Instant.ofEpochMilli(1), storeId, reason);
}
Also used : StoreId(org.neo4j.storageengine.api.StoreId) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 59 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class DetachedCheckpointLogEntryParser method parse.

@Override
LogEntry parse(KernelVersion version, ReadableChecksumChannel channel, LogPositionMarker marker, CommandReaderFactory commandReaderFactory) throws IOException {
    long logVersion = channel.getLong();
    long byteOffset = channel.getLong();
    long checkpointTimeMillis = channel.getLong();
    StoreId storeId = new StoreId(channel.getLong(), channel.getLong(), channel.getLong(), channel.getLong(), channel.getLong());
    short reasonBytesLength = channel.getShort();
    byte[] bytes = new byte[MAX_DESCRIPTION_LENGTH];
    channel.get(bytes, MAX_DESCRIPTION_LENGTH);
    String reason = new String(bytes, 0, reasonBytesLength, UTF_8);
    channel.endChecksumAndValidate();
    return new LogEntryDetachedCheckpoint(version, new LogPosition(logVersion, byteOffset), checkpointTimeMillis, storeId, reason);
}
Also used : StoreId(org.neo4j.storageengine.api.StoreId) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 60 with LogPosition

use of org.neo4j.kernel.impl.transaction.log.LogPosition in project neo4j by neo4j.

the class CheckTxLogsTest method shouldDetectAnInconsistentCheckPointPointingToALogFileGreaterThanMaxLogVersion.

@Test
public void shouldDetectAnInconsistentCheckPointPointingToALogFileGreaterThanMaxLogVersion() throws Exception {
    // given
    File log = logFile(1);
    writeCheckPoint(log, 2, LOG_HEADER_SIZE);
    CapturingInconsistenciesHandler handler = new CapturingInconsistenciesHandler();
    CheckTxLogs checker = new CheckTxLogs(System.out, fsRule.get());
    // when
    checker.validateCheckPoints(new PhysicalLogFiles(storeDirectory, fsRule.get()), handler);
    // then
    assertEquals(1, handler.checkPointInconsistencies.size());
    assertEquals(1, handler.checkPointInconsistencies.get(0).logVersion);
    assertEquals(new LogPosition(2, LOG_HEADER_SIZE), handler.checkPointInconsistencies.get(0).logPosition);
    assertThat(handler.checkPointInconsistencies.get(0).size, lessThan(0L));
}
Also used : File(java.io.File) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.Test)

Aggregations

LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)115 Test (org.junit.jupiter.api.Test)50 File (java.io.File)16 Test (org.junit.Test)16 LogFile (org.neo4j.kernel.impl.transaction.log.files.LogFile)15 Path (java.nio.file.Path)14 IOException (java.io.IOException)11 LogPositionMarker (org.neo4j.kernel.impl.transaction.log.LogPositionMarker)10 LogFiles (org.neo4j.kernel.impl.transaction.log.files.LogFiles)10 CheckpointFile (org.neo4j.kernel.impl.transaction.log.files.checkpoint.CheckpointFile)10 StoreId (org.neo4j.storageengine.api.StoreId)10 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)9 InMemoryClosableChannel (org.neo4j.kernel.impl.transaction.log.InMemoryClosableChannel)8 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)8 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)8 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)8 LogTailInformation (org.neo4j.kernel.impl.transaction.log.files.LogTailInformation)8 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)7 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)7 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)7