Search in sources :

Example 11 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class RecoveryTest method shouldTruncateLogAfterSinglePartialTransaction.

@Test
public void shouldTruncateLogAfterSinglePartialTransaction() throws Exception {
    // GIVEN
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    final LogPositionMarker marker = new LogPositionMarker();
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            // incomplete tx
            // <-- marker has the last good position
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 5L, 4L, new byte[0]);
            return true;
        }
    });
    // WHEN
    boolean recoveryRequired = recover(logFiles);
    // THEN
    assertTrue(recoveryRequired);
    assertEquals(marker.getByteOffset(), file.length());
}
Also used : Consumer(java.util.function.Consumer) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) Pair(org.neo4j.helpers.collection.Pair) Test(org.junit.Test)

Example 12 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class RecoveryTest method writeSomeData.

private void writeSomeData(File file, Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException> visitor) throws IOException {
    try (LogVersionedStoreChannel versionedStoreChannel = new PhysicalLogVersionedStoreChannel(fileSystemRule.get().open(file, "rw"), logVersion, CURRENT_LOG_VERSION);
        final PositionAwarePhysicalFlushableChannel writableLogChannel = new PositionAwarePhysicalFlushableChannel(versionedStoreChannel)) {
        writeLogHeader(writableLogChannel, logVersion, 2L);
        Consumer<LogPositionMarker> consumer = marker -> {
            try {
                writableLogChannel.getCurrentPosition(marker);
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        };
        LogEntryWriter first = new LogEntryWriter(writableLogChannel);
        visitor.visit(Pair.of(first, consumer));
    }
}
Also used : LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogHeaderWriter.writeLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderWriter.writeLogHeader) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) DeadSimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.DeadSimpleTransactionIdStore) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) Assert.fail(org.junit.Assert.fail) DefaultRecoverySPI(org.neo4j.kernel.recovery.DefaultRecoverySPI) BASE_TX_COMMIT_TIMESTAMP(org.neo4j.kernel.impl.transaction.log.TransactionIdStore.BASE_TX_COMMIT_TIMESTAMP) Recovery(org.neo4j.kernel.recovery.Recovery) StorageEngine(org.neo4j.storageengine.api.StorageEngine) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) CURRENT_LOG_VERSION(org.neo4j.kernel.impl.transaction.log.entry.LogVersions.CURRENT_LOG_VERSION) TestDirectory(org.neo4j.test.rule.TestDirectory) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogEntryStart(org.neo4j.kernel.impl.transaction.log.entry.LogEntryStart) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) Matchers.any(org.mockito.Matchers.any) CommittedTransactionRepresentation(org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation) Mockito.inOrder(org.mockito.Mockito.inOrder) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) Mockito.mock(org.mockito.Mockito.mock) PhysicalLogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore) TransactionIdStore(org.neo4j.kernel.impl.transaction.log.TransactionIdStore) LogicalTransactionStore(org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Pair(org.neo4j.helpers.collection.Pair) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) Mockito.verifyZeroInteractions(org.mockito.Mockito.verifyZeroInteractions) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) NO_MONITOR(org.neo4j.kernel.recovery.PositionToRecoverFrom.NO_MONITOR) InOrder(org.mockito.InOrder) CheckPoint(org.neo4j.kernel.impl.transaction.log.entry.CheckPoint) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) TransactionMetadataCache(org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) Mockito.times(org.mockito.Mockito.times) OnePhaseCommit(org.neo4j.kernel.impl.transaction.log.entry.OnePhaseCommit) LatestCheckPointFinder(org.neo4j.kernel.recovery.LatestCheckPointFinder) File(java.io.File) Consumer(java.util.function.Consumer) Rule(org.junit.Rule) DefaultFileSystemRule(org.neo4j.test.rule.fs.DefaultFileSystemRule) Visitor(org.neo4j.helpers.collection.Visitor) Assert.assertEquals(org.junit.Assert.assertEquals) LogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.LogVersionedStoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) PositionAwarePhysicalFlushableChannel(org.neo4j.kernel.impl.transaction.log.PositionAwarePhysicalFlushableChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker)

Example 13 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class RecoveryTest method shouldTruncateLogAfterLastCompleteTransactionAfterSuccessfullRecovery.

@Test
public void shouldTruncateLogAfterLastCompleteTransactionAfterSuccessfullRecovery() throws Exception {
    // GIVEN
    final PhysicalLogFiles logFiles = new PhysicalLogFiles(directory.directory(), "log", fileSystemRule.get());
    File file = logFiles.getLogFileForVersion(logVersion);
    final LogPositionMarker marker = new LogPositionMarker();
    writeSomeData(file, new Visitor<Pair<LogEntryWriter, Consumer<LogPositionMarker>>, IOException>() {

        @Override
        public boolean visit(Pair<LogEntryWriter, Consumer<LogPositionMarker>> pair) throws IOException {
            LogEntryWriter writer = pair.first();
            Consumer<LogPositionMarker> consumer = pair.other();
            // last committed tx
            writer.writeStartEntry(0, 1, 2L, 3L, new byte[0]);
            writer.writeCommitEntry(4L, 5L);
            // incomplete tx
            // <-- marker has the last good position
            consumer.accept(marker);
            writer.writeStartEntry(0, 1, 5L, 4L, new byte[0]);
            return true;
        }
    });
    // WHEN
    boolean recoveryRequired = recover(logFiles);
    // THEN
    assertTrue(recoveryRequired);
    assertEquals(marker.getByteOffset(), file.length());
}
Also used : Consumer(java.util.function.Consumer) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) IOException(java.io.IOException) LogFile(org.neo4j.kernel.impl.transaction.log.LogFile) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) File(java.io.File) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) Pair(org.neo4j.helpers.collection.Pair) Test(org.junit.Test)

Example 14 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class LatestCheckPointFinderTest method logFile.

private LogCreator logFile(Entry... entries) {
    return (logVersion, positions) -> {
        try {
            AtomicLong lastTxId = new AtomicLong();
            Supplier<Long> lastTxIdSupplier = () -> lastTxId.get();
            LogVersionRepository logVersionRepository = new DeadSimpleLogVersionRepository(logVersion);
            LifeSupport life = new LifeSupport();
            life.start();
            PhysicalLogFile logFile = life.add(new PhysicalLogFile(fsRule.get(), logFiles, mebiBytes(1), lastTxIdSupplier, logVersionRepository, NO_MONITOR, new LogHeaderCache(10)));
            try {
                FlushablePositionAwareChannel writeChannel = logFile.getWriter();
                LogPositionMarker positionMarker = new LogPositionMarker();
                LogEntryWriter writer = new LogEntryWriter(writeChannel);
                for (Entry entry : entries) {
                    LogPosition currentPosition = writeChannel.getCurrentPosition(positionMarker).newPosition();
                    positions.put(entry, currentPosition);
                    if (entry instanceof StartEntry) {
                        writer.writeStartEntry(0, 0, 0, 0, new byte[0]);
                    } else if (entry instanceof CommitEntry) {
                        CommitEntry commitEntry = (CommitEntry) entry;
                        writer.writeCommitEntry(commitEntry.txId, 0);
                        lastTxId.set(commitEntry.txId);
                    } else if (entry instanceof CheckPointEntry) {
                        CheckPointEntry checkPointEntry = (CheckPointEntry) entry;
                        Entry target = checkPointEntry.withPositionOfEntry;
                        LogPosition logPosition = target != null ? positions.get(target) : currentPosition;
                        assert logPosition != null : "No registered log position for " + target;
                        writer.writeCheckPointEntry(logPosition);
                    } else if (entry instanceof PositionEntry) {
                    // Don't write anything, this entry is just for registering a position so that
                    // another CheckPointEntry can refer to it
                    } else {
                        throw new IllegalArgumentException("Unknown entry " + entry);
                    }
                }
            } finally {
                life.shutdown();
            }
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    };
}
Also used : ByteUnit.mebiBytes(org.neo4j.io.ByteUnit.mebiBytes) Arrays(java.util.Arrays) EphemeralFileSystemRule(org.neo4j.test.rule.fs.EphemeralFileSystemRule) RunWith(org.junit.runner.RunWith) HashMap(java.util.HashMap) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Supplier(java.util.function.Supplier) ReadableClosablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.ReadableClosablePositionAwareChannel) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) NO_MONITOR(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile.NO_MONITOR) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) LatestCheckPoint(org.neo4j.kernel.recovery.LatestCheckPointFinder.LatestCheckPoint) Map(java.util.Map) PhysicalLogFiles(org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles) Parameterized(org.junit.runners.Parameterized) NO_TRANSACTION_ID(org.neo4j.kernel.recovery.LatestCheckPointFinder.LatestCheckPoint.NO_TRANSACTION_ID) Before(org.junit.Before) Collection(java.util.Collection) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) Test(org.junit.Test) IOException(java.io.IOException) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) File(java.io.File) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) UncheckedIOException(java.io.UncheckedIOException) AtomicLong(java.util.concurrent.atomic.AtomicLong) Rule(org.junit.Rule) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) FlushablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.FlushablePositionAwareChannel) Assert.assertEquals(org.junit.Assert.assertEquals) UncheckedIOException(java.io.UncheckedIOException) IOException(java.io.IOException) UncheckedIOException(java.io.UncheckedIOException) LogVersionRepository(org.neo4j.kernel.impl.transaction.log.LogVersionRepository) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) AtomicLong(java.util.concurrent.atomic.AtomicLong) LifeSupport(org.neo4j.kernel.lifecycle.LifeSupport) Supplier(java.util.function.Supplier) LogEntryWriter(org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter) LogHeaderCache(org.neo4j.kernel.impl.transaction.log.LogHeaderCache) PhysicalLogFile(org.neo4j.kernel.impl.transaction.log.PhysicalLogFile) FlushablePositionAwareChannel(org.neo4j.kernel.impl.transaction.log.FlushablePositionAwareChannel) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition)

Example 15 with LogPositionMarker

use of org.neo4j.kernel.impl.transaction.log.LogPositionMarker in project neo4j by neo4j.

the class LogPositionMarkerTest method shouldReturnUnspecifiedIfNothingHasBeenMarked.

@Test
void shouldReturnUnspecifiedIfNothingHasBeenMarked() {
    // given
    final LogPositionMarker marker = new LogPositionMarker();
    // when
    final LogPosition logPosition = marker.newPosition();
    // given
    assertEquals(LogPosition.UNSPECIFIED, logPosition);
}
Also used : LogPositionMarker(org.neo4j.kernel.impl.transaction.log.LogPositionMarker) LogPosition(org.neo4j.kernel.impl.transaction.log.LogPosition) Test(org.junit.jupiter.api.Test)

Aggregations

LogPositionMarker (org.neo4j.kernel.impl.transaction.log.LogPositionMarker)22 LogEntryWriter (org.neo4j.kernel.impl.transaction.log.entry.LogEntryWriter)16 Test (org.junit.jupiter.api.Test)12 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)12 IOException (java.io.IOException)10 Path (java.nio.file.Path)9 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)9 File (java.io.File)7 Consumer (java.util.function.Consumer)7 Test (org.junit.Test)7 PhysicalLogFile (org.neo4j.kernel.impl.transaction.log.PhysicalLogFile)7 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)7 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)7 Pair (org.neo4j.helpers.collection.Pair)6 LogFile (org.neo4j.kernel.impl.transaction.log.LogFile)6 LogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.LogicalTransactionStore)6 PhysicalLogicalTransactionStore (org.neo4j.kernel.impl.transaction.log.PhysicalLogicalTransactionStore)6 TransactionMetadataCache (org.neo4j.kernel.impl.transaction.log.TransactionMetadataCache)6 StorageEngine (org.neo4j.storageengine.api.StorageEngine)6 CommittedTransactionRepresentation (org.neo4j.kernel.impl.transaction.CommittedTransactionRepresentation)5