Search in sources :

Example 46 with LogFiles

use of org.neo4j.kernel.impl.transaction.log.files.LogFiles in project neo4j by neo4j.

the class TransactionRangeDiagnosticsTest method logWithTransactionsInNextToOldestLog.

private static LogFiles logWithTransactionsInNextToOldestLog(long logVersion, long prevLogLastTxId) throws IOException {
    LogFiles files = logWithTransactions(logVersion, logVersion + 1, prevLogLastTxId);
    var logFile = files.getLogFile();
    when(logFile.hasAnyEntries(logVersion)).thenReturn(false);
    return files;
}
Also used : LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) TransactionLogFiles(org.neo4j.kernel.impl.transaction.log.files.TransactionLogFiles)

Example 47 with LogFiles

use of org.neo4j.kernel.impl.transaction.log.files.LogFiles in project neo4j by neo4j.

the class LogPruningTest method setUp.

@BeforeEach
void setUp() {
    fs = mock(FileSystemAbstraction.class);
    logFiles = mock(LogFiles.class);
    LogFile logFile = mock(LogFile.class);
    when(logFiles.getLogFile()).thenReturn(logFile);
    when(logFiles.getCheckpointFile()).thenReturn(mock(CheckpointFile.class));
    doAnswer(inv -> Path.of(String.valueOf(inv.getArguments()[0]))).when(logFile).getLogFileForVersion(anyLong());
    logProvider = new AssertableLogProvider();
    clock = mock(SystemNanoClock.class);
    factory = mock(LogPruneStrategyFactory.class);
}
Also used : LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) SystemNanoClock(org.neo4j.time.SystemNanoClock) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) CheckpointFile(org.neo4j.kernel.impl.transaction.log.files.checkpoint.CheckpointFile) AssertableLogProvider(org.neo4j.logging.AssertableLogProvider) BeforeEach(org.junit.jupiter.api.BeforeEach)

Example 48 with LogFiles

use of org.neo4j.kernel.impl.transaction.log.files.LogFiles in project neo4j by neo4j.

the class TransactionLogAppendAndRotateIT method shouldKeepTransactionsIntactWhenConcurrentlyRotationAndAppending.

@Test
void shouldKeepTransactionsIntactWhenConcurrentlyRotationAndAppending() throws Throwable {
    // GIVEN
    LogVersionRepository logVersionRepository = new SimpleLogVersionRepository();
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fileSystem).withLogVersionRepository(logVersionRepository).withRotationThreshold(ByteUnit.mebiBytes(1)).withTransactionIdStore(new SimpleTransactionIdStore()).withLogEntryReader(logEntryReader()).withStoreId(StoreId.UNKNOWN).build();
    life.add(logFiles);
    final AtomicBoolean end = new AtomicBoolean();
    AllTheMonitoring monitoring = new AllTheMonitoring(end, 100);
    TransactionIdStore txIdStore = new SimpleTransactionIdStore();
    TransactionMetadataCache metadataCache = new TransactionMetadataCache();
    monitoring.setLogFile(logFiles.getLogFile());
    Health health = new DatabaseHealth(mock(DatabasePanicEventGenerator.class), NullLog.getInstance());
    LogRotation rotation = transactionLogRotation(logFiles, Clock.systemUTC(), health, monitoring);
    final TransactionAppender appender = life.add(new BatchingTransactionAppender(logFiles, rotation, metadataCache, txIdStore, health));
    // WHEN
    Race race = new Race();
    for (int i = 0; i < 4; i++) {
        race.addContestant(() -> {
            while (!end.get()) {
                try {
                    appender.append(new TransactionToApply(sillyTransaction(1_000), CursorContext.NULL), LogAppendEvent.NULL);
                } catch (Exception e) {
                    e.printStackTrace(System.out);
                    end.set(true);
                    fail(e.getMessage(), e);
                }
            }
        });
    }
    race.addContestant(endAfterMax(250, MILLISECONDS, end, monitoring));
    race.go();
    // THEN
    assertTrue(monitoring.numberOfRotations() > 0);
}
Also used : DatabaseHealth(org.neo4j.monitoring.DatabaseHealth) TransactionToApply(org.neo4j.kernel.impl.api.TransactionToApply) TransactionIdStore(org.neo4j.storageengine.api.TransactionIdStore) SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) Health(org.neo4j.monitoring.Health) DatabaseHealth(org.neo4j.monitoring.DatabaseHealth) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) DatabasePanicEventGenerator(org.neo4j.kernel.monitoring.DatabasePanicEventGenerator) LogVersionRepository(org.neo4j.storageengine.api.LogVersionRepository) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) IOException(java.io.IOException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Race(org.neo4j.test.Race) LogRotation(org.neo4j.kernel.impl.transaction.log.rotation.LogRotation) FileLogRotation.transactionLogRotation(org.neo4j.kernel.impl.transaction.log.rotation.FileLogRotation.transactionLogRotation) Test(org.junit.jupiter.api.Test)

Example 49 with LogFiles

use of org.neo4j.kernel.impl.transaction.log.files.LogFiles in project neo4j by neo4j.

the class TransactionLogFileTest method skipLogFileWithoutHeader.

@Test
void skipLogFileWithoutHeader() throws IOException {
    LogFiles logFiles = buildLogFiles();
    life.add(logFiles);
    life.start();
    // simulate new file without header presence
    logVersionRepository.incrementAndGetVersion(NULL);
    fileSystem.write(logFiles.getLogFile().getLogFileForVersion(logVersionRepository.getCurrentLogVersion())).close();
    transactionIdStore.transactionCommitted(5L, 5, 5L, NULL);
    PhysicalLogicalTransactionStore.LogVersionLocator versionLocator = new PhysicalLogicalTransactionStore.LogVersionLocator(4L);
    logFiles.getLogFile().accept(versionLocator);
    LogPosition logPosition = versionLocator.getLogPosition();
    assertEquals(1, logPosition.getLogVersion());
}
Also used : LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) Test(org.junit.jupiter.api.Test)

Example 50 with LogFiles

use of org.neo4j.kernel.impl.transaction.log.files.LogFiles in project neo4j by neo4j.

the class TransactionLogFileTest method shouldCloseChannelInFailedAttemptToReadHeaderAfterOpen.

@Test
void shouldCloseChannelInFailedAttemptToReadHeaderAfterOpen() throws Exception {
    // GIVEN a file which returns 1/2 log header size worth of bytes
    FileSystemAbstraction fs = mock(FileSystemAbstraction.class);
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fs).withTransactionIdStore(transactionIdStore).withLogVersionRepository(logVersionRepository).withLogEntryReader(logEntryReader()).build();
    int logVersion = 0;
    Path logFile = logFiles.getLogFile().getLogFileForVersion(logVersion);
    StoreChannel channel = mock(StoreChannel.class);
    when(channel.read(any(ByteBuffer.class))).thenReturn(CURRENT_FORMAT_LOG_HEADER_SIZE / 2);
    when(fs.fileExists(logFile)).thenReturn(true);
    when(fs.read(eq(logFile))).thenReturn(channel);
    // WHEN
    assertThrows(IncompleteLogHeaderException.class, () -> logFiles.getLogFile().openForVersion(logVersion));
    verify(channel).close();
}
Also used : Path(java.nio.file.Path) DelegatingFileSystemAbstraction(org.neo4j.io.fs.DelegatingFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) StoreChannel(org.neo4j.io.fs.StoreChannel) DelegatingStoreChannel(org.neo4j.io.fs.DelegatingStoreChannel) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) ByteBuffer(java.nio.ByteBuffer) Test(org.junit.jupiter.api.Test)

Aggregations

LogFiles (org.neo4j.kernel.impl.transaction.log.files.LogFiles)61 Test (org.junit.jupiter.api.Test)33 LogFile (org.neo4j.kernel.impl.transaction.log.files.LogFile)21 SimpleTransactionIdStore (org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore)16 Path (java.nio.file.Path)13 IOException (java.io.IOException)10 SimpleLogVersionRepository (org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository)10 LogPosition (org.neo4j.kernel.impl.transaction.log.LogPosition)9 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)9 Lifespan (org.neo4j.kernel.lifecycle.Lifespan)8 TransactionIdStore (org.neo4j.storageengine.api.TransactionIdStore)8 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)7 StoreChannel (org.neo4j.io.fs.StoreChannel)7 DatabaseLayout (org.neo4j.io.layout.DatabaseLayout)7 LifeSupport (org.neo4j.kernel.lifecycle.LifeSupport)7 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)5 BeforeEach (org.junit.jupiter.api.BeforeEach)5 IndexingService (org.neo4j.kernel.impl.api.index.IndexingService)4 LogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader)4 UncheckedIOException (java.io.UncheckedIOException)3