Search in sources :

Example 1 with TestCommandReaderFactory

use of org.neo4j.kernel.impl.api.TestCommandReaderFactory in project neo4j by neo4j.

the class BatchingTransactionAppenderConcurrencyTest method databasePanicShouldHandleOutOfMemoryErrors.

@Test
void databasePanicShouldHandleOutOfMemoryErrors() throws IOException, InterruptedException {
    final CountDownLatch panicLatch = new CountDownLatch(1);
    final CountDownLatch adversaryLatch = new CountDownLatch(1);
    OutOfMemoryAwareFileSystem fs = new OutOfMemoryAwareFileSystem();
    life.add(new FileSystemLifecycleAdapter(fs));
    DatabaseHealth slowPanicDatabaseHealth = new SlowPanickingDatabaseHealth(panicLatch, adversaryLatch);
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fs).withLogVersionRepository(logVersionRepository).withTransactionIdStore(transactionIdStore).withDatabaseHealth(slowPanicDatabaseHealth).withLogEntryReader(new VersionAwareLogEntryReader(new TestCommandReaderFactory())).withStoreId(StoreId.UNKNOWN).build();
    life.add(logFiles);
    final BatchingTransactionAppender appender = life.add(new BatchingTransactionAppender(logFiles, logRotation, transactionMetadataCache, transactionIdStore, slowPanicDatabaseHealth));
    life.start();
    // Commit initial transaction
    appender.append(tx(), LogAppendEvent.NULL);
    // Try to commit one transaction, will fail during flush with OOM, but not actually panic
    fs.shouldOOM = true;
    Future<Long> failingTransaction = executor.submit(() -> appender.append(tx(), LogAppendEvent.NULL));
    panicLatch.await();
    // Try to commit one additional transaction, should fail since database has already panicked
    fs.shouldOOM = false;
    var e = assertThrows(IOException.class, () -> appender.append(tx(), new LogAppendEvent.Empty() {

        @Override
        public LogForceWaitEvent beginLogForceWait() {
            adversaryLatch.countDown();
            return super.beginLogForceWait();
        }
    }));
    assertThat(e).hasMessageContaining("The database has encountered a critical error");
    // Check that we actually got an OutOfMemoryError
    var executionException = assertThrows(ExecutionException.class, failingTransaction::get);
    assertThat(executionException).hasCauseInstanceOf(OutOfMemoryError.class);
    // Check number of transactions, should only have one
    LogEntryReader logEntryReader = new VersionAwareLogEntryReader(new TestCommandReaderFactory());
    LogFile logFile = logFiles.getLogFile();
    assertThat(logFile.getLowestLogVersion()).isEqualTo(logFile.getHighestLogVersion());
    long version = logFile.getHighestLogVersion();
    try (LogVersionedStoreChannel channel = logFile.openForVersion(version);
        ReadAheadLogChannel readAheadLogChannel = new ReadAheadLogChannel(channel, INSTANCE);
        LogEntryCursor cursor = new LogEntryCursor(logEntryReader, readAheadLogChannel)) {
        LogEntry entry;
        long numberOfTransactions = 0;
        while (cursor.next()) {
            entry = cursor.get();
            if (entry instanceof LogEntryCommit) {
                numberOfTransactions++;
            }
        }
        assertThat(numberOfTransactions).isEqualTo(1L);
    }
}
Also used : FileSystemLifecycleAdapter(org.neo4j.io.fs.FileSystemLifecycleAdapter) DatabaseHealth(org.neo4j.monitoring.DatabaseHealth) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) TransactionLogFiles(org.neo4j.kernel.impl.transaction.log.files.TransactionLogFiles) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) CountDownLatch(java.util.concurrent.CountDownLatch) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) TransactionLogFile(org.neo4j.kernel.impl.transaction.log.files.TransactionLogFile) LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) LogEntryCommit(org.neo4j.kernel.impl.transaction.log.entry.LogEntryCommit) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry) Test(org.junit.jupiter.api.Test)

Example 2 with TestCommandReaderFactory

use of org.neo4j.kernel.impl.api.TestCommandReaderFactory in project neo4j by neo4j.

the class BatchingTransactionAppenderConcurrencyTest method shouldHaveAllConcurrentAppendersSeePanic.

/*
     * There was an issue where if multiple concurrent appending threads did append and they moved on
     * to await a force, where the force would fail and the one doing the force would raise a panic...
     * the other threads may not notice the panic and move on to mark those transactions as committed
     * and notice the panic later (which would be too late).
     */
@Test
void shouldHaveAllConcurrentAppendersSeePanic() throws Throwable {
    // GIVEN
    Adversary adversary = new ClassGuardedAdversary(new CountingAdversary(1, true), failMethod(TransactionLogFile.class, "force"));
    EphemeralFileSystemAbstraction efs = new EphemeralFileSystemAbstraction();
    FileSystemAbstraction fs = new AdversarialFileSystemAbstraction(adversary, efs);
    life.add(new FileSystemLifecycleAdapter(fs));
    DatabaseHealth databaseHealth = new DatabaseHealth(mock(DatabasePanicEventGenerator.class), NullLog.getInstance());
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fs).withLogVersionRepository(logVersionRepository).withTransactionIdStore(transactionIdStore).withDatabaseHealth(databaseHealth).withLogEntryReader(new VersionAwareLogEntryReader(new TestCommandReaderFactory())).withStoreId(StoreId.UNKNOWN).build();
    life.add(logFiles);
    final BatchingTransactionAppender appender = life.add(new BatchingTransactionAppender(logFiles, logRotation, transactionMetadataCache, transactionIdStore, databaseHealth));
    life.start();
    // WHEN
    int numberOfAppenders = 10;
    final CountDownLatch trap = new CountDownLatch(numberOfAppenders);
    final LogAppendEvent beforeForceTrappingEvent = new LogAppendEvent.Empty() {

        @Override
        public LogForceWaitEvent beginLogForceWait() {
            trap.countDown();
            awaitLatch(trap);
            return super.beginLogForceWait();
        }
    };
    Race race = new Race();
    for (int i = 0; i < numberOfAppenders; i++) {
        race.addContestant(() -> {
            // Good, we know that this test uses an adversarial file system which will throw
            // an exception in LogFile#force, and since all these transactions
            // will append and be forced in the same batch, where the force will fail then
            // all these transactions should fail. If there's any transaction not failing then
            // it just didn't notice the panic, which would be potentially hazardous.
            assertThrows(IOException.class, () -> appender.append(tx(), beforeForceTrappingEvent));
        });
    }
    // THEN perform the race. The relevant assertions are made inside the contestants.
    race.go();
}
Also used : FileSystemLifecycleAdapter(org.neo4j.io.fs.FileSystemLifecycleAdapter) DatabaseHealth(org.neo4j.monitoring.DatabaseHealth) CountingAdversary(org.neo4j.adversaries.CountingAdversary) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.io.fs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) ClassGuardedAdversary(org.neo4j.adversaries.ClassGuardedAdversary) TransactionLogFile(org.neo4j.kernel.impl.transaction.log.files.TransactionLogFile) EphemeralFileSystemAbstraction(org.neo4j.io.fs.EphemeralFileSystemAbstraction) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) TransactionLogFiles(org.neo4j.kernel.impl.transaction.log.files.TransactionLogFiles) DatabasePanicEventGenerator(org.neo4j.kernel.monitoring.DatabasePanicEventGenerator) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) CountDownLatch(java.util.concurrent.CountDownLatch) LogAppendEvent(org.neo4j.kernel.impl.transaction.tracing.LogAppendEvent) Race(org.neo4j.test.Race) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) Adversary(org.neo4j.adversaries.Adversary) CountingAdversary(org.neo4j.adversaries.CountingAdversary) ClassGuardedAdversary(org.neo4j.adversaries.ClassGuardedAdversary) Test(org.junit.jupiter.api.Test)

Example 3 with TestCommandReaderFactory

use of org.neo4j.kernel.impl.api.TestCommandReaderFactory in project neo4j by neo4j.

the class TransactionLogFileRotateAndReadRaceIT method shouldNotSeeEmptyLogFileWhenReadingTransactionStream.

@Test
void shouldNotSeeEmptyLogFileWhenReadingTransactionStream() throws Exception {
    // GIVEN
    LogVersionRepository logVersionRepository = new SimpleLogVersionRepository();
    Config cfg = Config.newBuilder().set(GraphDatabaseSettings.neo4j_home, databaseLayout.getNeo4jLayout().homeDirectory()).set(GraphDatabaseSettings.preallocate_logical_logs, false).set(GraphDatabaseSettings.logical_log_rotation_threshold, ByteUnit.kibiBytes(128)).build();
    LogFiles logFiles = LogFilesBuilder.builder(databaseLayout, fs).withLogVersionRepository(logVersionRepository).withTransactionIdStore(new SimpleTransactionIdStore()).withLogEntryReader(new VersionAwareLogEntryReader(new TestCommandReaderFactory())).withConfig(cfg).withStoreId(StoreId.UNKNOWN).build();
    life.add(logFiles);
    LogFile logFile = logFiles.getLogFile();
    var writer = logFile.getTransactionLogWriter();
    LogPositionMarker startPosition = new LogPositionMarker();
    writer.getCurrentPosition(startPosition);
    // WHEN
    AtomicBoolean end = new AtomicBoolean();
    byte[] dataChunk = new byte[100];
    // one thread constantly writing to and rotating the channel
    CountDownLatch startSignal = new CountDownLatch(1);
    Future<Void> writeFuture = t2.execute(() -> {
        ThreadLocalRandom random = ThreadLocalRandom.current();
        startSignal.countDown();
        int rotations = 0;
        while (!end.get()) {
            int bytesToWrite = random.nextInt(1, dataChunk.length);
            writer.getChannel().put(dataChunk, bytesToWrite);
            if (logFile.rotationNeeded()) {
                logFile.rotate();
                // Let's just close the gap to the reader so that it gets closer to the "hot zone"
                // where the rotation happens.
                writer.getCurrentPosition(startPosition);
                if (++rotations > LIMIT_ROTATIONS) {
                    end.set(true);
                }
            }
        }
        return null;
    });
    assertTrue(startSignal.await(10, SECONDS));
    // one thread reading through the channel
    try {
        int reads = 0;
        while (!end.get()) {
            try (ReadableLogChannel reader = logFile.getReader(startPosition.newPosition())) {
                deplete(reader);
            }
            if (++reads > LIMIT_READS) {
                end.set(true);
            }
        }
    } finally {
        writeFuture.get();
    }
// THEN simply getting here means this was successful
}
Also used : SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) Config(org.neo4j.configuration.Config) LogFiles(org.neo4j.kernel.impl.transaction.log.files.LogFiles) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) CountDownLatch(java.util.concurrent.CountDownLatch) LogVersionRepository(org.neo4j.storageengine.api.LogVersionRepository) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) LogFile(org.neo4j.kernel.impl.transaction.log.files.LogFile) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) ThreadLocalRandom(java.util.concurrent.ThreadLocalRandom) Test(org.junit.jupiter.api.Test)

Example 4 with TestCommandReaderFactory

use of org.neo4j.kernel.impl.api.TestCommandReaderFactory in project neo4j by neo4j.

the class LogTestUtils method filterTransactionLogFile.

private static void filterTransactionLogFile(FileSystemAbstraction fileSystem, Path file, final LogHook<LogEntry> filter, ChannelNativeAccessor channelNativeAccessor) throws IOException {
    filter.file(file);
    try (StoreChannel in = fileSystem.read(file)) {
        LogHeader logHeader = readLogHeader(ByteBuffers.allocate(CURRENT_FORMAT_LOG_HEADER_SIZE, INSTANCE), in, true, file);
        assert logHeader != null : "Looks like we tried to read a log header of an empty pre-allocated file.";
        PhysicalLogVersionedStoreChannel inChannel = new PhysicalLogVersionedStoreChannel(in, logHeader.getLogVersion(), logHeader.getLogFormatVersion(), file, channelNativeAccessor);
        ReadableLogChannel inBuffer = new ReadAheadLogChannel(inChannel, INSTANCE);
        LogEntryReader entryReader = new VersionAwareLogEntryReader(new TestCommandReaderFactory());
        LogEntry entry;
        while ((entry = entryReader.readLogEntry(inBuffer)) != null) {
            filter.test(entry);
        }
    }
}
Also used : ReadableLogChannel(org.neo4j.kernel.impl.transaction.log.ReadableLogChannel) StoreChannel(org.neo4j.io.fs.StoreChannel) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) LogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) PhysicalLogVersionedStoreChannel(org.neo4j.kernel.impl.transaction.log.PhysicalLogVersionedStoreChannel) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory) ReadAheadLogChannel(org.neo4j.kernel.impl.transaction.log.ReadAheadLogChannel) LogHeaderReader.readLogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeaderReader.readLogHeader) LogHeader(org.neo4j.kernel.impl.transaction.log.entry.LogHeader) LogEntry(org.neo4j.kernel.impl.transaction.log.entry.LogEntry)

Example 5 with TestCommandReaderFactory

use of org.neo4j.kernel.impl.api.TestCommandReaderFactory in project neo4j by neo4j.

the class TransactionLogFilesTest method createLogFiles.

private LogFiles createLogFiles() throws Exception {
    var files = LogFilesBuilder.builder(databaseLayout, fileSystem).withTransactionIdStore(new SimpleTransactionIdStore()).withLogVersionRepository(new SimpleLogVersionRepository()).withLogEntryReader(new VersionAwareLogEntryReader(new TestCommandReaderFactory())).withStoreId(StoreId.UNKNOWN).build();
    files.init();
    return files;
}
Also used : SimpleTransactionIdStore(org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore) SimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository) VersionAwareLogEntryReader(org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader) TestCommandReaderFactory(org.neo4j.kernel.impl.api.TestCommandReaderFactory)

Aggregations

TestCommandReaderFactory (org.neo4j.kernel.impl.api.TestCommandReaderFactory)5 VersionAwareLogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.VersionAwareLogEntryReader)5 CountDownLatch (java.util.concurrent.CountDownLatch)3 Test (org.junit.jupiter.api.Test)3 LogFiles (org.neo4j.kernel.impl.transaction.log.files.LogFiles)3 FileSystemLifecycleAdapter (org.neo4j.io.fs.FileSystemLifecycleAdapter)2 SimpleLogVersionRepository (org.neo4j.kernel.impl.transaction.SimpleLogVersionRepository)2 SimpleTransactionIdStore (org.neo4j.kernel.impl.transaction.SimpleTransactionIdStore)2 LogEntry (org.neo4j.kernel.impl.transaction.log.entry.LogEntry)2 LogEntryReader (org.neo4j.kernel.impl.transaction.log.entry.LogEntryReader)2 LogFile (org.neo4j.kernel.impl.transaction.log.files.LogFile)2 TransactionLogFile (org.neo4j.kernel.impl.transaction.log.files.TransactionLogFile)2 TransactionLogFiles (org.neo4j.kernel.impl.transaction.log.files.TransactionLogFiles)2 DatabaseHealth (org.neo4j.monitoring.DatabaseHealth)2 ThreadLocalRandom (java.util.concurrent.ThreadLocalRandom)1 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)1 Adversary (org.neo4j.adversaries.Adversary)1 ClassGuardedAdversary (org.neo4j.adversaries.ClassGuardedAdversary)1 CountingAdversary (org.neo4j.adversaries.CountingAdversary)1 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)1