Search in sources :

Example 1 with FileSystemLifecycleAdapter

use of org.neo4j.io.fs.FileSystemLifecycleAdapter in project neo4j by neo4j.

the class BatchingTransactionAppenderConcurrencyTest method shouldHaveAllConcurrentAppendersSeePanic.

/*
     * There was an issue where if multiple concurrent appending threads did append and they moved on
     * to await a force, where the force would fail and the one doing the force would raise a panic...
     * the other threads may not notice the panic and move on to mark those transactions as committed
     * and notice the panic later (which would be too late).
     */
@Test
public void shouldHaveAllConcurrentAppendersSeePanic() throws Throwable {
    // GIVEN
    Adversary adversary = new ClassGuardedAdversary(new CountingAdversary(1, true), failMethod(BatchingTransactionAppender.class, "force"));
    EphemeralFileSystemAbstraction efs = new EphemeralFileSystemAbstraction();
    File directory = new File("dir").getCanonicalFile();
    efs.mkdirs(directory);
    FileSystemAbstraction fs = new AdversarialFileSystemAbstraction(adversary, efs);
    life.add(new FileSystemLifecycleAdapter(fs));
    DatabaseHealth databaseHealth = new DatabaseHealth(mock(DatabasePanicEventGenerator.class), NullLog.getInstance());
    PhysicalLogFiles logFiles = new PhysicalLogFiles(directory, fs);
    LogFile logFile = life.add(new PhysicalLogFile(fs, logFiles, kibiBytes(10), transactionIdStore::getLastCommittedTransactionId, new DeadSimpleLogVersionRepository(0), new PhysicalLogFile.Monitor.Adapter(), logHeaderCache));
    final BatchingTransactionAppender appender = life.add(new BatchingTransactionAppender(logFile, logRotation, transactionMetadataCache, transactionIdStore, legacyIndexTransactionOrdering, databaseHealth));
    life.start();
    // WHEN
    int numberOfAppenders = 10;
    final CountDownLatch trap = new CountDownLatch(numberOfAppenders);
    final LogAppendEvent beforeForceTrappingEvent = new LogAppendEvent.Empty() {

        @Override
        public LogForceWaitEvent beginLogForceWait() {
            trap.countDown();
            awaitLatch(trap);
            return super.beginLogForceWait();
        }
    };
    Race race = new Race();
    for (int i = 0; i < numberOfAppenders; i++) {
        race.addContestant(() -> {
            try {
                // Append to the log, the LogAppenderEvent will have all of the appending threads
                // do wait for all of the other threads to start the force thing
                appender.append(tx(), beforeForceTrappingEvent);
                fail("No transaction should be considered appended");
            } catch (IOException e) {
            // Good, we know that this test uses an adversarial file system which will throw
            // an exception in BatchingTransactionAppender#force, and since all these transactions
            // will append and be forced in the same batch, where the force will fail then
            // all these transactions should fail. If there's any transaction not failing then
            // it just didn't notice the panic, which would be potentially hazardous.
            }
        });
    }
    // THEN perform the race. The relevant assertions are made inside the contestants.
    race.go();
}
Also used : FileSystemLifecycleAdapter(org.neo4j.io.fs.FileSystemLifecycleAdapter) DatabaseHealth(org.neo4j.kernel.internal.DatabaseHealth) CountingAdversary(org.neo4j.adversaries.CountingAdversary) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) ClassGuardedAdversary(org.neo4j.adversaries.ClassGuardedAdversary) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) DatabasePanicEventGenerator(org.neo4j.kernel.impl.core.DatabasePanicEventGenerator) FileSystemLifecycleAdapter(org.neo4j.io.fs.FileSystemLifecycleAdapter) IOException(java.io.IOException) CountDownLatch(java.util.concurrent.CountDownLatch) LogAppendEvent(org.neo4j.kernel.impl.transaction.tracing.LogAppendEvent) DeadSimpleLogVersionRepository(org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository) Race(org.neo4j.test.Race) File(java.io.File) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) Adversary(org.neo4j.adversaries.Adversary) CountingAdversary(org.neo4j.adversaries.CountingAdversary) ClassGuardedAdversary(org.neo4j.adversaries.ClassGuardedAdversary) Test(org.junit.Test)

Example 2 with FileSystemLifecycleAdapter

use of org.neo4j.io.fs.FileSystemLifecycleAdapter in project neo4j by neo4j.

the class ArbiterBootstrapper method start.

@SafeVarargs
@Override
public final int start(File homeDir, Optional<File> configFile, Pair<String, String>... configOverrides) {
    Config config = getConfig(configFile, configOverrides);
    try {
        DefaultFileSystemAbstraction fileSystem = new DefaultFileSystemAbstraction();
        life.add(new FileSystemLifecycleAdapter(fileSystem));
        life.add(new Neo4jJobScheduler());
        new ClusterClientModule(life, new Dependencies(), new Monitors(), config, logService(fileSystem, config), new NotElectableElectionCredentialsProvider());
    } catch (LifecycleException e) {
        @SuppressWarnings({ "ThrowableResultOfMethodCallIgnored", "unchecked" }) Throwable cause = peel(e, Predicates.<Throwable>instanceOf(LifecycleException.class));
        if (cause instanceof ChannelException) {
            System.err.println("ERROR: " + cause.getMessage() + (cause.getCause() != null ? ", caused by:" + cause.getCause().getMessage() : ""));
        } else {
            System.err.println("ERROR: Unknown error");
            throw e;
        }
    }
    addShutdownHook();
    life.start();
    return 0;
}
Also used : FileSystemLifecycleAdapter(org.neo4j.io.fs.FileSystemLifecycleAdapter) Neo4jJobScheduler(org.neo4j.kernel.impl.util.Neo4jJobScheduler) LifecycleException(org.neo4j.kernel.lifecycle.LifecycleException) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) Config(org.neo4j.kernel.configuration.Config) NotElectableElectionCredentialsProvider(org.neo4j.cluster.protocol.election.NotElectableElectionCredentialsProvider) Monitors(org.neo4j.kernel.monitoring.Monitors) Dependencies(org.neo4j.kernel.impl.util.Dependencies) ClusterClientModule(org.neo4j.cluster.client.ClusterClientModule) ChannelException(org.jboss.netty.channel.ChannelException)

Aggregations

FileSystemLifecycleAdapter (org.neo4j.io.fs.FileSystemLifecycleAdapter)2 File (java.io.File)1 IOException (java.io.IOException)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 ChannelException (org.jboss.netty.channel.ChannelException)1 Test (org.junit.Test)1 Adversary (org.neo4j.adversaries.Adversary)1 ClassGuardedAdversary (org.neo4j.adversaries.ClassGuardedAdversary)1 CountingAdversary (org.neo4j.adversaries.CountingAdversary)1 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)1 ClusterClientModule (org.neo4j.cluster.client.ClusterClientModule)1 NotElectableElectionCredentialsProvider (org.neo4j.cluster.protocol.election.NotElectableElectionCredentialsProvider)1 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)1 DefaultFileSystemAbstraction (org.neo4j.io.fs.DefaultFileSystemAbstraction)1 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)1 Config (org.neo4j.kernel.configuration.Config)1 DatabasePanicEventGenerator (org.neo4j.kernel.impl.core.DatabasePanicEventGenerator)1 DeadSimpleLogVersionRepository (org.neo4j.kernel.impl.transaction.DeadSimpleLogVersionRepository)1 LogAppendEvent (org.neo4j.kernel.impl.transaction.tracing.LogAppendEvent)1 Dependencies (org.neo4j.kernel.impl.util.Dependencies)1