Search in sources :

Example 36 with EphemeralFileSystemAbstraction

use of org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction in project neo4j by neo4j.

the class ReplicatedLockTokenStateMachineTest method shouldPersistAndRecoverState.

@Test
public void shouldPersistAndRecoverState() throws Exception {
    // given
    EphemeralFileSystemAbstraction fsa = fileSystemRule.get();
    fsa.mkdir(testDir.directory());
    StateMarshal<ReplicatedLockTokenState> marshal = new ReplicatedLockTokenState.Marshal(new MemberId.Marshal());
    MemberId memberA = member(0);
    MemberId memberB = member(1);
    int candidateId;
    DurableStateStorage<ReplicatedLockTokenState> storage = new DurableStateStorage<>(fsa, testDir.directory(), "state", marshal, 100, NullLogProvider.getInstance());
    try (Lifespan lifespan = new Lifespan(storage)) {
        ReplicatedLockTokenStateMachine stateMachine = new ReplicatedLockTokenStateMachine(storage);
        // when
        candidateId = 0;
        stateMachine.applyCommand(new ReplicatedLockTokenRequest(memberA, candidateId), 0, r -> {
        });
        candidateId = 1;
        stateMachine.applyCommand(new ReplicatedLockTokenRequest(memberB, candidateId), 1, r -> {
        });
        stateMachine.flush();
        fsa.crash();
    }
    // then
    DurableStateStorage<ReplicatedLockTokenState> storage2 = new DurableStateStorage<>(fsa, testDir.directory(), "state", marshal, 100, NullLogProvider.getInstance());
    try (Lifespan lifespan = new Lifespan(storage2)) {
        ReplicatedLockTokenState initialState = storage2.getInitialState();
        assertEquals(memberB, initialState.get().owner());
        assertEquals(candidateId, initialState.get().id());
    }
}
Also used : MemberId(org.neo4j.causalclustering.identity.MemberId) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) StateMarshal(org.neo4j.causalclustering.core.state.storage.StateMarshal) DurableStateStorage(org.neo4j.causalclustering.core.state.storage.DurableStateStorage) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Example 37 with EphemeralFileSystemAbstraction

use of org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction in project neo4j by neo4j.

the class ReplicatedLockTokenStateMachineTest method shouldBeIdempotent.

@Test
public void shouldBeIdempotent() throws Exception {
    // given
    EphemeralFileSystemAbstraction fsa = fileSystemRule.get();
    fsa.mkdir(testDir.directory());
    StateMarshal<ReplicatedLockTokenState> marshal = new ReplicatedLockTokenState.Marshal(new MemberId.Marshal());
    DurableStateStorage<ReplicatedLockTokenState> storage = new DurableStateStorage<>(fsa, testDir.directory(), "state", marshal, 100, NullLogProvider.getInstance());
    try (Lifespan lifespan = new Lifespan(storage)) {
        ReplicatedLockTokenStateMachine stateMachine = new ReplicatedLockTokenStateMachine(storage);
        MemberId memberA = member(0);
        MemberId memberB = member(1);
        stateMachine.applyCommand(new ReplicatedLockTokenRequest(memberA, 0), 3, r -> {
        });
        // when
        stateMachine.applyCommand(new ReplicatedLockTokenRequest(memberB, 1), 2, r -> {
        });
        // then
        assertEquals(memberA, stateMachine.currentToken().owner());
    }
}
Also used : MemberId(org.neo4j.causalclustering.identity.MemberId) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) StateMarshal(org.neo4j.causalclustering.core.state.storage.StateMarshal) DurableStateStorage(org.neo4j.causalclustering.core.state.storage.DurableStateStorage) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Example 38 with EphemeralFileSystemAbstraction

use of org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction in project neo4j by neo4j.

the class DurableStateStorageIT method shouldProperlyRecoveryAfterCrashOnFileForceDuringWrite.

@Test
public void shouldProperlyRecoveryAfterCrashOnFileForceDuringWrite() throws Exception {
    EphemeralFileSystemAbstraction normalFSA = fileSystemRule.get();
    /*
         * Magic number warning. For a rotation threshold of 14, 990 operations on file A falls on a force() of the
         * current active file. This has been discovered via experimentation. The end result is that there is a
         * flush (but not write) a value. This should be recoverable. Interestingly, the failure semantics are a bit
         * unclear on what should happen to that value. We assume that exception during persistence requires recovery
         * to discover if the last argument made it to disk or not. Since we use an EFSA, force is not necessary and
         * the value that caused the failure is actually "persisted" and recovered.
         */
    AdversarialFileSystemAbstraction breakingFSA = new AdversarialFileSystemAbstraction(new MethodGuardedAdversary(new CountingAdversary(40, true), StoreChannel.class.getMethod("force", boolean.class)), normalFSA);
    SelectiveFileSystemAbstraction combinedFSA = new SelectiveFileSystemAbstraction(new File(new File(testDir.directory(), "long-state"), "long.a"), breakingFSA, normalFSA);
    long lastValue = 0;
    try (LongState persistedState = new LongState(combinedFSA, testDir.directory(), 14)) {
        while (// it will break from the Exception that AFS will throw
        true) {
            long tempValue = lastValue + 1;
            persistedState.setTheState(tempValue);
            lastValue = tempValue;
        }
    } catch (Exception expected) {
        // this stack trace should contain force()
        ensureStackTraceContainsExpectedMethod(expected.getStackTrace(), "force");
    }
    try (LongState restoredState = new LongState(normalFSA, testDir.directory(), 14)) {
        assertThat(restoredState.getTheState(), greaterThanOrEqualTo(lastValue));
    }
}
Also used : MethodGuardedAdversary(org.neo4j.adversaries.MethodGuardedAdversary) CountingAdversary(org.neo4j.adversaries.CountingAdversary) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) SelectiveFileSystemAbstraction(org.neo4j.graphdb.mockfs.SelectiveFileSystemAbstraction) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) File(java.io.File) IOException(java.io.IOException) EndOfStreamException(org.neo4j.causalclustering.messaging.EndOfStreamException) Test(org.junit.Test)

Example 39 with EphemeralFileSystemAbstraction

use of org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction in project neo4j by neo4j.

the class DurableStateStorageIT method shouldRecoverAfterCrashUnderLoad.

@Test
public void shouldRecoverAfterCrashUnderLoad() throws Exception {
    EphemeralFileSystemAbstraction delegate = fileSystemRule.get();
    AdversarialFileSystemAbstraction fsa = new AdversarialFileSystemAbstraction(new MethodGuardedAdversary(new CountingAdversary(100, true), StoreChannel.class.getMethod("writeAll", ByteBuffer.class)), delegate);
    long lastValue = 0;
    try (LongState persistedState = new LongState(fsa, testDir.directory(), 14)) {
        while (// it will break from the Exception that AFS will throw
        true) {
            long tempValue = lastValue + 1;
            persistedState.setTheState(tempValue);
            lastValue = tempValue;
        }
    } catch (Exception expected) {
        ensureStackTraceContainsExpectedMethod(expected.getStackTrace(), "writeAll");
    }
    try (LongState restoredState = new LongState(delegate, testDir.directory(), 4)) {
        assertEquals(lastValue, restoredState.getTheState());
    }
}
Also used : MethodGuardedAdversary(org.neo4j.adversaries.MethodGuardedAdversary) CountingAdversary(org.neo4j.adversaries.CountingAdversary) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) IOException(java.io.IOException) EndOfStreamException(org.neo4j.causalclustering.messaging.EndOfStreamException) Test(org.junit.Test)

Example 40 with EphemeralFileSystemAbstraction

use of org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction in project neo4j by neo4j.

the class DurableStateStorageIT method shouldProperlyRecoveryAfterCrashingDuringRecovery.

@Test
public void shouldProperlyRecoveryAfterCrashingDuringRecovery() throws Exception {
    EphemeralFileSystemAbstraction normalFSA = fileSystemRule.get();
    long lastValue = 0;
    try (LongState persistedState = new LongState(normalFSA, testDir.directory(), 14)) {
        for (int i = 0; i < 100; i++) {
            long tempValue = lastValue + 1;
            persistedState.setTheState(tempValue);
            lastValue = tempValue;
        }
    }
    try {
        // We create a new state that will attempt recovery. The AFS will make it fail on open() of one of the files
        new LongState(new AdversarialFileSystemAbstraction(new MethodGuardedAdversary(new CountingAdversary(1, true), FileSystemAbstraction.class.getMethod("open", File.class, String.class)), normalFSA), testDir.directory(), 14);
        fail("Should have failed recovery");
    } catch (Exception expected) {
        // this stack trace should contain open()
        ensureStackTraceContainsExpectedMethod(expected.getCause().getStackTrace(), "open");
    }
    // Recovery over the normal filesystem after a failed recovery should proceed correctly
    try (LongState recoveredState = new LongState(normalFSA, testDir.directory(), 14)) {
        assertThat(recoveredState.getTheState(), greaterThanOrEqualTo(lastValue));
    }
}
Also used : MethodGuardedAdversary(org.neo4j.adversaries.MethodGuardedAdversary) CountingAdversary(org.neo4j.adversaries.CountingAdversary) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) SelectiveFileSystemAbstraction(org.neo4j.graphdb.mockfs.SelectiveFileSystemAbstraction) FileSystemAbstraction(org.neo4j.io.fs.FileSystemAbstraction) EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) File(java.io.File) IOException(java.io.IOException) EndOfStreamException(org.neo4j.causalclustering.messaging.EndOfStreamException) Test(org.junit.Test)

Aggregations

EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)54 Test (org.junit.Test)37 File (java.io.File)27 TestGraphDatabaseFactory (org.neo4j.test.TestGraphDatabaseFactory)10 StoreChannel (org.neo4j.io.fs.StoreChannel)8 IOException (java.io.IOException)7 AdversarialFileSystemAbstraction (org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction)7 Before (org.junit.Before)6 CountingAdversary (org.neo4j.adversaries.CountingAdversary)6 Transaction (org.neo4j.graphdb.Transaction)6 ByteBuffer (java.nio.ByteBuffer)5 MethodGuardedAdversary (org.neo4j.adversaries.MethodGuardedAdversary)5 StateRecoveryManager (org.neo4j.causalclustering.core.state.StateRecoveryManager)5 EndOfStreamException (org.neo4j.causalclustering.messaging.EndOfStreamException)5 GraphDatabaseService (org.neo4j.graphdb.GraphDatabaseService)5 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)4 Node (org.neo4j.graphdb.Node)4 SelectiveFileSystemAbstraction (org.neo4j.graphdb.mockfs.SelectiveFileSystemAbstraction)4 FileSystemAbstraction (org.neo4j.io.fs.FileSystemAbstraction)4 NodeRecord (org.neo4j.kernel.impl.store.record.NodeRecord)4