Search in sources :

Example 26 with Lifespan

use of org.neo4j.kernel.lifecycle.Lifespan in project neo4j by neo4j.

the class DurableStateStorageTest method shouldClearFileOnFirstUse.

@Test
public void shouldClearFileOnFirstUse() throws Throwable {
    // given
    EphemeralFileSystemAbstraction fsa = fileSystemRule.get();
    fsa.mkdir(testDir.directory());
    int rotationCount = 10;
    DurableStateStorage<AtomicInteger> storage = new DurableStateStorage<>(fsa, testDir.directory(), "state", new AtomicIntegerMarshal(), rotationCount, NullLogProvider.getInstance());
    int largestValueWritten = 0;
    try (Lifespan lifespan = new Lifespan(storage)) {
        for (; largestValueWritten < rotationCount * 2; largestValueWritten++) {
            storage.persistStoreData(new AtomicInteger(largestValueWritten));
        }
    }
    // now both files are full. We reopen, then write some more.
    storage = lifeRule.add(new DurableStateStorage<>(fsa, testDir.directory(), "state", new AtomicIntegerMarshal(), rotationCount, NullLogProvider.getInstance()));
    storage.persistStoreData(new AtomicInteger(largestValueWritten++));
    storage.persistStoreData(new AtomicInteger(largestValueWritten++));
    storage.persistStoreData(new AtomicInteger(largestValueWritten));
    /*
         * We have written stuff in fileA but not gotten to the end (resulting in rotation). The largestValueWritten
         * should nevertheless be correct
         */
    ByteBuffer forReadingBackIn = ByteBuffer.allocate(10_000);
    StoreChannel lastWrittenTo = fsa.open(stateFileA(), "r");
    lastWrittenTo.read(forReadingBackIn);
    forReadingBackIn.flip();
    AtomicInteger lastRead = null;
    while (true) {
        try {
            lastRead = new AtomicInteger(forReadingBackIn.getInt());
        } catch (BufferUnderflowException e) {
            break;
        }
    }
    // then
    assertNotNull(lastRead);
    assertEquals(largestValueWritten, lastRead.get());
}
Also used : EphemeralFileSystemAbstraction(org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) StoreChannel(org.neo4j.io.fs.StoreChannel) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) ByteBuffer(java.nio.ByteBuffer) BufferUnderflowException(java.nio.BufferUnderflowException) Test(org.junit.Test)

Example 27 with Lifespan

use of org.neo4j.kernel.lifecycle.Lifespan in project neo4j by neo4j.

the class CountsRotationTest method shouldUnMapThePrestateFileWhenTimingOutOnRotationAndAllowForShutdownInTheFailedRotationState.

@Test
public void shouldUnMapThePrestateFileWhenTimingOutOnRotationAndAllowForShutdownInTheFailedRotationState() throws Throwable {
    // Given
    dbBuilder.newGraphDatabase().shutdown();
    CountsTracker store = createCountsTracker(pageCache, Config.defaults().augment(Collections.singletonMap(GraphDatabaseSettings.counts_store_rotation_timeout.name(), "100ms")));
    try (Lifespan lifespan = new Lifespan(store)) {
        try (CountsAccessor.Updater updater = store.apply(2).get()) {
            updater.incrementNodeCount(0, 1);
        }
        try {
            // when
            store.rotate(3);
            fail("should have thrown");
        } catch (RotationTimeoutException ex) {
        // good
        }
    }
    // and also no exceptions closing the page cache
    pageCache.close();
}
Also used : RotationTimeoutException(org.neo4j.kernel.impl.store.kvstore.RotationTimeoutException) CountsAccessor(org.neo4j.kernel.impl.api.CountsAccessor) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Example 28 with Lifespan

use of org.neo4j.kernel.lifecycle.Lifespan in project neo4j by neo4j.

the class CountsRotationTest method shouldRotateCountsStoreWhenClosingTheDatabase.

@Test
public void shouldRotateCountsStoreWhenClosingTheDatabase() throws IOException {
    // GIVEN
    GraphDatabaseAPI db = (GraphDatabaseAPI) dbBuilder.newGraphDatabase();
    try (Transaction tx = db.beginTx()) {
        db.createNode(A);
        tx.success();
    }
    // WHEN
    db.shutdown();
    // THEN
    assertTrue(fs.fileExists(alphaStoreFile()));
    assertTrue(fs.fileExists(betaStoreFile()));
    try (Lifespan life = new Lifespan()) {
        CountsTracker store = life.add(createCountsTracker(pageCache));
        // a transaction for creating the label and a transaction for the node
        assertEquals(BASE_TX_ID + 1 + 1, store.txId());
        assertEquals(INITIAL_MINOR_VERSION, store.minorVersion());
        // one for all nodes and one for the created "A" label
        assertEquals(1 + 1, store.totalEntriesStored());
        assertEquals(1 + 1, allRecords(store).size());
    }
}
Also used : GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Transaction(org.neo4j.graphdb.Transaction) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Example 29 with Lifespan

use of org.neo4j.kernel.lifecycle.Lifespan in project neo4j by neo4j.

the class CountsTrackerTest method shouldBeAbleToReadUpToDateValueWhileAnotherThreadIsPerformingRotation.

@Test
public void shouldBeAbleToReadUpToDateValueWhileAnotherThreadIsPerformingRotation() throws Exception {
    // given
    CountsOracle oracle = someData();
    final int firstTransaction = 2, secondTransaction = 3;
    try (Lifespan life = new Lifespan()) {
        CountsTracker tracker = life.add(newTracker());
        oracle.update(tracker, firstTransaction);
        tracker.rotate(firstTransaction);
    }
    // when
    final CountsOracle delta = new CountsOracle();
    {
        CountsOracle.Node n1 = delta.node(1);
        // Label 4 has not been used before...
        CountsOracle.Node n2 = delta.node(1, 4);
        delta.relationship(n1, 1, n2);
        // relationshipType 2 has not been used before...
        delta.relationship(n2, 2, n1);
    }
    delta.update(oracle);
    try (Lifespan life = new Lifespan()) {
        final Barrier.Control barrier = new Barrier.Control();
        CountsTracker tracker = life.add(new CountsTracker(resourceManager.logProvider(), resourceManager.fileSystem(), resourceManager.pageCache(), Config.empty(), resourceManager.testPath()) {

            @Override
            protected boolean include(CountsKey countsKey, ReadableBuffer value) {
                barrier.reached();
                return super.include(countsKey, value);
            }
        });
        Future<Void> task = threading.execute((ThrowingFunction<CountsTracker, Void, RuntimeException>) t -> {
            try {
                delta.update(t, secondTransaction);
                t.rotate(secondTransaction);
            } catch (IOException e) {
                throw new AssertionError(e);
            }
            return null;
        }, tracker);
        // then
        barrier.await();
        oracle.verify(tracker);
        barrier.release();
        task.get();
        oracle.verify(tracker);
    }
}
Also used : CountsKey(org.neo4j.kernel.impl.store.counts.keys.CountsKey) Registers(org.neo4j.register.Registers) Assert.assertSame(org.junit.Assert.assertSame) Future(java.util.concurrent.Future) FakeClock(org.neo4j.time.FakeClock) CountsVisitor(org.neo4j.kernel.impl.api.CountsVisitor) Mockito.verifyNoMoreInteractions(org.mockito.Mockito.verifyNoMoreInteractions) CountsKeyFactory(org.neo4j.kernel.impl.store.counts.keys.CountsKeyFactory) DebugUtil.classNameContains(org.neo4j.kernel.impl.util.DebugUtil.classNameContains) Assert.fail(org.junit.Assert.fail) ReadableBuffer(org.neo4j.kernel.impl.store.kvstore.ReadableBuffer) STARTED(org.neo4j.test.rule.Resources.InitialLifecycle.STARTED) Config(org.neo4j.kernel.configuration.Config) CountsOracle(org.neo4j.kernel.impl.store.CountsOracle) Barrier(org.neo4j.test.Barrier) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Predicate(java.util.function.Predicate) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) MILLISECONDS(java.util.concurrent.TimeUnit.MILLISECONDS) Register(org.neo4j.register.Register) IOFunction(org.neo4j.function.IOFunction) ThrowingFunction(org.neo4j.function.ThrowingFunction) File(java.io.File) Assert.assertNotEquals(org.junit.Assert.assertNotEquals) Mockito.verify(org.mockito.Mockito.verify) ExecutionException(java.util.concurrent.ExecutionException) DataInitializer(org.neo4j.kernel.impl.store.kvstore.DataInitializer) FILE_IN_EXISTING_DIRECTORY(org.neo4j.test.rule.Resources.TestPath.FILE_IN_EXISTING_DIRECTORY) DebugUtil.stackTraceContains(org.neo4j.kernel.impl.util.DebugUtil.stackTraceContains) RotationTimeoutException(org.neo4j.kernel.impl.store.kvstore.RotationTimeoutException) Rule(org.junit.Rule) CountsAccessor(org.neo4j.kernel.impl.api.CountsAccessor) Resources(org.neo4j.test.rule.Resources) ThreadingRule(org.neo4j.test.rule.concurrent.ThreadingRule) Clock(java.time.Clock) DebugUtil.methodIs(org.neo4j.kernel.impl.util.DebugUtil.methodIs) GraphDatabaseSettings(org.neo4j.graphdb.factory.GraphDatabaseSettings) Clocks(org.neo4j.time.Clocks) Assert.assertEquals(org.junit.Assert.assertEquals) SECONDS(java.util.concurrent.TimeUnit.SECONDS) Predicates.all(org.neo4j.function.Predicates.all) Mockito.mock(org.mockito.Mockito.mock) Barrier(org.neo4j.test.Barrier) CountsKey(org.neo4j.kernel.impl.store.counts.keys.CountsKey) IOException(java.io.IOException) ReadableBuffer(org.neo4j.kernel.impl.store.kvstore.ReadableBuffer) CountsOracle(org.neo4j.kernel.impl.store.CountsOracle) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Example 30 with Lifespan

use of org.neo4j.kernel.lifecycle.Lifespan in project neo4j by neo4j.

the class CountsTrackerTest method shouldUpdateCountsOnExistingStore.

@Test
public void shouldUpdateCountsOnExistingStore() throws Exception {
    // given
    CountsOracle oracle = someData();
    int firstTx = 2, secondTx = 3;
    try (Lifespan life = new Lifespan()) {
        CountsTracker tracker = life.add(newTracker());
        oracle.update(tracker, firstTx);
        tracker.rotate(firstTx);
        oracle.verify(tracker);
        // when
        CountsOracle delta = new CountsOracle();
        {
            CountsOracle.Node n1 = delta.node(1);
            // Label 4 has not been used before...
            CountsOracle.Node n2 = delta.node(1, 4);
            delta.relationship(n1, 1, n2);
            // relationshipType 2 has not been used before...
            delta.relationship(n2, 2, n1);
        }
        delta.update(tracker, secondTx);
        delta.update(oracle);
        // then
        oracle.verify(tracker);
        // when
        tracker.rotate(secondTx);
    }
    // then
    try (Lifespan life = new Lifespan()) {
        oracle.verify(life.add(newTracker()));
    }
}
Also used : CountsOracle(org.neo4j.kernel.impl.store.CountsOracle) Lifespan(org.neo4j.kernel.lifecycle.Lifespan) Test(org.junit.Test)

Aggregations

Lifespan (org.neo4j.kernel.lifecycle.Lifespan)32 Test (org.junit.Test)22 GraphDatabaseAPI (org.neo4j.kernel.internal.GraphDatabaseAPI)8 Transaction (org.neo4j.graphdb.Transaction)6 File (java.io.File)5 Node (org.neo4j.graphdb.Node)4 DurableStateStorage (org.neo4j.causalclustering.core.state.storage.DurableStateStorage)3 EphemeralFileSystemAbstraction (org.neo4j.graphdb.mockfs.EphemeralFileSystemAbstraction)3 StoreChannel (org.neo4j.io.fs.StoreChannel)3 CountsOracle (org.neo4j.kernel.impl.store.CountsOracle)3 PhysicalLogFiles (org.neo4j.kernel.impl.transaction.log.PhysicalLogFiles)3 ReadOnlyTransactionIdStore (org.neo4j.kernel.impl.transaction.log.ReadOnlyTransactionIdStore)3 TransactionIdStore (org.neo4j.kernel.impl.transaction.log.TransactionIdStore)3 Monitors (org.neo4j.kernel.monitoring.Monitors)3 IOException (java.io.IOException)2 ByteBuffer (java.nio.ByteBuffer)2 Future (java.util.concurrent.Future)2 StateMarshal (org.neo4j.causalclustering.core.state.storage.StateMarshal)2 MemberId (org.neo4j.causalclustering.identity.MemberId)2 PageCache (org.neo4j.io.pagecache.PageCache)2