Search in sources :

Example 6 with TestDatabaseManagementServiceBuilder

use of org.neo4j.test.TestDatabaseManagementServiceBuilder in project neo4j by neo4j.

the class PartialTransactionFailureIT method concurrentlyCommittingTransactionsMustNotRotateOutLoggedCommandsOfFailingTransaction.

@Test
void concurrentlyCommittingTransactionsMustNotRotateOutLoggedCommandsOfFailingTransaction() throws Exception {
    final ClassGuardedAdversary adversary = new ClassGuardedAdversary(new CountingAdversary(1, false), Command.RelationshipCommand.class);
    adversary.disable();
    Path storeDir = testDirectory.homePath();
    final Map<Setting<?>, Object> params = Map.of(GraphDatabaseSettings.pagecache_memory, "8m");
    managementService = new TestDatabaseManagementServiceBuilder(storeDir).setFileSystem(new AdversarialFileSystemAbstraction(adversary)).setConfig(params).build();
    GraphDatabaseAPI db = (GraphDatabaseAPI) managementService.database(DEFAULT_DATABASE_NAME);
    Node a;
    Node b;
    Node c;
    Node d;
    try (Transaction tx = db.beginTx()) {
        a = tx.createNode();
        b = tx.createNode();
        c = tx.createNode();
        d = tx.createNode();
        tx.commit();
    }
    adversary.enable();
    CountDownLatch latch = new CountDownLatch(1);
    Thread t1 = new Thread(createRelationship(db, a, b, latch), "T1");
    Thread t2 = new Thread(createRelationship(db, c, d, latch), "T2");
    t1.start();
    t2.start();
    // Wait for both threads to get going
    t1.join(10);
    t2.join(10);
    latch.countDown();
    // Wait for the transactions to finish
    t1.join(25000);
    t2.join(25000);
    managementService.shutdown();
    // We should observe the store in a consistent state
    managementService = new TestDatabaseManagementServiceBuilder(storeDir).setConfig(params).build();
    GraphDatabaseService database = managementService.database(DEFAULT_DATABASE_NAME);
    try (Transaction tx = database.beginTx()) {
        Node x = tx.getNodeById(a.getId());
        Node y = tx.getNodeById(b.getId());
        Node z = tx.getNodeById(c.getId());
        Node w = tx.getNodeById(d.getId());
        Iterator<Relationship> itrRelX = x.getRelationships().iterator();
        Iterator<Relationship> itrRelY = y.getRelationships().iterator();
        Iterator<Relationship> itrRelZ = z.getRelationships().iterator();
        Iterator<Relationship> itrRelW = w.getRelationships().iterator();
        if (itrRelX.hasNext() != itrRelY.hasNext()) {
            fail("Node x and y have inconsistent relationship counts");
        } else if (itrRelX.hasNext()) {
            Relationship rel = itrRelX.next();
            assertEquals(rel, itrRelY.next());
            assertFalse(itrRelX.hasNext());
            assertFalse(itrRelY.hasNext());
        }
        if (itrRelZ.hasNext() != itrRelW.hasNext()) {
            fail("Node z and w have inconsistent relationship counts");
        } else if (itrRelZ.hasNext()) {
            Relationship rel = itrRelZ.next();
            assertEquals(rel, itrRelW.next());
            assertFalse(itrRelZ.hasNext());
            assertFalse(itrRelW.hasNext());
        }
    }
}
Also used : Path(java.nio.file.Path) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) CountingAdversary(org.neo4j.adversaries.CountingAdversary) ClassGuardedAdversary(org.neo4j.adversaries.ClassGuardedAdversary) Setting(org.neo4j.graphdb.config.Setting) Node(org.neo4j.graphdb.Node) CountDownLatch(java.util.concurrent.CountDownLatch) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Transaction(org.neo4j.graphdb.Transaction) Command(org.neo4j.internal.recordstorage.Command) Relationship(org.neo4j.graphdb.Relationship) AdversarialFileSystemAbstraction(org.neo4j.adversaries.fs.AdversarialFileSystemAbstraction) Test(org.junit.jupiter.api.Test)

Example 7 with TestDatabaseManagementServiceBuilder

use of org.neo4j.test.TestDatabaseManagementServiceBuilder in project neo4j by neo4j.

the class TransactionMonitorTest method shouldCountTerminatedTransactions.

@ParameterizedTest(name = "{0}")
@MethodSource("parameters")
void shouldCountTerminatedTransactions(String name, ThrowingConsumer<Transaction, Exception> txConsumer, boolean isWriteTx) throws Exception {
    DatabaseManagementService managementService = new TestDatabaseManagementServiceBuilder().impermanent().build();
    GraphDatabaseAPI db = (GraphDatabaseAPI) managementService.database(DEFAULT_DATABASE_NAME);
    try {
        TransactionCounters counts = db.getDependencyResolver().resolveDependency(TransactionCounters.class);
        TransactionCountersChecker checker = new TransactionCountersChecker(counts);
        try (Transaction tx = db.beginTx()) {
            txConsumer.accept(tx);
            tx.terminate();
        }
        checker.verifyTerminated(isWriteTx, counts);
    } finally {
        managementService.shutdown();
    }
}
Also used : TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Transaction(org.neo4j.graphdb.Transaction) DatabaseManagementService(org.neo4j.dbms.api.DatabaseManagementService) TransactionCounters(org.neo4j.kernel.impl.transaction.stats.TransactionCounters) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 8 with TestDatabaseManagementServiceBuilder

use of org.neo4j.test.TestDatabaseManagementServiceBuilder in project neo4j by neo4j.

the class ReuseStorageSpaceIT method shouldPrioritizeFreelistWhenConcurrentlyAllocating.

@Test
void shouldPrioritizeFreelistWhenConcurrentlyAllocating() throws Exception {
    DatabaseManagementService dbms = new TestDatabaseManagementServiceBuilder(directory.homePath()).setConfig(GraphDatabaseInternalSettings.force_small_id_cache, true).build();
    try {
        // given
        GraphDatabaseAPI db = (GraphDatabaseAPI) dbms.database(DEFAULT_DATABASE_NAME);
        int numNodes = 40_000;
        MutableLongSet nodeIds = createNodes(db, numNodes);
        try (Transaction tx = db.beginTx()) {
            nodeIds.forEach(nodeId -> tx.getNodeById(nodeId).delete());
            tx.commit();
        }
        db.getDependencyResolver().resolveDependency(IdController.class).maintenance();
        // First create 40,000 nodes, then delete them, ensure ID maintenance has run and allocate concurrently
        int numThreads = 4;
        Collection<Callable<MutableLongSet>> allocators = new ArrayList<>();
        for (int i = 0; i < numThreads; i++) {
            allocators.add(() -> createNodes(db, numNodes / numThreads));
        }
        ExecutorService executor = Executors.newFixedThreadPool(numThreads);
        List<Future<MutableLongSet>> results = executor.invokeAll(allocators);
        MutableLongSet reallocatedNodeIds = LongSets.mutable.withInitialCapacity(numNodes);
        for (Future<MutableLongSet> result : results) {
            reallocatedNodeIds.addAll(result.get());
        }
        assertThat(reallocatedNodeIds).as(diff(nodeIds, reallocatedNodeIds)).isEqualTo(nodeIds);
    } finally {
        dbms.shutdown();
    }
}
Also used : ArrayList(java.util.ArrayList) Callable(java.util.concurrent.Callable) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) MutableLongSet(org.eclipse.collections.api.set.primitive.MutableLongSet) GraphDatabaseAPI(org.neo4j.kernel.internal.GraphDatabaseAPI) Transaction(org.neo4j.graphdb.Transaction) ExecutorService(java.util.concurrent.ExecutorService) IdController(org.neo4j.internal.id.IdController) Future(java.util.concurrent.Future) DatabaseManagementService(org.neo4j.dbms.api.DatabaseManagementService) Test(org.junit.jupiter.api.Test)

Example 9 with TestDatabaseManagementServiceBuilder

use of org.neo4j.test.TestDatabaseManagementServiceBuilder in project neo4j by neo4j.

the class FulltextIndexConsistencyCheckIT method mustDiscoverNodeInIndexMissingFromStore.

@Disabled("Turns out that this is not something that the consistency checker actually looks for, currently. " + "The test is disabled until the consistency checker is extended with checks that will discover this sort of inconsistency.")
@Test
void mustDiscoverNodeInIndexMissingFromStore() throws Exception {
    GraphDatabaseService db = createDatabase();
    try (Transaction tx = db.beginTx()) {
        tx.execute(format(NODE_CREATE, "nodes", asStrList("Label"), asStrList("prop"))).close();
        tx.commit();
    }
    long nodeId;
    try (Transaction tx = db.beginTx()) {
        tx.schema().awaitIndexesOnline(2, TimeUnit.MINUTES);
        Node node = tx.createNode(Label.label("Label"));
        nodeId = node.getId();
        node.setProperty("prop", "value");
        tx.commit();
    }
    // Remove the property without updating the index
    managementService.shutdown();
    DatabaseManagementService managementService = new TestDatabaseManagementServiceBuilder(databaseLayout).setFileSystem(fs).removeExtensions(INDEX_PROVIDERS_FILTER).addExtension(new FailingGenericNativeIndexProviderFactory(SKIP_ONLINE_UPDATES)).addExtension(new TokenIndexProviderFactory()).build();
    db = managementService.database(DEFAULT_DATABASE_NAME);
    try (Transaction tx = db.beginTx()) {
        tx.getNodeById(nodeId).removeProperty("prop");
        tx.commit();
    }
    managementService.shutdown();
    ConsistencyCheckService.Result result = checkConsistency();
    assertFalse(result.isSuccessful());
}
Also used : GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) Transaction(org.neo4j.graphdb.Transaction) Node(org.neo4j.graphdb.Node) TokenIndexProviderFactory(org.neo4j.kernel.impl.index.schema.TokenIndexProviderFactory) FailingGenericNativeIndexProviderFactory(org.neo4j.kernel.impl.index.schema.FailingGenericNativeIndexProviderFactory) ConsistencyCheckService(org.neo4j.consistency.ConsistencyCheckService) DatabaseManagementService(org.neo4j.dbms.api.DatabaseManagementService) Test(org.junit.jupiter.api.Test) Disabled(org.junit.jupiter.api.Disabled)

Example 10 with TestDatabaseManagementServiceBuilder

use of org.neo4j.test.TestDatabaseManagementServiceBuilder in project neo4j by neo4j.

the class IndexConfigMigrationIT method create3_5Database.

@Disabled("Here as reference for how 3.5 db was created")
@Test
void create3_5Database() throws Exception {
    Path storeDir = tempStoreDirectory();
    DatabaseManagementServiceBuilder builder = new TestDatabaseManagementServiceBuilder(storeDir);
    setSpatialConfig(builder);
    DatabaseManagementService dbms = builder.build();
    GraphDatabaseService db = dbms.database(DEFAULT_DATABASE_NAME);
    createIndex(db, NATIVE_BTREE10.providerName(), label1);
    // createIndex( db, NATIVE20.providerName(), label2 ); // <- Those index providers are removed in 4.0, but here for reference.
    // createIndex( db, NATIVE10.providerName(), label3 );
    // createIndex( db, LUCENE10.providerName(), label4 );
    createSpatialData(db, label1, label2, label3, label4);
    for (FulltextIndexDescription fulltextIndex : FulltextIndexDescription.values()) {
        createFulltextIndex(db, fulltextIndex.indexProcedure, fulltextIndex.indexName, fulltextIndex.tokenName, propKey, fulltextIndex.configMap);
    }
    dbms.shutdown();
    Path zipFile = storeDir.resolveSibling(storeDir.getFileName().toString() + ".zip");
    ZipUtils.zip(new DefaultFileSystemAbstraction(), storeDir, zipFile);
    System.out.println("Db created in " + zipFile.toAbsolutePath());
}
Also used : Path(java.nio.file.Path) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) DefaultFileSystemAbstraction(org.neo4j.io.fs.DefaultFileSystemAbstraction) TestDatabaseManagementServiceBuilder(org.neo4j.test.TestDatabaseManagementServiceBuilder) DatabaseManagementServiceBuilder(org.neo4j.dbms.api.DatabaseManagementServiceBuilder) DatabaseManagementService(org.neo4j.dbms.api.DatabaseManagementService) Test(org.junit.jupiter.api.Test) Disabled(org.junit.jupiter.api.Disabled)

Aggregations

TestDatabaseManagementServiceBuilder (org.neo4j.test.TestDatabaseManagementServiceBuilder)135 DatabaseManagementService (org.neo4j.dbms.api.DatabaseManagementService)74 Test (org.junit.jupiter.api.Test)48 Transaction (org.neo4j.graphdb.Transaction)42 GraphDatabaseService (org.neo4j.graphdb.GraphDatabaseService)38 GraphDatabaseAPI (org.neo4j.kernel.internal.GraphDatabaseAPI)30 Path (java.nio.file.Path)23 BeforeEach (org.junit.jupiter.api.BeforeEach)18 Node (org.neo4j.graphdb.Node)18 EphemeralFileSystemAbstraction (org.neo4j.io.fs.EphemeralFileSystemAbstraction)13 AssertableLogProvider (org.neo4j.logging.AssertableLogProvider)11 PageCache (org.neo4j.io.pagecache.PageCache)10 UncloseableDelegatingFileSystemAbstraction (org.neo4j.io.fs.UncloseableDelegatingFileSystemAbstraction)9 DatabaseLayout (org.neo4j.io.layout.DatabaseLayout)8 KernelTransaction (org.neo4j.kernel.api.KernelTransaction)7 TokenIndexProviderFactory (org.neo4j.kernel.impl.index.schema.TokenIndexProviderFactory)7 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)6 Dependencies (org.neo4j.collection.Dependencies)6 Config (org.neo4j.configuration.Config)6 DatabaseManagementServiceBuilder (org.neo4j.dbms.api.DatabaseManagementServiceBuilder)6