Search in sources :

Example 46 with DoubleLatch

use of org.neo4j.test.DoubleLatch in project neo4j by neo4j.

the class IndexSamplingControllerTest method shouldNotStartOtherSamplingWhenSamplingAllTheIndexes.

@Test
public void shouldNotStartOtherSamplingWhenSamplingAllTheIndexes() {
    // given
    final AtomicInteger totalCount = new AtomicInteger(0);
    final AtomicInteger concurrentCount = new AtomicInteger(0);
    final DoubleLatch jobLatch = new DoubleLatch();
    final DoubleLatch testLatch = new DoubleLatch();
    IndexSamplingJobFactory jobFactory = (indexId, proxy) -> {
        if (!concurrentCount.compareAndSet(0, 1)) {
            throw new IllegalStateException("count !== 0 on create");
        }
        totalCount.incrementAndGet();
        jobLatch.waitForAllToStart();
        testLatch.startAndWaitForAllToStart();
        jobLatch.waitForAllToFinish();
        concurrentCount.decrementAndGet();
        testLatch.finish();
        return null;
    };
    final IndexSamplingController controller = new IndexSamplingController(samplingConfig, jobFactory, jobQueue, tracker, snapshotProvider, scheduler, always(true));
    when(tracker.canExecuteMoreSamplingJobs()).thenReturn(true);
    when(indexProxy.getState()).thenReturn(ONLINE);
    // when running once
    new Thread(runController(controller, TRIGGER_REBUILD_UPDATED)).start();
    jobLatch.startAndWaitForAllToStart();
    testLatch.waitForAllToStart();
    // then blocking on first job
    assertEquals(1, concurrentCount.get());
    // when running a second time
    controller.sampleIndexes(BACKGROUND_REBUILD_UPDATED);
    // then no concurrent job execution
    jobLatch.finish();
    testLatch.waitForAllToFinish();
    // and finally exactly one job has run to completion
    assertEquals(0, concurrentCount.get());
    assertEquals(1, totalCount.get());
}
Also used : IndexMap(org.neo4j.kernel.impl.api.index.IndexMap) IndexProxy(org.neo4j.kernel.impl.api.index.IndexProxy) Predicates(org.neo4j.function.Predicates) NewIndexDescriptorFactory(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptorFactory) Test(org.junit.Test) Mockito.times(org.mockito.Mockito.times) DoubleLatch(org.neo4j.test.DoubleLatch) Mockito.when(org.mockito.Mockito.when) NewIndexDescriptor(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor) JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) ONLINE(org.neo4j.kernel.api.index.InternalIndexState.ONLINE) TRIGGER_REBUILD_UPDATED(org.neo4j.kernel.impl.api.index.sampling.IndexSamplingMode.TRIGGER_REBUILD_UPDATED) IndexMapSnapshotProvider(org.neo4j.kernel.impl.api.index.IndexMapSnapshotProvider) Mockito.verify(org.mockito.Mockito.verify) Mockito.never(org.mockito.Mockito.never) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Mockito.verifyNoMoreInteractions(org.mockito.Mockito.verifyNoMoreInteractions) FAILED(org.neo4j.kernel.api.index.InternalIndexState.FAILED) BACKGROUND_REBUILD_UPDATED(org.neo4j.kernel.impl.api.index.sampling.IndexSamplingMode.BACKGROUND_REBUILD_UPDATED) Assert.assertEquals(org.junit.Assert.assertEquals) Mockito.mock(org.mockito.Mockito.mock) POPULATING(org.neo4j.kernel.api.index.InternalIndexState.POPULATING) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) DoubleLatch(org.neo4j.test.DoubleLatch) Test(org.junit.Test)

Example 47 with DoubleLatch

use of org.neo4j.test.DoubleLatch in project neo4j by neo4j.

the class IndexSamplingJobTrackerTest method shouldAcceptNewJobWhenRunningJobFinishes.

@Test(timeout = 5_000)
public void shouldAcceptNewJobWhenRunningJobFinishes() throws Throwable {
    // Given
    when(config.jobLimit()).thenReturn(1);
    JobScheduler jobScheduler = new Neo4jJobScheduler();
    jobScheduler.init();
    final IndexSamplingJobTracker jobTracker = new IndexSamplingJobTracker(config, jobScheduler);
    final DoubleLatch latch = new DoubleLatch();
    final AtomicBoolean lastJobExecuted = new AtomicBoolean();
    jobTracker.scheduleSamplingJob(new IndexSamplingJob() {

        @Override
        public long indexId() {
            return indexId11;
        }

        @Override
        public void run() {
            latch.waitForAllToStart();
        }
    });
    // When
    Executors.newSingleThreadExecutor().execute(() -> {
        jobTracker.waitUntilCanExecuteMoreSamplingJobs();
        jobTracker.scheduleSamplingJob(new IndexSamplingJob() {

            @Override
            public long indexId() {
                return indexId22;
            }

            @Override
            public void run() {
                lastJobExecuted.set(true);
                latch.finish();
            }
        });
    });
    assertFalse(jobTracker.canExecuteMoreSamplingJobs());
    latch.startAndWaitForAllToStart();
    latch.waitForAllToFinish();
    // Then
    assertTrue(lastJobExecuted.get());
}
Also used : JobScheduler(org.neo4j.kernel.impl.util.JobScheduler) Neo4jJobScheduler(org.neo4j.kernel.impl.util.Neo4jJobScheduler) Neo4jJobScheduler(org.neo4j.kernel.impl.util.Neo4jJobScheduler) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) DoubleLatch(org.neo4j.test.DoubleLatch) Test(org.junit.Test)

Example 48 with DoubleLatch

use of org.neo4j.test.DoubleLatch in project neo4j by neo4j.

the class NodeGetUniqueFromIndexSeekIT method shouldBlockUniqueIndexSeekFromCompetingTransaction.

@Test(timeout = 1000)
public void shouldBlockUniqueIndexSeekFromCompetingTransaction() throws Exception {
    // This is the interleaving that we are trying to verify works correctly:
    // ----------------------------------------------------------------------
    // Thread1 (main)        : Thread2
    // create unique node    :
    // lookup(node)          :
    // open start latch ----->
    //    |                  : lookup(node)
    // wait for T2 to block  :      |
    //                       :    *block*
    // commit --------------->   *unblock*
    // wait for T2 end latch :      |
    //                       : finish transaction
    //                       : open end latch
    // *unblock* <-------------‘
    // assert that we complete before timeout
    final DoubleLatch latch = new DoubleLatch();
    final NewIndexDescriptor index = createUniquenessConstraint(labelId, propertyId1);
    final String value = "value";
    DataWriteOperations dataStatement = dataWriteOperationsInNewTransaction();
    long nodeId = dataStatement.nodeCreate();
    dataStatement.nodeAddLabel(nodeId, labelId);
    // This adds the node to the unique index and should take an index write lock
    dataStatement.nodeSetProperty(nodeId, Property.stringProperty(propertyId1, value));
    Runnable runnableForThread2 = () -> {
        latch.waitForAllToStart();
        try (Transaction tx = db.beginTx()) {
            try (Statement statement1 = statementContextSupplier.get()) {
                statement1.readOperations().nodeGetFromUniqueIndexSeek(index, exact(propertyId1, value));
            }
            tx.success();
        } catch (IndexNotFoundKernelException | IndexNotApplicableKernelException | IndexBrokenKernelException e) {
            throw new RuntimeException(e);
        } finally {
            latch.finish();
        }
    };
    Thread thread2 = new Thread(runnableForThread2, "Transaction Thread 2");
    thread2.start();
    latch.startAndWaitForAllToStart();
    //noinspection UnusedLabel
    spinUntilBlocking: for (; ; ) {
        if (thread2.getState() == Thread.State.TIMED_WAITING || thread2.getState() == Thread.State.WAITING) {
            break;
        }
        Thread.yield();
    }
    commit();
    latch.waitForAllToFinish();
}
Also used : NewIndexDescriptor(org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor) Transaction(org.neo4j.graphdb.Transaction) DataWriteOperations(org.neo4j.kernel.api.DataWriteOperations) Statement(org.neo4j.kernel.api.Statement) DoubleLatch(org.neo4j.test.DoubleLatch) Test(org.junit.Test)

Aggregations

DoubleLatch (org.neo4j.test.DoubleLatch)48 Test (org.junit.Test)37 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)15 Collections.emptyMap (java.util.Collections.emptyMap)9 Map (java.util.Map)9 MapUtil.stringMap (org.neo4j.helpers.collection.MapUtil.stringMap)9 OffsetDateTime (java.time.OffsetDateTime)5 JobScheduler (org.neo4j.kernel.impl.util.JobScheduler)5 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)4 NewIndexDescriptor (org.neo4j.kernel.api.schema_new.index.NewIndexDescriptor)4 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)3 Server (org.eclipse.jetty.server.Server)3 SchemaIndexTestHelper.mockIndexProxy (org.neo4j.kernel.impl.api.index.SchemaIndexTestHelper.mockIndexProxy)3 Neo4jJobScheduler (org.neo4j.kernel.impl.util.Neo4jJobScheduler)3 AtomicReference (java.util.concurrent.atomic.AtomicReference)2 Assert.assertEquals (org.junit.Assert.assertEquals)2 Mockito.mock (org.mockito.Mockito.mock)2 Mockito.never (org.mockito.Mockito.never)2 Mockito.times (org.mockito.Mockito.times)2 Mockito.verify (org.mockito.Mockito.verify)2