Examples with Pool org.openstreetmap.atlas.utilities.threads.Pool used on opensource projects

Example 1 with Pool

use of org.openstreetmap.atlas.utilities.threads.Pool in project atlas-checks by osmlab.

the class IntegrityCheckSparkJob method executeChecks.

/**
 * Executes all {@link BaseCheck}s on the given {@link Atlas}. Each check runs in a separate
 * thread. The checks go over all {@link AtlasEntity}s and {@link Relation}s.
 * {@link ComplexEntity}s can be processed by using the appropriate {@link Finder} and adding
 * them to the {@link Iterable} of objects.
 *
 * @param atlas
 *            the {@link Atlas} on which the checks will be run
 * @param checksToRun
 *            the set of {@link BaseCheck}s to execute
 * @param configuration
 *            {@link MapRouletteConfiguration} to create a new {@link MapRouletteClient}s
 */
@SuppressWarnings("rawtypes")
private static void executeChecks(final String country, final Atlas atlas, final Set<BaseCheck> checksToRun, final MapRouletteConfiguration configuration) {
    final Pool checkExecutionPool = new Pool(checksToRun.size(), "Check execution pool", POOL_DURATION_BEFORE_KILL);
    checksToRun.stream().filter(check -> check.validCheckForCountry(country)).forEach(check -> checkExecutionPool.queue(new RunnableCheck(country, check, new MultiIterable<>(atlas.items(), atlas.relations(), findComplexEntities(check, atlas)), MapRouletteClient.instance(configuration))));
    checkExecutionPool.close();
}

Example 2 with Pool

use of org.openstreetmap.atlas.utilities.threads.Pool in project atlas-checks by osmlab.

the class EventBusTest method testProcessCount.

private void testProcessCount(final int threadCount, final int eventCount) {
    final TestProcessor testProcessor = new TestProcessor();
    final EventBus eventBus = new EventBus();
    eventBus.register(testProcessor);
    // Send events
    final Pool threadPool = new Pool(threadCount, "Test event pool", Duration.ONE_MINUTE);
    for (int threadIndex = 0; threadIndex < threadCount; threadIndex++) {
        threadPool.queue(() -> {
            for (int index = 0; index < eventCount; index++) {
                eventBus.post(new TestEvent("test " + index));
            }
        });
    }
    threadPool.close();
    // Complete
    eventBus.post(new ShutdownEvent());
    // Validate
    Assert.assertEquals(eventCount * threadCount, testProcessor.getProcessCount());
    Assert.assertEquals(1, testProcessor.getCompleteCount());
}

Example 3 with Pool

use of org.openstreetmap.atlas.utilities.threads.Pool in project atlas-checks by osmlab.

the class EventServiceTest method testProcessCount.

private void testProcessCount(final int threadCount, final int eventCount) {
    final TestProcessor testProcessor = new TestProcessor();
    final EventService eventService = EventService.get("Test service for " + threadCount + "-" + eventCount);
    eventService.register(testProcessor);
    // Send events
    final Pool threadPool = new Pool(threadCount, "Test pool for " + threadCount + "-" + eventCount, Duration.ONE_MINUTE);
    for (int threadIndex = 0; threadIndex < threadCount; threadIndex++) {
        threadPool.queue(() -> {
            for (int index = 0; index < eventCount; index++) {
                eventService.post(new TestEvent("test " + index));
            }
        });
    }
    threadPool.close();
    // Complete
    eventService.complete();
    // Validate
    Assert.assertEquals(eventCount * threadCount, testProcessor.getProcessCount());
    Assert.assertEquals(1, testProcessor.getCompleteCount());
}