Examples with HadoopExecutorService org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService used on opensource projects

Example 1 with HadoopExecutorService

use of org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService in project ignite by apache.

the class HadoopExecutorServiceTest method testShutdown.

/**
     * @throws Exception If failed.
     */
public void testShutdown() throws Exception {
    for (int i = 0; i < 5; i++) {
        final HadoopExecutorService exec = new HadoopExecutorService(log, "_GRID_NAME_", 10, 5);
        final LongAdder8 sum = new LongAdder8();
        final AtomicBoolean finish = new AtomicBoolean();
        IgniteInternalFuture<?> fut = multithreadedAsync(new Callable<Object>() {

            @Override
            public Object call() throws Exception {
                while (!finish.get()) {
                    exec.submit(new Callable<Void>() {

                        @Override
                        public Void call() throws Exception {
                            sum.increment();
                            return null;
                        }
                    });
                }
                return null;
            }
        }, 19);
        Thread.sleep(200);
        assertTrue(exec.shutdown(50));
        long res = sum.sum();
        assertTrue(res > 0);
        finish.set(true);
        fut.get();
        // Nothing was executed after shutdown.
        assertEquals(res, sum.sum());
        X.println("_ ok");
    }
}

Example 2 with HadoopExecutorService

use of org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService in project ignite by apache.

the class HadoopExecutorServiceTest method testExecutesAll.

/**
     * @throws Exception If failed.
     */
public void testExecutesAll() throws Exception {
    final HadoopExecutorService exec = new HadoopExecutorService(log, "_GRID_NAME_", 10, 5);
    for (int i = 0; i < 5; i++) {
        final int loops = 5000;
        int threads = 17;
        final LongAdder8 sum = new LongAdder8();
        multithreaded(new Callable<Object>() {

            @Override
            public Object call() throws Exception {
                for (int i = 0; i < loops; i++) {
                    exec.submit(new Callable<Void>() {

                        @Override
                        public Void call() throws Exception {
                            sum.increment();
                            return null;
                        }
                    });
                }
                return null;
            }
        }, threads);
        while (exec.active() != 0) {
            X.println("__ active: " + exec.active());
            Thread.sleep(200);
        }
        assertEquals(threads * loops, sum.sum());
        X.println("_ ok");
    }
    assertTrue(exec.shutdown(0));
}

Example 3 with HadoopExecutorService

use of org.apache.ignite.internal.processors.hadoop.taskexecutor.HadoopExecutorService in project ignite by apache.

the class HadoopChildProcessRunner method initializeExecutors.

/**
     * Creates executor services.
     *
     */
private void initializeExecutors() {
    int cpus = Runtime.getRuntime().availableProcessors();
    execSvc = new HadoopExecutorService(log, "", cpus * 2, 1024);
}