Examples with DAG com.hazelcast.jet.core.DAG used on opensource projects

Example 96 with DAG

use of com.hazelcast.jet.core.DAG in project hazelcast by hazelcast.

the class JetClassLoaderTest method when_processorSupplierCalled_then_contextClassLoaderSet.

@Test
public void when_processorSupplierCalled_then_contextClassLoaderSet() {
    DAG dag = new DAG();
    dag.newVertex("v", new LeakClassLoaderPS()).localParallelism(1);
    Config config = smallInstanceWithResourceUploadConfig();
    HazelcastInstance instance = createHazelcastInstance(config);
    // When
    instance.getJet().newJob(dag).join();
    assertClassLoaders(LeakClassLoaderPS.classLoaders);
    assertThat(LeakClassLoaderPS.classLoaders).containsKeys("init", "get", "close");
}

Example 97 with DAG

use of com.hazelcast.jet.core.DAG in project hazelcast by hazelcast.

the class OrderedBatchParallelismTest method applyTransformAndGetDag.

private DAG applyTransformAndGetDag(FunctionEx<BatchStage<Long>, BatchStage<Long>> transform) {
    PipelineImpl p = (PipelineImpl) Pipeline.create().setPreserveOrder(true);
    BatchStage<Long> source = p.readFrom(TestSources.items(1L)).setLocalParallelism(UPSTREAM_PARALLELISM);
    BatchStage<Long> applied = source.apply(transform);
    applied.mapStateful(LongAccumulator::new, (s, x) -> x).writeTo(Sinks.noop());
    return p.toDag(PIPELINE_CTX);
}

Example 98 with DAG

use of com.hazelcast.jet.core.DAG in project hazelcast by hazelcast.

the class JobMetrics_NonSharedClusterTest method when_metricsCollectionOff_then_emptyMetrics.

@Test
public void when_metricsCollectionOff_then_emptyMetrics() {
    Config config = smallInstanceConfig();
    config.getMetricsConfig().setEnabled(false);
    HazelcastInstance inst = createHazelcastInstance(config);
    DAG dag = new DAG();
    dag.newVertex("v1", (SupplierEx<Processor>) NoOutputSourceP::new).localParallelism(1);
    Job job = inst.getJet().newJob(dag, JOB_CONFIG_WITH_METRICS);
    assertTrue(job.getMetrics().metrics().isEmpty());
}

Example 99 with DAG

use of com.hazelcast.jet.core.DAG in project hazelcast by hazelcast.

the class JobMetrics_StressTest method stressTest.

private void stressTest(Function<Job, Runnable> restart) throws Throwable {
    DAG dag = buildDag();
    Job job = instance.getJet().newJob(dag, JOB_CONFIG_WITH_METRICS);
    try {
        assertTrueEventually(() -> assertEquals(JobStatus.RUNNING, job.getStatus()));
        ObtainMetricsThread obtainMetrics = new ObtainMetricsThread(job);
        Thread restartThread = new Thread(restart.apply(job));
        Thread obtainThread = new Thread(obtainMetrics);
        restartThread.start();
        obtainThread.start();
        restartThread.join();
        obtainMetrics.stop = true;
        obtainThread.join();
        if (restartThreadException != null) {
            throw restartThreadException;
        }
        if (obtainMetricsThreadException != null) {
            throw obtainMetricsThreadException;
        }
    } finally {
        ditchJob(job, instance);
    }
}

Example 100 with DAG

use of com.hazelcast.jet.core.DAG in project hazelcast by hazelcast.

the class JobMetrics_StressTest method buildDag.

private DAG buildDag() {
    DAG dag = new DAG();
    dag.newVertex("p", (SupplierEx<Processor>) IncrementingProcessor::new);
    return dag;
}