Examples with Vertex com.hazelcast.jet.core.Vertex used on opensource projects

Example 26 with Vertex

use of com.hazelcast.jet.core.Vertex in project hazelcast-jet by hazelcast.

the class WriteFilePTest method when_slowSource_then_fileFlushedAfterEachItem.

@Test
public void when_slowSource_then_fileFlushedAfterEachItem() {
    // Given
    int numItems = 10;
    DAG dag = new DAG();
    Vertex source = dag.newVertex("source", () -> new SlowSourceP(semaphore, numItems)).localParallelism(1);
    Vertex sink = dag.newVertex("sink", writeFileP(directory.toString(), Object::toString, StandardCharsets.UTF_8, false)).localParallelism(1);
    dag.edge(between(source, sink));
    Job job = instance.newJob(dag);
    // wait, until the file is created
    assertTrueEventually(() -> assertTrue(Files.exists(file)));
    for (int i = 0; i < numItems; i++) {
        // When
        semaphore.release();
        int finalI = i;
        // Then
        assertTrueEventually(() -> checkFileContents(StandardCharsets.UTF_8, finalI + 1), 5);
    }
    // wait for the job to finish
    job.join();
}

Example 27 with Vertex

use of com.hazelcast.jet.core.Vertex in project hazelcast-jet by hazelcast.

the class WatermarkMaxRetention_IntegrationTest method test_onEdgeCoalescing.

@Test
public void test_onEdgeCoalescing() {
    DAG dag = new DAG();
    // a vertex with two processor instances, one will emit a wm and the other won't
    Vertex source = dag.newVertex("source", (int count) -> asList(new ListSource(singletonList(new Watermark(1))), new StuckForeverSourceP())).localParallelism(2);
    Vertex map = dag.newVertex("map", MapWmToStringP::new).localParallelism(1);
    Vertex sink = dag.newVertex("sink", writeListP(SINK_NAME));
    dag.edge(between(source, map)).edge(between(map, sink));
    doTest(dag);
}

Example 28 with Vertex

use of com.hazelcast.jet.core.Vertex in project hazelcast-jet by hazelcast.

the class WatermarkMaxRetention_IntegrationTest method test_onVertexCoalescing.

@Test
public void test_onVertexCoalescing() {
    DAG dag = new DAG();
    Vertex source0 = dag.newVertex("source0", () -> new ListSource(singletonList(new Watermark(1)))).localParallelism(1);
    Vertex source1 = dag.newVertex("source1", StuckForeverSourceP::new);
    // a vertex with two inputs, one will emit a wm and the other won't
    Vertex map = dag.newVertex("map", MapWmToStringP::new).localParallelism(1);
    Vertex sink = dag.newVertex("sink", writeListP(SINK_NAME));
    dag.edge(from(source0).to(map, 0)).edge(from(source1).to(map, 1)).edge(between(map, sink));
    doTest(dag);
}

Example 29 with Vertex

use of com.hazelcast.jet.core.Vertex in project hazelcast-jet by hazelcast.

the class HazelcastConnectorTest method when_readMap_withPredicateAndDistributedFunction.

@Test
public void when_readMap_withPredicateAndDistributedFunction() {
    IMapJet<Integer, Integer> sourceMap = jetInstance.getMap(sourceName);
    range(0, ENTRY_COUNT).forEach(i -> sourceMap.put(i, i));
    DAG dag = new DAG();
    Vertex source = dag.newVertex("source", readMapP(sourceName, e -> !e.getKey().equals(0), Map.Entry::getKey));
    Vertex sink = dag.newVertex("sink", writeListP(sinkName));
    dag.edge(between(source, sink));
    jetInstance.newJob(dag).join();
    IListJet<Object> list = jetInstance.getList(sinkName);
    assertEquals(ENTRY_COUNT - 1, list.size());
    assertFalse(list.contains(0));
    assertTrue(list.contains(1));
}

Example 30 with Vertex

use of com.hazelcast.jet.core.Vertex in project hazelcast-jet by hazelcast.

the class HazelcastConnectorTest method when_readMap_and_writeMap.

@Test
public void when_readMap_and_writeMap() {
    IMapJet<Integer, Integer> sourceMap = jetInstance.getMap(sourceName);
    range(0, ENTRY_COUNT).forEach(i -> sourceMap.put(i, i));
    DAG dag = new DAG();
    Vertex source = dag.newVertex("source", readMapP(sourceName));
    Vertex sink = dag.newVertex("sink", writeMapP(sinkName));
    dag.edge(between(source, sink));
    jetInstance.newJob(dag).join();
    assertEquals(ENTRY_COUNT, jetInstance.getMap(sinkName).size());
}