Examples with BatchSource com.hazelcast.jet.pipeline.BatchSource used on opensource projects

Example 11 with BatchSource

use of com.hazelcast.jet.pipeline.BatchSource in project hazelcast by hazelcast.

the class CommonElasticSourcesTest method given_multipleIndexes_when_readFromElasticSourceWithIndex_then_resultHasNoDocumentFromOtherIndex.

@Test
public void given_multipleIndexes_when_readFromElasticSourceWithIndex_then_resultHasNoDocumentFromOtherIndex() {
    indexDocument("my-index-1", of("name", "Frantisek"));
    indexDocument("my-index-2", of("name", "Vladimir"));
    Pipeline p = Pipeline.create();
    BatchSource<String> source = new ElasticSourceBuilder<>().clientFn(elasticClientSupplier()).searchRequestFn(() -> new SearchRequest("my-index-1")).mapToItemFn(hit -> (String) hit.getSourceAsMap().get("name")).build();
    p.readFrom(source).writeTo(Sinks.list(results));
    submitJob(p);
    assertThat(results).containsOnlyOnce("Frantisek");
}

Example 12 with BatchSource

use of com.hazelcast.jet.pipeline.BatchSource in project hazelcast by hazelcast.

the class LocalElasticSourcesTest method when_readFromElasticSource_then_shouldCloseAllCreatedClients.

@Test
public void when_readFromElasticSource_then_shouldCloseAllCreatedClients() throws IOException {
    indexDocument("my-index", of("name", "Frantisek"));
    Pipeline p = Pipeline.create();
    BatchSource<String> source = new ElasticSourceBuilder<>().clientFn(() -> {
        RestClientBuilder builder = spy(ElasticSupport.elasticClientSupplier().get());
        when(builder.build()).thenAnswer(invocation -> {
            Object result = invocation.callRealMethod();
            RestClient elasticClient = (RestClient) spy(result);
            ClientHolder.elasticClients.add(elasticClient);
            return elasticClient;
        });
        return builder;
    }).searchRequestFn(() -> new SearchRequest("my-index")).mapToItemFn(hit -> (String) hit.getSourceAsMap().get("name")).build();
    p.readFrom(source).writeTo(Sinks.logger());
    submitJob(p);
    for (RestClient elasticClient : ClientHolder.elasticClients) {
        verify(elasticClient).close();
    }
}