Examples with BulkIO org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.BulkIO used on opensource projects

Example 1 with BulkIO

use of org.apache.beam.sdk.io.elasticsearch.ElasticsearchIO.BulkIO in project beam by apache.

the class ElasticsearchIOTestCommon method testDocToBulkAndBulkIO.

/**
 * Test that DocToBulk and BulkIO can be constructed and operate independently of Write.
 */
void testDocToBulkAndBulkIO() throws Exception {
    DocToBulk docToBulk = ElasticsearchIO.docToBulk().withConnectionConfiguration(connectionConfiguration);
    BulkIO bulkIO = ElasticsearchIO.bulkIO().withConnectionConfiguration(connectionConfiguration);
    List<String> data = ElasticsearchIOTestUtils.createDocuments(numDocs, ElasticsearchIOTestUtils.InjectionMode.DO_NOT_INJECT_INVALID_DOCS);
    pipeline.apply(Create.of(data)).apply(docToBulk).apply(bulkIO);
    pipeline.run();
    long currentNumDocs = refreshIndexAndGetCurrentNumDocs(connectionConfiguration, restClient);
    assertEquals(numDocs, currentNumDocs);
    int count = countByScientistName(connectionConfiguration, restClient, "Einstein", null);
    assertEquals(numDocs / NUM_SCIENTISTS, count);
}