use of com.google.cloud.teleport.v2.elasticsearch.options.BigQueryToElasticsearchOptions in project DataflowTemplates by GoogleCloudPlatform.
the class BigQueryToElasticsearch method main.
/**
* Main entry point for pipeline execution.
*
* @param args Command line arguments to the pipeline.
*/
public static void main(String[] args) {
BigQueryToElasticsearchOptions options = PipelineOptionsFactory.fromArgs(args).withValidation().as(BigQueryToElasticsearchOptions.class);
run(options);
}
use of com.google.cloud.teleport.v2.elasticsearch.options.BigQueryToElasticsearchOptions in project DataflowTemplates by GoogleCloudPlatform.
the class BigQueryToElasticsearch method run.
/**
* Runs the pipeline with the supplied options.
*
* @param options The execution parameters to the pipeline.
* @return The result of the pipeline execution.
*/
private static PipelineResult run(BigQueryToElasticsearchOptions options) {
// Create the pipeline.
Pipeline pipeline = Pipeline.create(options);
/*
* Steps: 1) Read records from BigQuery via BigQueryIO.
* 2) Create json string from Table Row.
* 3) Write records to Elasticsearch.
*
*
* Step #1: Read from BigQuery. If a query is provided then it is used to get the TableRows.
*/
pipeline.apply("ReadFromBigQuery", ReadBigQuery.newBuilder().setOptions(options.as(BigQueryToElasticsearchOptions.class)).build()).apply("TableRowsToJsonDocument", ParDo.of(new TableRowToJsonFn())).apply("WriteToElasticsearch", WriteToElasticsearch.newBuilder().setOptions(options.as(BigQueryToElasticsearchOptions.class)).build());
return pipeline.run();
}
Aggregations