Examples with JobInfo org.apache.beam.runners.fnexecution.provisioning.JobInfo used on opensource projects

Example 11 with JobInfo

use of org.apache.beam.runners.fnexecution.provisioning.JobInfo in project beam by apache.

the class SparkJobInvoker method createJobInvocation.

static JobInvocation createJobInvocation(String invocationId, String retrievalToken, ListeningExecutorService executorService, Pipeline pipeline, SparkPipelineOptions sparkOptions) {
    JobInfo jobInfo = JobInfo.create(invocationId, sparkOptions.getJobName(), retrievalToken, PipelineOptionsTranslation.toProto(sparkOptions));
    PortablePipelineRunner pipelineRunner;
    if (Strings.isNullOrEmpty(sparkOptions.as(PortablePipelineOptions.class).getOutputExecutablePath())) {
        pipelineRunner = new SparkPipelineRunner(sparkOptions);
    } else {
        pipelineRunner = new PortablePipelineJarCreator(SparkPipelineRunner.class);
    }
    return new JobInvocation(jobInfo, executorService, pipeline, pipelineRunner);
}