Search in sources :

Example 21 with PortablePipelineOptions

use of org.apache.beam.sdk.options.PortablePipelineOptions in project beam by apache.

the class ExpansionService method createPipeline.

protected Pipeline createPipeline() {
    // TODO: [BEAM-12599]: implement proper validation
    PipelineOptions effectiveOpts = PipelineOptionsFactory.create();
    PortablePipelineOptions portableOptions = effectiveOpts.as(PortablePipelineOptions.class);
    PortablePipelineOptions specifiedOptions = pipelineOptions.as(PortablePipelineOptions.class);
    Optional.ofNullable(specifiedOptions.getDefaultEnvironmentType()).ifPresent(portableOptions::setDefaultEnvironmentType);
    Optional.ofNullable(specifiedOptions.getDefaultEnvironmentConfig()).ifPresent(portableOptions::setDefaultEnvironmentConfig);
    effectiveOpts.as(ExperimentalOptions.class).setExperiments(pipelineOptions.as(ExperimentalOptions.class).getExperiments());
    effectiveOpts.setRunner(NotRunnableRunner.class);
    return Pipeline.create(effectiveOpts);
}
Also used : PortablePipelineOptions(org.apache.beam.sdk.options.PortablePipelineOptions) PipelineOptions(org.apache.beam.sdk.options.PipelineOptions) PortablePipelineOptions(org.apache.beam.sdk.options.PortablePipelineOptions) ExperimentalOptions(org.apache.beam.sdk.options.ExperimentalOptions)

Aggregations

PortablePipelineOptions (org.apache.beam.sdk.options.PortablePipelineOptions)21 Test (org.junit.Test)10 Struct (org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.Struct)4 IOException (java.io.IOException)3 StateRequestHandler (org.apache.beam.runners.fnexecution.state.StateRequestHandler)3 ExperimentalOptions (org.apache.beam.sdk.options.ExperimentalOptions)3 ByteString (org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString)3 Optional (java.util.Optional)2 Environment (org.apache.beam.model.pipeline.v1.RunnerApi.Environment)2 EnvironmentFactory (org.apache.beam.runners.fnexecution.environment.EnvironmentFactory)2 Provider (org.apache.beam.runners.fnexecution.environment.EnvironmentFactory.Provider)2 RemoteEnvironment (org.apache.beam.runners.fnexecution.environment.RemoteEnvironment)2 ServerFactory (org.apache.beam.sdk.fn.server.ServerFactory)2 Matchers.containsString (org.hamcrest.Matchers.containsString)2 ByteArrayOutputStream (java.io.ByteArrayOutputStream)1 File (java.io.File)1 FileOutputStream (java.io.FileOutputStream)1 Arrays (java.util.Arrays)1 Collection (java.util.Collection)1 Collections (java.util.Collections)1