Search in sources :

Example 26 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class DataflowPipelineTranslatorTest method buildPipelineOptions.

private static DataflowPipelineOptions buildPipelineOptions() throws IOException {
    GcsUtil mockGcsUtil = mock(GcsUtil.class);
    when(mockGcsUtil.expand(any(GcsPath.class))).then(new Answer<List<GcsPath>>() {

        @Override
        public List<GcsPath> answer(InvocationOnMock invocation) throws Throwable {
            return ImmutableList.of((GcsPath) invocation.getArguments()[0]);
        }
    });
    when(mockGcsUtil.bucketAccessible(any(GcsPath.class))).thenReturn(true);
    DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
    options.setRunner(DataflowRunner.class);
    options.setGcpCredential(new TestCredential());
    options.setJobName("some-job-name");
    options.setProject("some-project");
    options.setTempLocation(GcsPath.fromComponents("somebucket", "some/path").toString());
    options.setFilesToStage(new LinkedList<String>());
    options.setDataflowClient(buildMockDataflow(new IsValidCreateRequest()));
    options.setGcsUtil(mockGcsUtil);
    return options;
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) InvocationOnMock(org.mockito.invocation.InvocationOnMock) GcsPath(org.apache.beam.sdk.util.gcsfs.GcsPath) List(java.util.List) TupleTagList(org.apache.beam.sdk.values.TupleTagList) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) LinkedList(java.util.LinkedList) Structs.getString(org.apache.beam.runners.dataflow.util.Structs.getString) GcsUtil(org.apache.beam.sdk.util.GcsUtil)

Example 27 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class BatchStatefulParDoOverridesTest method buildPipelineOptions.

private static DataflowPipelineOptions buildPipelineOptions() throws IOException {
    GcsUtil mockGcsUtil = mock(GcsUtil.class);
    when(mockGcsUtil.expand(any(GcsPath.class))).then(new Answer<List<GcsPath>>() {

        @Override
        public List<GcsPath> answer(InvocationOnMock invocation) throws Throwable {
            return ImmutableList.of((GcsPath) invocation.getArguments()[0]);
        }
    });
    when(mockGcsUtil.bucketAccessible(any(GcsPath.class))).thenReturn(true);
    DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
    options.setRunner(DataflowRunner.class);
    options.setGcpCredential(new TestCredential());
    options.setJobName("some-job-name");
    options.setProject("some-project");
    options.setTempLocation(GcsPath.fromComponents("somebucket", "some/path").toString());
    options.setFilesToStage(new LinkedList<String>());
    options.setGcsUtil(mockGcsUtil);
    return options;
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) InvocationOnMock(org.mockito.invocation.InvocationOnMock) GcsPath(org.apache.beam.sdk.util.gcsfs.GcsPath) TupleTagList(org.apache.beam.sdk.values.TupleTagList) ImmutableList(com.google.common.collect.ImmutableList) LinkedList(java.util.LinkedList) List(java.util.List) GcsUtil(org.apache.beam.sdk.util.GcsUtil)

Example 28 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class PubsubGrpcClientTest method setup.

@Before
public void setup() {
    channelName = String.format("%s-%s", PubsubGrpcClientTest.class.getName(), ThreadLocalRandom.current().nextInt());
    inProcessChannel = InProcessChannelBuilder.forName(channelName).directExecutor().build();
    testCredentials = new TestCredential();
    client = new PubsubGrpcClient(TIMESTAMP_ATTRIBUTE, ID_ATTRIBUTE, 10, inProcessChannel, testCredentials);
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) Before(org.junit.Before)

Aggregations

TestCredential (org.apache.beam.sdk.extensions.gcp.auth.TestCredential)28 Test (org.junit.Test)19 DataflowPipelineOptions (org.apache.beam.runners.dataflow.options.DataflowPipelineOptions)14 Before (org.junit.Before)5 GcsPath (org.apache.beam.sdk.util.gcsfs.GcsPath)4 Pipeline (org.apache.beam.sdk.Pipeline)3 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)3 InvocationOnMock (org.mockito.invocation.InvocationOnMock)3 BigtableOptions (com.google.cloud.bigtable.config.BigtableOptions)2 ImmutableList (com.google.common.collect.ImmutableList)2 File (java.io.File)2 LinkedList (java.util.LinkedList)2 List (java.util.List)2 GcpOptions (org.apache.beam.sdk.extensions.gcp.options.GcpOptions)2 GcsOptions (org.apache.beam.sdk.extensions.gcp.options.GcsOptions)2 GoogleApiTracer (org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions.GoogleApiTracer)2 GcsUtil (org.apache.beam.sdk.util.GcsUtil)2 TupleTagList (org.apache.beam.sdk.values.TupleTagList)2 Matchers.containsString (org.hamcrest.Matchers.containsString)2 Matchers.anyString (org.mockito.Matchers.anyString)2