Search in sources :

Example 1 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class MonitoringUtilTest method testOverridesEndpointWithStagedDataflowEndpoint.

@Test
public void testOverridesEndpointWithStagedDataflowEndpoint() {
    DataflowPipelineOptions options = PipelineOptionsFactory.create().as(DataflowPipelineOptions.class);
    options.setProject(PROJECT_ID);
    options.setGcpCredential(new TestCredential());
    String stagingDataflowEndpoint = "v0neverExisted";
    options.setDataflowEndpoint(stagingDataflowEndpoint);
    String cancelCommand = MonitoringUtil.getGcloudCancelCommand(options, JOB_ID);
    assertEquals("CLOUDSDK_API_ENDPOINT_OVERRIDES_DATAFLOW=https://dataflow.googleapis.com/v0neverExisted/ " + "gcloud beta dataflow jobs --project=someProject cancel 1234", cancelCommand);
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) Test(org.junit.Test)

Example 2 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class GcsPathValidatorTest method setUp.

@Before
public void setUp() throws Exception {
    MockitoAnnotations.initMocks(this);
    when(mockGcsUtil.bucketAccessible(any(GcsPath.class))).thenReturn(true);
    GcsOptions options = PipelineOptionsFactory.as(GcsOptions.class);
    options.setGcpCredential(new TestCredential());
    options.setGcsUtil(mockGcsUtil);
    validator = GcsPathValidator.fromOptions(options);
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) GcsPath(org.apache.beam.sdk.util.gcsfs.GcsPath) GcsOptions(org.apache.beam.sdk.extensions.gcp.options.GcsOptions) Before(org.junit.Before)

Example 3 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class DataflowRunnerTest method testToString.

@Test
public void testToString() {
    DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
    options.setJobName("TestJobName");
    options.setProject("test-project");
    options.setTempLocation("gs://test/temp/location");
    options.setGcpCredential(new TestCredential());
    options.setPathValidatorClass(NoopPathValidator.class);
    options.setRunner(DataflowRunner.class);
    assertEquals("DataflowRunner#testjobname", DataflowRunner.fromOptions(options).toString());
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) Test(org.junit.Test)

Example 4 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class DataflowRunnerTest method testGcpTempAndNoTempLocationSucceeds.

@Test
public void testGcpTempAndNoTempLocationSucceeds() throws Exception {
    DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
    options.setRunner(DataflowRunner.class);
    options.setGcpCredential(new TestCredential());
    options.setProject("foo-project");
    options.setGcpTempLocation(VALID_TEMP_BUCKET);
    options.setGcsUtil(mockGcsUtil);
    DataflowRunner.fromOptions(options);
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) Test(org.junit.Test)

Example 5 with TestCredential

use of org.apache.beam.sdk.extensions.gcp.auth.TestCredential in project beam by apache.

the class DataflowRunnerTest method testTempLocationAndNoGcpTempLocationSucceeds.

@Test
public void testTempLocationAndNoGcpTempLocationSucceeds() throws Exception {
    DataflowPipelineOptions options = PipelineOptionsFactory.as(DataflowPipelineOptions.class);
    options.setRunner(DataflowRunner.class);
    options.setGcpCredential(new TestCredential());
    options.setProject("foo-project");
    options.setTempLocation(VALID_TEMP_BUCKET);
    options.setGcsUtil(mockGcsUtil);
    DataflowRunner.fromOptions(options);
}
Also used : TestCredential(org.apache.beam.sdk.extensions.gcp.auth.TestCredential) DataflowPipelineOptions(org.apache.beam.runners.dataflow.options.DataflowPipelineOptions) Test(org.junit.Test)

Aggregations

TestCredential (org.apache.beam.sdk.extensions.gcp.auth.TestCredential)28 Test (org.junit.Test)19 DataflowPipelineOptions (org.apache.beam.runners.dataflow.options.DataflowPipelineOptions)14 Before (org.junit.Before)5 GcsPath (org.apache.beam.sdk.util.gcsfs.GcsPath)4 Pipeline (org.apache.beam.sdk.Pipeline)3 TestPipeline (org.apache.beam.sdk.testing.TestPipeline)3 InvocationOnMock (org.mockito.invocation.InvocationOnMock)3 BigtableOptions (com.google.cloud.bigtable.config.BigtableOptions)2 ImmutableList (com.google.common.collect.ImmutableList)2 File (java.io.File)2 LinkedList (java.util.LinkedList)2 List (java.util.List)2 GcpOptions (org.apache.beam.sdk.extensions.gcp.options.GcpOptions)2 GcsOptions (org.apache.beam.sdk.extensions.gcp.options.GcsOptions)2 GoogleApiTracer (org.apache.beam.sdk.extensions.gcp.options.GoogleApiDebugOptions.GoogleApiTracer)2 GcsUtil (org.apache.beam.sdk.util.GcsUtil)2 TupleTagList (org.apache.beam.sdk.values.TupleTagList)2 Matchers.containsString (org.hamcrest.Matchers.containsString)2 Matchers.anyString (org.mockito.Matchers.anyString)2