Search in sources :

Example 1 with ApiErrorExtractor

use of com.google.cloud.hadoop.util.ApiErrorExtractor in project beam by apache.

the class BigQueryServicesImplTest method testStartLoadJobRetry.

/**
   * Tests that {@link BigQueryServicesImpl.JobServiceImpl#startLoadJob} succeeds with a retry.
   */
@Test
public void testStartLoadJobRetry() throws IOException, InterruptedException {
    Job testJob = new Job();
    JobReference jobRef = new JobReference();
    jobRef.setJobId("jobId");
    jobRef.setProjectId("projectId");
    testJob.setJobReference(jobRef);
    // First response is 403 rate limited, second response has valid payload.
    when(response.getContentType()).thenReturn(Json.MEDIA_TYPE);
    when(response.getStatusCode()).thenReturn(403).thenReturn(200);
    when(response.getContent()).thenReturn(toStream(errorWithReasonAndStatus("rateLimitExceeded", 403))).thenReturn(toStream(testJob));
    Sleeper sleeper = new FastNanoClockAndSleeper();
    JobServiceImpl.startJob(testJob, new ApiErrorExtractor(), bigquery, sleeper, BackOffAdapter.toGcpBackOff(FluentBackoff.DEFAULT.backoff()));
    verify(response, times(2)).getStatusCode();
    verify(response, times(2)).getContent();
    verify(response, times(2)).getContentType();
}
Also used : JobReference(com.google.api.services.bigquery.model.JobReference) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) MockSleeper(com.google.api.client.testing.util.MockSleeper) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) Sleeper(com.google.api.client.util.Sleeper) Job(com.google.api.services.bigquery.model.Job) ApiErrorExtractor(com.google.cloud.hadoop.util.ApiErrorExtractor) Test(org.junit.Test)

Example 2 with ApiErrorExtractor

use of com.google.cloud.hadoop.util.ApiErrorExtractor in project beam by apache.

the class BigQueryServicesImplTest method testStartLoadJobSucceeds.

/**
   * Tests that {@link BigQueryServicesImpl.JobServiceImpl#startLoadJob} succeeds.
   */
@Test
public void testStartLoadJobSucceeds() throws IOException, InterruptedException {
    Job testJob = new Job();
    JobReference jobRef = new JobReference();
    jobRef.setJobId("jobId");
    jobRef.setProjectId("projectId");
    testJob.setJobReference(jobRef);
    when(response.getContentType()).thenReturn(Json.MEDIA_TYPE);
    when(response.getStatusCode()).thenReturn(200);
    when(response.getContent()).thenReturn(toStream(testJob));
    Sleeper sleeper = new FastNanoClockAndSleeper();
    JobServiceImpl.startJob(testJob, new ApiErrorExtractor(), bigquery, sleeper, BackOffAdapter.toGcpBackOff(FluentBackoff.DEFAULT.backoff()));
    verify(response, times(1)).getStatusCode();
    verify(response, times(1)).getContent();
    verify(response, times(1)).getContentType();
    expectedLogs.verifyInfo(String.format("Started BigQuery job: %s", jobRef));
}
Also used : JobReference(com.google.api.services.bigquery.model.JobReference) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) MockSleeper(com.google.api.client.testing.util.MockSleeper) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) Sleeper(com.google.api.client.util.Sleeper) Job(com.google.api.services.bigquery.model.Job) ApiErrorExtractor(com.google.cloud.hadoop.util.ApiErrorExtractor) Test(org.junit.Test)

Example 3 with ApiErrorExtractor

use of com.google.cloud.hadoop.util.ApiErrorExtractor in project beam by apache.

the class BigQueryServicesImplTest method testStartLoadJobSucceedsAlreadyExists.

/**
   * Tests that {@link BigQueryServicesImpl.JobServiceImpl#startLoadJob} succeeds
   * with an already exist job.
   */
@Test
public void testStartLoadJobSucceedsAlreadyExists() throws IOException, InterruptedException {
    Job testJob = new Job();
    JobReference jobRef = new JobReference();
    jobRef.setJobId("jobId");
    jobRef.setProjectId("projectId");
    testJob.setJobReference(jobRef);
    // 409 means already exists
    when(response.getStatusCode()).thenReturn(409);
    Sleeper sleeper = new FastNanoClockAndSleeper();
    JobServiceImpl.startJob(testJob, new ApiErrorExtractor(), bigquery, sleeper, BackOffAdapter.toGcpBackOff(FluentBackoff.DEFAULT.backoff()));
    verify(response, times(1)).getStatusCode();
    verify(response, times(1)).getContent();
    verify(response, times(1)).getContentType();
    expectedLogs.verifyNotLogged("Started BigQuery job");
}
Also used : JobReference(com.google.api.services.bigquery.model.JobReference) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) MockSleeper(com.google.api.client.testing.util.MockSleeper) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) Sleeper(com.google.api.client.util.Sleeper) Job(com.google.api.services.bigquery.model.Job) ApiErrorExtractor(com.google.cloud.hadoop.util.ApiErrorExtractor) Test(org.junit.Test)

Aggregations

MockSleeper (com.google.api.client.testing.util.MockSleeper)3 Sleeper (com.google.api.client.util.Sleeper)3 Job (com.google.api.services.bigquery.model.Job)3 JobReference (com.google.api.services.bigquery.model.JobReference)3 ApiErrorExtractor (com.google.cloud.hadoop.util.ApiErrorExtractor)3 FastNanoClockAndSleeper (org.apache.beam.sdk.util.FastNanoClockAndSleeper)3 Test (org.junit.Test)3