Search in sources :

Example 1 with ListJobsResponse

use of com.google.api.services.dataflow.model.ListJobsResponse in project beam by apache.

the class DataflowRunner method getJobIdFromName.

/**
   * Finds the id for the running job of the given name.
   */
private String getJobIdFromName(String jobName) {
    try {
        ListJobsResponse listResult;
        String token = null;
        do {
            listResult = dataflowClient.listJobs(token);
            token = listResult.getNextPageToken();
            for (Job job : listResult.getJobs()) {
                if (job.getName().equals(jobName) && MonitoringUtil.toState(job.getCurrentState()).equals(State.RUNNING)) {
                    return job.getId();
                }
            }
        } while (token != null);
    } catch (GoogleJsonResponseException e) {
        throw new RuntimeException("Got error while looking up jobs: " + (e.getDetails() != null ? e.getDetails().getMessage() : e), e);
    } catch (IOException e) {
        throw new RuntimeException("Got error while looking up jobs: ", e);
    }
    throw new IllegalArgumentException("Could not find running job named " + jobName);
}
Also used : GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) ListJobsResponse(com.google.api.services.dataflow.model.ListJobsResponse) StringUtils.byteArrayToJsonString(org.apache.beam.sdk.util.StringUtils.byteArrayToJsonString) IOException(java.io.IOException) DataflowTemplateJob(org.apache.beam.runners.dataflow.util.DataflowTemplateJob) Job(com.google.api.services.dataflow.model.Job)

Example 2 with ListJobsResponse

use of com.google.api.services.dataflow.model.ListJobsResponse in project beam by apache.

the class DataflowRunnerTest method buildMockDataflow.

private Dataflow buildMockDataflow() throws IOException {
    Dataflow mockDataflowClient = mock(Dataflow.class);
    Dataflow.Projects mockProjects = mock(Dataflow.Projects.class);
    Dataflow.Projects.Locations mockLocations = mock(Dataflow.Projects.Locations.class);
    Dataflow.Projects.Locations.Jobs.Create mockRequest = mock(Dataflow.Projects.Locations.Jobs.Create.class);
    Dataflow.Projects.Locations.Jobs.List mockList = mock(Dataflow.Projects.Locations.Jobs.List.class);
    when(mockDataflowClient.projects()).thenReturn(mockProjects);
    when(mockProjects.locations()).thenReturn(mockLocations);
    when(mockLocations.jobs()).thenReturn(mockJobs);
    when(mockJobs.create(eq(PROJECT_ID), eq(REGION_ID), isA(Job.class))).thenReturn(mockRequest);
    when(mockJobs.list(eq(PROJECT_ID), eq(REGION_ID))).thenReturn(mockList);
    when(mockList.setPageToken(anyString())).thenReturn(mockList);
    when(mockList.execute()).thenReturn(new ListJobsResponse().setJobs(Arrays.asList(new Job().setName("oldjobname").setId("oldJobId").setCurrentState("JOB_STATE_RUNNING"))));
    Job resultJob = new Job();
    resultJob.setId("newid");
    when(mockRequest.execute()).thenReturn(resultJob);
    return mockDataflowClient;
}
Also used : ListJobsResponse(com.google.api.services.dataflow.model.ListJobsResponse) Job(com.google.api.services.dataflow.model.Job) DataflowRunner.getContainerImageForJob(org.apache.beam.runners.dataflow.DataflowRunner.getContainerImageForJob) Dataflow(com.google.api.services.dataflow.Dataflow)

Aggregations

Job (com.google.api.services.dataflow.model.Job)2 ListJobsResponse (com.google.api.services.dataflow.model.ListJobsResponse)2 GoogleJsonResponseException (com.google.api.client.googleapis.json.GoogleJsonResponseException)1 Dataflow (com.google.api.services.dataflow.Dataflow)1 IOException (java.io.IOException)1 DataflowRunner.getContainerImageForJob (org.apache.beam.runners.dataflow.DataflowRunner.getContainerImageForJob)1 DataflowTemplateJob (org.apache.beam.runners.dataflow.util.DataflowTemplateJob)1 StringUtils.byteArrayToJsonString (org.apache.beam.sdk.util.StringUtils.byteArrayToJsonString)1