Search in sources :

Example 1 with NanoClock

use of com.google.api.client.util.NanoClock in project beam by apache.

the class DataflowPipelineJobTest method testWaitUntilFinishNoRepeatedLogs.

/**
   * Tests that a {@link DataflowPipelineJob} does not duplicate messages.
   */
@Test
public void testWaitUntilFinishNoRepeatedLogs() throws Exception {
    DataflowPipelineJob job = new DataflowPipelineJob(mockDataflowClient, JOB_ID, options, null);
    Sleeper sleeper = new ZeroSleeper();
    NanoClock nanoClock = mock(NanoClock.class);
    Instant separatingTimestamp = new Instant(42L);
    JobMessage theMessage = infoMessage(separatingTimestamp, "nothing");
    MonitoringUtil mockMonitor = mock(MonitoringUtil.class);
    when(mockMonitor.getJobMessages(anyString(), anyLong())).thenReturn(ImmutableList.of(theMessage));
    // The Job just always reports "running" across all calls
    Job fakeJob = new Job();
    fakeJob.setCurrentState("JOB_STATE_RUNNING");
    when(mockDataflowClient.getJob(anyString())).thenReturn(fakeJob);
    // After waitUntilFinish the DataflowPipelineJob should record the latest message timestamp
    when(nanoClock.nanoTime()).thenReturn(0L).thenReturn(2000000000L);
    job.waitUntilFinish(Duration.standardSeconds(1), mockHandler, sleeper, nanoClock, mockMonitor);
    verify(mockHandler).process(ImmutableList.of(theMessage));
    // Second waitUntilFinish should request jobs with `separatingTimestamp` so the monitor
    // will only return new messages
    when(nanoClock.nanoTime()).thenReturn(3000000000L).thenReturn(6000000000L);
    job.waitUntilFinish(Duration.standardSeconds(1), mockHandler, sleeper, nanoClock, mockMonitor);
    verify(mockMonitor).getJobMessages(anyString(), eq(separatingTimestamp.getMillis()));
}
Also used : MonitoringUtil(org.apache.beam.runners.dataflow.util.MonitoringUtil) Instant(org.joda.time.Instant) JobMessage(com.google.api.services.dataflow.model.JobMessage) FastNanoClockAndSleeper(org.apache.beam.sdk.util.FastNanoClockAndSleeper) Sleeper(com.google.api.client.util.Sleeper) NanoClock(com.google.api.client.util.NanoClock) Job(com.google.api.services.dataflow.model.Job) Test(org.junit.Test)

Aggregations

NanoClock (com.google.api.client.util.NanoClock)1 Sleeper (com.google.api.client.util.Sleeper)1 Job (com.google.api.services.dataflow.model.Job)1 JobMessage (com.google.api.services.dataflow.model.JobMessage)1 MonitoringUtil (org.apache.beam.runners.dataflow.util.MonitoringUtil)1 FastNanoClockAndSleeper (org.apache.beam.sdk.util.FastNanoClockAndSleeper)1 Instant (org.joda.time.Instant)1 Test (org.junit.Test)1