Search in sources :

Example 1 with GetJobReportRequest

use of org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest in project hadoop by apache.

the class TestClientServiceDelegate method testRemoteExceptionFromHistoryServer.

@Test
public void testRemoteExceptionFromHistoryServer() throws Exception {
    MRClientProtocol historyServerProxy = mock(MRClientProtocol.class);
    when(historyServerProxy.getJobReport(getJobReportRequest())).thenThrow(new IOException("Job ID doesnot Exist"));
    ResourceMgrDelegate rm = mock(ResourceMgrDelegate.class);
    when(rm.getApplicationReport(TypeConverter.toYarn(oldJobId).getAppId())).thenReturn(null);
    ClientServiceDelegate clientServiceDelegate = getClientServiceDelegate(historyServerProxy, rm);
    try {
        clientServiceDelegate.getJobStatus(oldJobId);
        Assert.fail("Invoke should throw exception after retries.");
    } catch (IOException e) {
        Assert.assertTrue(e.getMessage().contains("Job ID doesnot Exist"));
    }
}
Also used : IOException(java.io.IOException) MRClientProtocol(org.apache.hadoop.mapreduce.v2.api.MRClientProtocol) Test(org.junit.Test)

Example 2 with GetJobReportRequest

use of org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest in project hadoop by apache.

the class TestClientServiceDelegate method testJobReportFromHistoryServer.

@Test
public void testJobReportFromHistoryServer() throws Exception {
    MRClientProtocol historyServerProxy = mock(MRClientProtocol.class);
    when(historyServerProxy.getJobReport(getJobReportRequest())).thenReturn(getJobReportResponseFromHistoryServer());
    ResourceMgrDelegate rm = mock(ResourceMgrDelegate.class);
    when(rm.getApplicationReport(TypeConverter.toYarn(oldJobId).getAppId())).thenReturn(null);
    ClientServiceDelegate clientServiceDelegate = getClientServiceDelegate(historyServerProxy, rm);
    JobStatus jobStatus = clientServiceDelegate.getJobStatus(oldJobId);
    Assert.assertNotNull(jobStatus);
    Assert.assertEquals("TestJobFilePath", jobStatus.getJobFile());
    Assert.assertEquals("http://TestTrackingUrl", jobStatus.getTrackingUrl());
    Assert.assertEquals(1.0f, jobStatus.getMapProgress(), 0.0f);
    Assert.assertEquals(1.0f, jobStatus.getReduceProgress(), 0.0f);
}
Also used : JobStatus(org.apache.hadoop.mapreduce.JobStatus) MRClientProtocol(org.apache.hadoop.mapreduce.v2.api.MRClientProtocol) Test(org.junit.Test)

Example 3 with GetJobReportRequest

use of org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest in project hadoop by apache.

the class TestClientServiceDelegate method getJobReportRequest.

private GetJobReportRequest getJobReportRequest() {
    GetJobReportRequest request = Records.newRecord(GetJobReportRequest.class);
    request.setJobId(jobId);
    return request;
}
Also used : GetJobReportRequest(org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest)

Example 4 with GetJobReportRequest

use of org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest in project hadoop by apache.

the class TestClientServiceDelegate method testUnknownAppInRM.

@Test
public void testUnknownAppInRM() throws Exception {
    MRClientProtocol historyServerProxy = mock(MRClientProtocol.class);
    when(historyServerProxy.getJobReport(getJobReportRequest())).thenReturn(getJobReportResponse());
    ClientServiceDelegate clientServiceDelegate = getClientServiceDelegate(historyServerProxy, getRMDelegate());
    JobStatus jobStatus = clientServiceDelegate.getJobStatus(oldJobId);
    Assert.assertNotNull(jobStatus);
}
Also used : JobStatus(org.apache.hadoop.mapreduce.JobStatus) MRClientProtocol(org.apache.hadoop.mapreduce.v2.api.MRClientProtocol) Test(org.junit.Test)

Example 5 with GetJobReportRequest

use of org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest in project hadoop by apache.

the class TestClientServiceDelegate method testRetriesOnConnectionFailure.

@Test
public void testRetriesOnConnectionFailure() throws Exception {
    MRClientProtocol historyServerProxy = mock(MRClientProtocol.class);
    when(historyServerProxy.getJobReport(getJobReportRequest())).thenThrow(new RuntimeException("1")).thenThrow(new RuntimeException("2")).thenReturn(getJobReportResponse());
    ResourceMgrDelegate rm = mock(ResourceMgrDelegate.class);
    when(rm.getApplicationReport(TypeConverter.toYarn(oldJobId).getAppId())).thenReturn(null);
    ClientServiceDelegate clientServiceDelegate = getClientServiceDelegate(historyServerProxy, rm);
    JobStatus jobStatus = clientServiceDelegate.getJobStatus(oldJobId);
    Assert.assertNotNull(jobStatus);
    verify(historyServerProxy, times(3)).getJobReport(any(GetJobReportRequest.class));
}
Also used : JobStatus(org.apache.hadoop.mapreduce.JobStatus) GetJobReportRequest(org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest) MRClientProtocol(org.apache.hadoop.mapreduce.v2.api.MRClientProtocol) Test(org.junit.Test)

Aggregations

Test (org.junit.Test)8 MRClientProtocol (org.apache.hadoop.mapreduce.v2.api.MRClientProtocol)7 GetJobReportRequest (org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportRequest)7 IOException (java.io.IOException)5 JobStatus (org.apache.hadoop.mapreduce.JobStatus)5 JobReport (org.apache.hadoop.mapreduce.v2.api.records.JobReport)5 GetJobReportResponse (org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetJobReportResponse)3 Configuration (org.apache.hadoop.conf.Configuration)2 GetTaskAttemptReportRequest (org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetTaskAttemptReportRequest)2 TaskAttemptReport (org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptReport)2 File (java.io.File)1 InetSocketAddress (java.net.InetSocketAddress)1 JobConf (org.apache.hadoop.mapred.JobConf)1 Counters (org.apache.hadoop.mapreduce.Counters)1 Job (org.apache.hadoop.mapreduce.Job)1 SleepJob (org.apache.hadoop.mapreduce.SleepJob)1 LogParams (org.apache.hadoop.mapreduce.v2.LogParams)1 HSClientProtocol (org.apache.hadoop.mapreduce.v2.api.HSClientProtocol)1 GetCountersRequest (org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetCountersRequest)1 GetDiagnosticsRequest (org.apache.hadoop.mapreduce.v2.api.protocolrecords.GetDiagnosticsRequest)1