Search in sources :

Example 6 with JobTaskEvent

use of org.apache.hadoop.mapreduce.v2.app.job.event.JobTaskEvent in project hadoop by apache.

the class TestJobImpl method testAbortJobCalledAfterKillingTasks.

@Test
public void testAbortJobCalledAfterKillingTasks() throws IOException {
    Configuration conf = new Configuration();
    conf.set(MRJobConfig.MR_AM_STAGING_DIR, stagingDir);
    conf.set(MRJobConfig.MR_AM_COMMITTER_CANCEL_TIMEOUT_MS, "1000");
    InlineDispatcher dispatcher = new InlineDispatcher();
    dispatcher.init(conf);
    dispatcher.start();
    OutputCommitter committer = Mockito.mock(OutputCommitter.class);
    CommitterEventHandler commitHandler = createCommitterEventHandler(dispatcher, committer);
    commitHandler.init(conf);
    commitHandler.start();
    JobImpl job = createRunningStubbedJob(conf, dispatcher, 2, null);
    //Fail one task. This should land the JobImpl in the FAIL_WAIT state
    job.handle(new JobTaskEvent(MRBuilderUtils.newTaskId(job.getID(), 1, TaskType.MAP), TaskState.FAILED));
    //Verify abort job hasn't been called
    Mockito.verify(committer, Mockito.never()).abortJob((JobContext) Mockito.any(), (State) Mockito.any());
    assertJobState(job, JobStateInternal.FAIL_WAIT);
    //Verify abortJob is called once and the job failed
    Mockito.verify(committer, Mockito.timeout(2000).times(1)).abortJob((JobContext) Mockito.any(), (State) Mockito.any());
    assertJobState(job, JobStateInternal.FAILED);
    dispatcher.stop();
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) Configuration(org.apache.hadoop.conf.Configuration) JobTaskEvent(org.apache.hadoop.mapreduce.v2.app.job.event.JobTaskEvent) InlineDispatcher(org.apache.hadoop.yarn.event.InlineDispatcher) CommitterEventHandler(org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler) Test(org.junit.Test)

Aggregations

JobTaskEvent (org.apache.hadoop.mapreduce.v2.app.job.event.JobTaskEvent)6 JobHistoryEvent (org.apache.hadoop.mapreduce.jobhistory.JobHistoryEvent)3 ArrayList (java.util.ArrayList)2 Configuration (org.apache.hadoop.conf.Configuration)2 OutputCommitter (org.apache.hadoop.mapreduce.OutputCommitter)2 TaskAttemptId (org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId)2 CommitterEventHandler (org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler)2 TaskAttempt (org.apache.hadoop.mapreduce.v2.app.job.TaskAttempt)2 TaskAttemptEvent (org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEvent)2 Test (org.junit.Test)2 HashMap (java.util.HashMap)1 Map (java.util.Map)1 CyclicBarrier (java.util.concurrent.CyclicBarrier)1 TaskAttemptID (org.apache.hadoop.mapreduce.TaskAttemptID)1 Event (org.apache.hadoop.mapreduce.jobhistory.Event)1 TaskAttemptInfo (org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.TaskAttemptInfo)1 TaskFailedEvent (org.apache.hadoop.mapreduce.jobhistory.TaskFailedEvent)1 TaskFinishedEvent (org.apache.hadoop.mapreduce.jobhistory.TaskFinishedEvent)1 TaskAttemptCompletionEvent (org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptCompletionEvent)1 TaskAttemptCompletionEventStatus (org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptCompletionEventStatus)1