Search in sources :

Example 1 with InitTransition

use of org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.InitTransition in project hadoop by apache.

the class TestJobImpl method testMetaInfoSizeOverMax.

@Test
public void testMetaInfoSizeOverMax() throws Exception {
    Configuration conf = new Configuration();
    JobID jobID = JobID.forName("job_1234567890000_0001");
    JobId jobId = TypeConverter.toYarn(jobID);
    MRAppMetrics mrAppMetrics = MRAppMetrics.create();
    JobImpl job = new JobImpl(jobId, ApplicationAttemptId.newInstance(ApplicationId.newInstance(0, 0), 0), conf, mock(EventHandler.class), null, new JobTokenSecretManager(), new Credentials(), null, null, mrAppMetrics, null, true, null, 0, null, null, null, null);
    InitTransition initTransition = new InitTransition() {

        @Override
        protected TaskSplitMetaInfo[] createSplits(JobImpl job, JobId jobId) {
            throw new YarnRuntimeException(EXCEPTIONMSG);
        }
    };
    JobEvent mockJobEvent = mock(JobEvent.class);
    JobStateInternal jobSI = initTransition.transition(job, mockJobEvent);
    Assert.assertTrue("When init fails, return value from InitTransition.transition should equal NEW.", jobSI.equals(JobStateInternal.NEW));
    Assert.assertTrue("Job diagnostics should contain YarnRuntimeException", job.getDiagnostics().toString().contains("YarnRuntimeException"));
    Assert.assertTrue("Job diagnostics should contain " + EXCEPTIONMSG, job.getDiagnostics().toString().contains(EXCEPTIONMSG));
}
Also used : Configuration(org.apache.hadoop.conf.Configuration) JobStateInternal(org.apache.hadoop.mapreduce.v2.app.job.JobStateInternal) CommitterEventHandler(org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler) EventHandler(org.apache.hadoop.yarn.event.EventHandler) YarnRuntimeException(org.apache.hadoop.yarn.exceptions.YarnRuntimeException) InitTransition(org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.InitTransition) JobEvent(org.apache.hadoop.mapreduce.v2.app.job.event.JobEvent) JobTokenSecretManager(org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager) TaskSplitMetaInfo(org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitMetaInfo) MRAppMetrics(org.apache.hadoop.mapreduce.v2.app.metrics.MRAppMetrics) JobID(org.apache.hadoop.mapreduce.JobID) JobId(org.apache.hadoop.mapreduce.v2.api.records.JobId) Credentials(org.apache.hadoop.security.Credentials) Test(org.junit.Test)

Example 2 with InitTransition

use of org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.InitTransition in project hadoop by apache.

the class TestJobImpl method testUberDecision.

private boolean testUberDecision(Configuration conf) {
    JobID jobID = JobID.forName("job_1234567890000_0001");
    JobId jobId = TypeConverter.toYarn(jobID);
    MRAppMetrics mrAppMetrics = MRAppMetrics.create();
    JobImpl job = new JobImpl(jobId, ApplicationAttemptId.newInstance(ApplicationId.newInstance(0, 0), 0), conf, mock(EventHandler.class), null, new JobTokenSecretManager(), new Credentials(), null, null, mrAppMetrics, null, true, null, 0, null, null, null, null);
    InitTransition initTransition = getInitTransition(2);
    JobEvent mockJobEvent = mock(JobEvent.class);
    initTransition.transition(job, mockJobEvent);
    boolean isUber = job.isUber();
    return isUber;
}
Also used : InitTransition(org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.InitTransition) JobEvent(org.apache.hadoop.mapreduce.v2.app.job.event.JobEvent) JobTokenSecretManager(org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager) CommitterEventHandler(org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler) EventHandler(org.apache.hadoop.yarn.event.EventHandler) MRAppMetrics(org.apache.hadoop.mapreduce.v2.app.metrics.MRAppMetrics) JobID(org.apache.hadoop.mapreduce.JobID) JobId(org.apache.hadoop.mapreduce.v2.api.records.JobId) Credentials(org.apache.hadoop.security.Credentials)

Aggregations

JobID (org.apache.hadoop.mapreduce.JobID)2 JobTokenSecretManager (org.apache.hadoop.mapreduce.security.token.JobTokenSecretManager)2 JobId (org.apache.hadoop.mapreduce.v2.api.records.JobId)2 CommitterEventHandler (org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler)2 JobEvent (org.apache.hadoop.mapreduce.v2.app.job.event.JobEvent)2 InitTransition (org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.InitTransition)2 MRAppMetrics (org.apache.hadoop.mapreduce.v2.app.metrics.MRAppMetrics)2 Credentials (org.apache.hadoop.security.Credentials)2 EventHandler (org.apache.hadoop.yarn.event.EventHandler)2 Configuration (org.apache.hadoop.conf.Configuration)1 TaskSplitMetaInfo (org.apache.hadoop.mapreduce.split.JobSplit.TaskSplitMetaInfo)1 JobStateInternal (org.apache.hadoop.mapreduce.v2.app.job.JobStateInternal)1 YarnRuntimeException (org.apache.hadoop.yarn.exceptions.YarnRuntimeException)1 Test (org.junit.Test)1