Search in sources :

Example 11 with YarnUncaughtExceptionHandler

use of org.apache.hadoop.yarn.YarnUncaughtExceptionHandler in project hadoop by apache.

the class MRAppMaster method main.

public static void main(String[] args) {
    try {
        mainStarted = true;
        Thread.setDefaultUncaughtExceptionHandler(new YarnUncaughtExceptionHandler());
        String containerIdStr = System.getenv(Environment.CONTAINER_ID.name());
        String nodeHostString = System.getenv(Environment.NM_HOST.name());
        String nodePortString = System.getenv(Environment.NM_PORT.name());
        String nodeHttpPortString = System.getenv(Environment.NM_HTTP_PORT.name());
        String appSubmitTimeStr = System.getenv(ApplicationConstants.APP_SUBMIT_TIME_ENV);
        validateInputParam(containerIdStr, Environment.CONTAINER_ID.name());
        validateInputParam(nodeHostString, Environment.NM_HOST.name());
        validateInputParam(nodePortString, Environment.NM_PORT.name());
        validateInputParam(nodeHttpPortString, Environment.NM_HTTP_PORT.name());
        validateInputParam(appSubmitTimeStr, ApplicationConstants.APP_SUBMIT_TIME_ENV);
        ContainerId containerId = ContainerId.fromString(containerIdStr);
        ApplicationAttemptId applicationAttemptId = containerId.getApplicationAttemptId();
        if (applicationAttemptId != null) {
            CallerContext.setCurrent(new CallerContext.Builder("mr_appmaster_" + applicationAttemptId.toString()).build());
        }
        long appSubmitTime = Long.parseLong(appSubmitTimeStr);
        MRAppMaster appMaster = new MRAppMaster(applicationAttemptId, containerId, nodeHostString, Integer.parseInt(nodePortString), Integer.parseInt(nodeHttpPortString), appSubmitTime);
        ShutdownHookManager.get().addShutdownHook(new MRAppMasterShutdownHook(appMaster), SHUTDOWN_HOOK_PRIORITY);
        JobConf conf = new JobConf(new YarnConfiguration());
        conf.addResource(new Path(MRJobConfig.JOB_CONF_FILE));
        MRWebAppUtil.initialize(conf);
        // log the system properties
        String systemPropsToLog = MRApps.getSystemPropertiesToLog(conf);
        if (systemPropsToLog != null) {
            LOG.info(systemPropsToLog);
        }
        String jobUserName = System.getenv(ApplicationConstants.Environment.USER.name());
        conf.set(MRJobConfig.USER_NAME, jobUserName);
        initAndStartAppMaster(appMaster, conf, jobUserName);
    } catch (Throwable t) {
        LOG.fatal("Error starting MRAppMaster", t);
        ExitUtil.terminate(1, t);
    }
}
Also used : Path(org.apache.hadoop.fs.Path) YarnUncaughtExceptionHandler(org.apache.hadoop.yarn.YarnUncaughtExceptionHandler) ContainerId(org.apache.hadoop.yarn.api.records.ContainerId) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) ApplicationAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId) JobConf(org.apache.hadoop.mapred.JobConf)

Aggregations

YarnUncaughtExceptionHandler (org.apache.hadoop.yarn.YarnUncaughtExceptionHandler)11 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)7 GenericOptionsParser (org.apache.hadoop.util.GenericOptionsParser)5 Configuration (org.apache.hadoop.conf.Configuration)3 InetSocketAddress (java.net.InetSocketAddress)2 Path (org.apache.hadoop.fs.Path)2 JobConf (org.apache.hadoop.mapred.JobConf)2 VisibleForTesting (com.google.common.annotations.VisibleForTesting)1 IOException (java.io.IOException)1 PrivilegedExceptionAction (java.security.PrivilegedExceptionAction)1 ArrayList (java.util.ArrayList)1 ScheduledExecutorService (java.util.concurrent.ScheduledExecutorService)1 FSError (org.apache.hadoop.fs.FSError)1 JobTokenIdentifier (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier)1 Credentials (org.apache.hadoop.security.Credentials)1 UserGroupInformation (org.apache.hadoop.security.UserGroupInformation)1 DiskErrorException (org.apache.hadoop.util.DiskChecker.DiskErrorException)1 ApplicationAttemptId (org.apache.hadoop.yarn.api.records.ApplicationAttemptId)1 ContainerId (org.apache.hadoop.yarn.api.records.ContainerId)1