Search in sources :

Example 1 with ApplicationMasterProcessLauncher

use of com.continuuity.weave.internal.appmaster.ApplicationMasterProcessLauncher in project weave by continuuity.

the class Hadoop20YarnAppClient method createLauncher.

@Override
public ProcessLauncher<ApplicationId> createLauncher(WeaveSpecification weaveSpec) throws Exception {
    // Request for new application
    final GetNewApplicationResponse response = yarnClient.getNewApplication();
    final ApplicationId appId = response.getApplicationId();
    // Setup the context for application submission
    final ApplicationSubmissionContext appSubmissionContext = Records.newRecord(ApplicationSubmissionContext.class);
    appSubmissionContext.setApplicationId(appId);
    appSubmissionContext.setApplicationName(weaveSpec.getName());
    appSubmissionContext.setUser(user);
    ApplicationSubmitter submitter = new ApplicationSubmitter() {

        @Override
        public ProcessController<YarnApplicationReport> submit(YarnLaunchContext launchContext, Resource capability) {
            ContainerLaunchContext context = launchContext.getLaunchContext();
            addRMToken(context);
            context.setUser(appSubmissionContext.getUser());
            context.setResource(adjustMemory(response, capability));
            appSubmissionContext.setAMContainerSpec(context);
            try {
                yarnClient.submitApplication(appSubmissionContext);
                return new ProcessControllerImpl(yarnClient, appId);
            } catch (YarnRemoteException e) {
                LOG.error("Failed to submit application {}", appId, e);
                throw Throwables.propagate(e);
            }
        }
    };
    return new ApplicationMasterProcessLauncher(appId, submitter);
}
Also used : ApplicationMasterProcessLauncher(com.continuuity.weave.internal.appmaster.ApplicationMasterProcessLauncher) GetNewApplicationResponse(org.apache.hadoop.yarn.api.protocolrecords.GetNewApplicationResponse) ApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext) Resource(org.apache.hadoop.yarn.api.records.Resource) ContainerLaunchContext(org.apache.hadoop.yarn.api.records.ContainerLaunchContext) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) ApplicationSubmitter(com.continuuity.weave.internal.appmaster.ApplicationSubmitter) YarnRemoteException(org.apache.hadoop.yarn.exceptions.YarnRemoteException)

Example 2 with ApplicationMasterProcessLauncher

use of com.continuuity.weave.internal.appmaster.ApplicationMasterProcessLauncher in project weave by continuuity.

the class Hadoop21YarnAppClient method createLauncher.

@Override
public ProcessLauncher<ApplicationId> createLauncher(WeaveSpecification weaveSpec) throws Exception {
    // Request for new application
    YarnClientApplication application = yarnClient.createApplication();
    final GetNewApplicationResponse response = application.getNewApplicationResponse();
    final ApplicationId appId = response.getApplicationId();
    // Setup the context for application submission
    final ApplicationSubmissionContext appSubmissionContext = application.getApplicationSubmissionContext();
    appSubmissionContext.setApplicationId(appId);
    appSubmissionContext.setApplicationName(weaveSpec.getName());
    ApplicationSubmitter submitter = new ApplicationSubmitter() {

        @Override
        public ProcessController<YarnApplicationReport> submit(YarnLaunchContext context, Resource capability) {
            ContainerLaunchContext launchContext = context.getLaunchContext();
            addRMToken(launchContext);
            appSubmissionContext.setAMContainerSpec(launchContext);
            appSubmissionContext.setResource(adjustMemory(response, capability));
            appSubmissionContext.setMaxAppAttempts(2);
            try {
                yarnClient.submitApplication(appSubmissionContext);
                return new ProcessControllerImpl(yarnClient, appId);
            } catch (Exception e) {
                LOG.error("Failed to submit application {}", appId, e);
                throw Throwables.propagate(e);
            }
        }
    };
    return new ApplicationMasterProcessLauncher(appId, submitter);
}
Also used : ApplicationMasterProcessLauncher(com.continuuity.weave.internal.appmaster.ApplicationMasterProcessLauncher) GetNewApplicationResponse(org.apache.hadoop.yarn.api.protocolrecords.GetNewApplicationResponse) YarnClientApplication(org.apache.hadoop.yarn.client.api.YarnClientApplication) ApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext) Resource(org.apache.hadoop.yarn.api.records.Resource) ContainerLaunchContext(org.apache.hadoop.yarn.api.records.ContainerLaunchContext) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) ApplicationSubmitter(com.continuuity.weave.internal.appmaster.ApplicationSubmitter)

Aggregations

ApplicationMasterProcessLauncher (com.continuuity.weave.internal.appmaster.ApplicationMasterProcessLauncher)2 ApplicationSubmitter (com.continuuity.weave.internal.appmaster.ApplicationSubmitter)2 GetNewApplicationResponse (org.apache.hadoop.yarn.api.protocolrecords.GetNewApplicationResponse)2 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)2 ApplicationSubmissionContext (org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext)2 ContainerLaunchContext (org.apache.hadoop.yarn.api.records.ContainerLaunchContext)2 Resource (org.apache.hadoop.yarn.api.records.Resource)2 YarnClientApplication (org.apache.hadoop.yarn.client.api.YarnClientApplication)1 YarnRemoteException (org.apache.hadoop.yarn.exceptions.YarnRemoteException)1