Search in sources :

Example 1 with AMLauncher

use of org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher in project hadoop by apache.

the class MockRMWithCustomAMLauncher method createAMLauncher.

@Override
protected ApplicationMasterLauncher createAMLauncher() {
    return new ApplicationMasterLauncher(getRMContext()) {

        @Override
        protected Runnable createRunnableLauncher(RMAppAttempt application, AMLauncherEventType event) {
            return new AMLauncher(context, application, event, getConfig()) {

                @Override
                protected ContainerManagementProtocol getContainerMgrProxy(ContainerId containerId) {
                    return containerManager;
                }

                @Override
                protected Token<AMRMTokenIdentifier> createAndSetAMRMToken() {
                    Token<AMRMTokenIdentifier> amRmToken = super.createAndSetAMRMToken();
                    InetSocketAddress serviceAddr = getConfig().getSocketAddr(YarnConfiguration.RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_PORT);
                    SecurityUtil.setTokenService(amRmToken, serviceAddr);
                    return amRmToken;
                }
            };
        }
    };
}
Also used : RMAppAttempt(org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttempt) AMRMTokenIdentifier(org.apache.hadoop.yarn.security.AMRMTokenIdentifier) ContainerId(org.apache.hadoop.yarn.api.records.ContainerId) ApplicationMasterLauncher(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher) InetSocketAddress(java.net.InetSocketAddress) AMLauncher(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher) AMLauncherEventType(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType)

Example 2 with AMLauncher

use of org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher in project hadoop by apache.

the class TestApplicationMasterLauncher method testRetriesOnFailures.

@Test
public void testRetriesOnFailures() throws Exception {
    final ContainerManagementProtocol mockProxy = mock(ContainerManagementProtocol.class);
    final StartContainersResponse mockResponse = mock(StartContainersResponse.class);
    when(mockProxy.startContainers(any(StartContainersRequest.class))).thenThrow(new NMNotYetReadyException("foo")).thenReturn(mockResponse);
    Configuration conf = new Configuration();
    conf.setInt(YarnConfiguration.RM_AM_MAX_ATTEMPTS, 1);
    conf.setInt(YarnConfiguration.CLIENT_NM_CONNECT_RETRY_INTERVAL_MS, 1);
    final DrainDispatcher dispatcher = new DrainDispatcher();
    MockRM rm = new MockRMWithCustomAMLauncher(conf, null) {

        @Override
        protected ApplicationMasterLauncher createAMLauncher() {
            return new ApplicationMasterLauncher(getRMContext()) {

                @Override
                protected Runnable createRunnableLauncher(RMAppAttempt application, AMLauncherEventType event) {
                    return new AMLauncher(context, application, event, getConfig()) {

                        @Override
                        protected YarnRPC getYarnRPC() {
                            YarnRPC mockRpc = mock(YarnRPC.class);
                            when(mockRpc.getProxy(any(Class.class), any(InetSocketAddress.class), any(Configuration.class))).thenReturn(mockProxy);
                            return mockRpc;
                        }
                    };
                }
            };
        }

        @Override
        protected Dispatcher createDispatcher() {
            return dispatcher;
        }
    };
    rm.start();
    MockNM nm1 = rm.registerNode("127.0.0.1:1234", 5120);
    RMApp app = rm.submitApp(2000);
    // kick the scheduling
    nm1.nodeHeartbeat(true);
    dispatcher.await();
    MockRM.waitForState(app.getCurrentAppAttempt(), RMAppAttemptState.LAUNCHED, 500);
}
Also used : DrainDispatcher(org.apache.hadoop.yarn.event.DrainDispatcher) RMApp(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMApp) RMAppAttempt(org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttempt) StartContainersResponse(org.apache.hadoop.yarn.api.protocolrecords.StartContainersResponse) Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) ApplicationMasterLauncher(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher) InetSocketAddress(java.net.InetSocketAddress) AMLauncherEventType(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType) NMNotYetReadyException(org.apache.hadoop.yarn.exceptions.NMNotYetReadyException) YarnRPC(org.apache.hadoop.yarn.ipc.YarnRPC) ContainerManagementProtocol(org.apache.hadoop.yarn.api.ContainerManagementProtocol) AMLauncher(org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher) Test(org.junit.Test)

Aggregations

InetSocketAddress (java.net.InetSocketAddress)2 AMLauncher (org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher)2 AMLauncherEventType (org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncherEventType)2 ApplicationMasterLauncher (org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher)2 RMAppAttempt (org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttempt)2 Configuration (org.apache.hadoop.conf.Configuration)1 ContainerManagementProtocol (org.apache.hadoop.yarn.api.ContainerManagementProtocol)1 StartContainersResponse (org.apache.hadoop.yarn.api.protocolrecords.StartContainersResponse)1 ContainerId (org.apache.hadoop.yarn.api.records.ContainerId)1 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)1 DrainDispatcher (org.apache.hadoop.yarn.event.DrainDispatcher)1 NMNotYetReadyException (org.apache.hadoop.yarn.exceptions.NMNotYetReadyException)1 YarnRPC (org.apache.hadoop.yarn.ipc.YarnRPC)1 AMRMTokenIdentifier (org.apache.hadoop.yarn.security.AMRMTokenIdentifier)1 RMApp (org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMApp)1 Test (org.junit.Test)1