Search in sources :

Example 56 with UserGroupInformation

use of org.apache.hadoop.security.UserGroupInformation in project weave by continuuity.

the class AMRMClientImpl method start.

@Override
public synchronized void start() {
    final YarnConfiguration conf = new YarnConfiguration(getConfig());
    final YarnRPC rpc = YarnRPC.create(conf);
    final InetSocketAddress rmAddress = conf.getSocketAddr(YarnConfiguration.RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_PORT);
    UserGroupInformation currentUser;
    try {
        currentUser = UserGroupInformation.getCurrentUser();
    } catch (IOException e) {
        throw new YarnException(e);
    }
    if (UserGroupInformation.isSecurityEnabled()) {
        String tokenURLEncodedStr = System.getenv().get(ApplicationConstants.APPLICATION_MASTER_TOKEN_ENV_NAME);
        Token<? extends TokenIdentifier> token = new Token<TokenIdentifier>();
        try {
            token.decodeFromUrlString(tokenURLEncodedStr);
        } catch (IOException e) {
            throw new YarnException(e);
        }
        SecurityUtil.setTokenService(token, rmAddress);
        if (LOG.isDebugEnabled()) {
            LOG.debug("AppMasterToken is " + token);
        }
        currentUser.addToken(token);
    }
    rmClient = currentUser.doAs(new PrivilegedAction<AMRMProtocol>() {

        @Override
        public AMRMProtocol run() {
            return (AMRMProtocol) rpc.getProxy(AMRMProtocol.class, rmAddress, conf);
        }
    });
    LOG.debug("Connecting to ResourceManager at " + rmAddress);
    super.start();
}
Also used : YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) PrivilegedAction(java.security.PrivilegedAction) InetSocketAddress(java.net.InetSocketAddress) Token(org.apache.hadoop.security.token.Token) AMRMProtocol(org.apache.hadoop.yarn.api.AMRMProtocol) YarnRPC(org.apache.hadoop.yarn.ipc.YarnRPC) IOException(java.io.IOException) YarnException(org.apache.hadoop.yarn.YarnException) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation)

Example 57 with UserGroupInformation

use of org.apache.hadoop.security.UserGroupInformation in project hadoop by apache.

the class AMLauncher method getContainerMgrProxy.

// Protected. For tests.
protected ContainerManagementProtocol getContainerMgrProxy(final ContainerId containerId) {
    final NodeId node = masterContainer.getNodeId();
    final InetSocketAddress containerManagerConnectAddress = NetUtils.createSocketAddrForHost(node.getHost(), node.getPort());
    final YarnRPC rpc = getYarnRPC();
    UserGroupInformation currentUser = UserGroupInformation.createRemoteUser(containerId.getApplicationAttemptId().toString());
    String user = rmContext.getRMApps().get(containerId.getApplicationAttemptId().getApplicationId()).getUser();
    org.apache.hadoop.yarn.api.records.Token token = rmContext.getNMTokenSecretManager().createNMToken(containerId.getApplicationAttemptId(), node, user);
    currentUser.addToken(ConverterUtils.convertFromYarn(token, containerManagerConnectAddress));
    return NMProxy.createNMProxy(conf, ContainerManagementProtocol.class, currentUser, rpc, containerManagerConnectAddress);
}
Also used : InetSocketAddress(java.net.InetSocketAddress) NodeId(org.apache.hadoop.yarn.api.records.NodeId) YarnRPC(org.apache.hadoop.yarn.ipc.YarnRPC) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation)

Example 58 with UserGroupInformation

use of org.apache.hadoop.security.UserGroupInformation in project hadoop by apache.

the class TestClientRMService method testMoveApplicationAdminTargetQueue.

@Test
public void testMoveApplicationAdminTargetQueue() throws Exception {
    ApplicationId applicationId = getApplicationId(1);
    UserGroupInformation aclUGI = UserGroupInformation.getCurrentUser();
    QueueACLsManager queueAclsManager = getQueueAclManager("allowed_queue", QueueACL.ADMINISTER_QUEUE, aclUGI);
    ApplicationACLsManager appAclsManager = getAppAclManager();
    ClientRMService rmService = createClientRMServiceForMoveApplicationRequest(applicationId, aclUGI.getShortUserName(), appAclsManager, queueAclsManager);
    // user is admin move to queue in acl
    MoveApplicationAcrossQueuesRequest moveAppRequest = MoveApplicationAcrossQueuesRequest.newInstance(applicationId, "allowed_queue");
    rmService.moveApplicationAcrossQueues(moveAppRequest);
    // user is admin move to queue not in acl
    moveAppRequest = MoveApplicationAcrossQueuesRequest.newInstance(applicationId, "not_allowed");
    try {
        rmService.moveApplicationAcrossQueues(moveAppRequest);
        Assert.fail("The request should fail with an AccessControlException");
    } catch (YarnException rex) {
        Assert.assertTrue("AccessControlException is expected", rex.getCause() instanceof AccessControlException);
    }
    // ACL is owned by "moveuser", move is performed as a different user
    aclUGI = UserGroupInformation.createUserForTesting("moveuser", new String[] {});
    queueAclsManager = getQueueAclManager("move_queue", QueueACL.ADMINISTER_QUEUE, aclUGI);
    appAclsManager = getAppAclManager();
    ClientRMService rmService2 = createClientRMServiceForMoveApplicationRequest(applicationId, aclUGI.getShortUserName(), appAclsManager, queueAclsManager);
    // no access to this queue
    MoveApplicationAcrossQueuesRequest moveAppRequest2 = MoveApplicationAcrossQueuesRequest.newInstance(applicationId, "move_queue");
    try {
        rmService2.moveApplicationAcrossQueues(moveAppRequest2);
        Assert.fail("The request should fail with an AccessControlException");
    } catch (YarnException rex) {
        Assert.assertTrue("AccessControlException is expected", rex.getCause() instanceof AccessControlException);
    }
    // execute the move as the acl owner
    // access to the queue OK: user allowed in this queue
    aclUGI.doAs(new PrivilegedExceptionAction<Object>() {

        @Override
        public Object run() throws Exception {
            return rmService2.moveApplicationAcrossQueues(moveAppRequest2);
        }
    });
}
Also used : AccessControlException(java.security.AccessControlException) Matchers.anyString(org.mockito.Matchers.anyString) YarnException(org.apache.hadoop.yarn.exceptions.YarnException) ApplicationNotFoundException(org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException) IOException(java.io.IOException) BrokenBarrierException(java.util.concurrent.BrokenBarrierException) AccessControlException(java.security.AccessControlException) YarnException(org.apache.hadoop.yarn.exceptions.YarnException) ApplicationACLsManager(org.apache.hadoop.yarn.server.security.ApplicationACLsManager) MoveApplicationAcrossQueuesRequest(org.apache.hadoop.yarn.api.protocolrecords.MoveApplicationAcrossQueuesRequest) QueueACLsManager(org.apache.hadoop.yarn.server.resourcemanager.security.QueueACLsManager) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation) Test(org.junit.Test)

Example 59 with UserGroupInformation

use of org.apache.hadoop.security.UserGroupInformation in project hadoop by apache.

the class TestAMAuthorization method testUnauthorizedAccess.

@Test
public void testUnauthorizedAccess() throws Exception {
    MyContainerManager containerManager = new MyContainerManager();
    rm = new MockRMWithAMS(conf, containerManager);
    rm.start();
    MockNM nm1 = rm.registerNode("localhost:1234", 5120);
    RMApp app = rm.submitApp(1024);
    nm1.nodeHeartbeat(true);
    int waitCount = 0;
    while (containerManager.containerTokens == null && waitCount++ < 40) {
        LOG.info("Waiting for AM Launch to happen..");
        Thread.sleep(1000);
    }
    Assert.assertNotNull(containerManager.containerTokens);
    RMAppAttempt attempt = app.getCurrentAppAttempt();
    ApplicationAttemptId applicationAttemptId = attempt.getAppAttemptId();
    waitForLaunchedState(attempt);
    final Configuration conf = rm.getConfig();
    final YarnRPC rpc = YarnRPC.create(conf);
    final InetSocketAddress serviceAddr = conf.getSocketAddr(YarnConfiguration.RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_ADDRESS, YarnConfiguration.DEFAULT_RM_SCHEDULER_PORT);
    UserGroupInformation currentUser = UserGroupInformation.createRemoteUser(applicationAttemptId.toString());
    // First try contacting NM without tokens
    ApplicationMasterProtocol client = currentUser.doAs(new PrivilegedAction<ApplicationMasterProtocol>() {

        @Override
        public ApplicationMasterProtocol run() {
            return (ApplicationMasterProtocol) rpc.getProxy(ApplicationMasterProtocol.class, serviceAddr, conf);
        }
    });
    RegisterApplicationMasterRequest request = Records.newRecord(RegisterApplicationMasterRequest.class);
    try {
        client.registerApplicationMaster(request);
        Assert.fail("Should fail with authorization error");
    } catch (Exception e) {
        if (isCause(AccessControlException.class, e)) {
            // Because there are no tokens, the request should be rejected as the
            // server side will assume we are trying simple auth.
            String expectedMessage = "";
            if (UserGroupInformation.isSecurityEnabled()) {
                expectedMessage = "Client cannot authenticate via:[TOKEN]";
            } else {
                expectedMessage = "SIMPLE authentication is not enabled.  Available:[TOKEN]";
            }
            Assert.assertTrue(e.getCause().getMessage().contains(expectedMessage));
        } else {
            throw e;
        }
    }
// TODO: Add validation of invalid authorization when there's more data in
// the AMRMToken
}
Also used : RMApp(org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMApp) RMAppAttempt(org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttempt) Configuration(org.apache.hadoop.conf.Configuration) YarnConfiguration(org.apache.hadoop.yarn.conf.YarnConfiguration) InetSocketAddress(java.net.InetSocketAddress) AccessControlException(org.apache.hadoop.security.AccessControlException) ApplicationMasterProtocol(org.apache.hadoop.yarn.api.ApplicationMasterProtocol) ApplicationAttemptId(org.apache.hadoop.yarn.api.records.ApplicationAttemptId) YarnRPC(org.apache.hadoop.yarn.ipc.YarnRPC) YarnException(org.apache.hadoop.yarn.exceptions.YarnException) IOException(java.io.IOException) AccessControlException(org.apache.hadoop.security.AccessControlException) RegisterApplicationMasterRequest(org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterRequest) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation) Test(org.junit.Test)

Example 60 with UserGroupInformation

use of org.apache.hadoop.security.UserGroupInformation in project hadoop by apache.

the class TestMiniMRProxyUser method ___testInvalidProxyUser.

@Test
public void ___testInvalidProxyUser() throws Exception {
    UserGroupInformation ugi = UserGroupInformation.createProxyUser("u2", UserGroupInformation.getLoginUser());
    ugi.doAs(new PrivilegedExceptionAction<Void>() {

        public Void run() throws Exception {
            try {
                mrRun();
                fail();
            } catch (RemoteException ex) {
            //nop
            } catch (Exception ex) {
                fail();
            }
            return null;
        }
    });
}
Also used : RemoteException(org.apache.hadoop.ipc.RemoteException) RemoteException(org.apache.hadoop.ipc.RemoteException) UserGroupInformation(org.apache.hadoop.security.UserGroupInformation) Test(org.junit.Test)

Aggregations

UserGroupInformation (org.apache.hadoop.security.UserGroupInformation)664 IOException (java.io.IOException)281 Test (org.junit.Test)242 Configuration (org.apache.hadoop.conf.Configuration)142 Path (org.apache.hadoop.fs.Path)105 FileSystem (org.apache.hadoop.fs.FileSystem)73 YarnException (org.apache.hadoop.yarn.exceptions.YarnException)57 AccessControlException (org.apache.hadoop.security.AccessControlException)54 PrivilegedExceptionAction (java.security.PrivilegedExceptionAction)51 FsPermission (org.apache.hadoop.fs.permission.FsPermission)49 Path (javax.ws.rs.Path)47 Token (org.apache.hadoop.security.token.Token)46 Produces (javax.ws.rs.Produces)45 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)45 RMApp (org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMApp)43 AuthorizationException (org.apache.hadoop.security.authorize.AuthorizationException)40 ArrayList (java.util.ArrayList)38 Text (org.apache.hadoop.io.Text)38 ApplicationAttemptId (org.apache.hadoop.yarn.api.records.ApplicationAttemptId)36 YarnConfiguration (org.apache.hadoop.yarn.conf.YarnConfiguration)35