Search in sources :

Example 6 with DAGClientAMProtocolBlockingPB

use of org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB in project tez by apache.

the class TezClient method waitForProxy.

private DAGClientAMProtocolBlockingPB waitForProxy() throws IOException, TezException, InterruptedException {
    long startTime = System.currentTimeMillis();
    long endTime = startTime + (clientTimeout * 1000);
    DAGClientAMProtocolBlockingPB proxy = null;
    while (true) {
        proxy = getAMProxy(sessionAppId);
        if (proxy != null) {
            break;
        }
        Thread.sleep(100l);
        if (clientTimeout != -1 && System.currentTimeMillis() > endTime) {
            break;
        }
    }
    return proxy;
}
Also used : DAGClientAMProtocolBlockingPB(org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB)

Example 7 with DAGClientAMProtocolBlockingPB

use of org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB in project tez by apache.

the class TestTezClient method configureAndCreateTezClient.

TezClientForTest configureAndCreateTezClient(Map<String, LocalResource> lrs, boolean isSession, TezConfiguration conf) throws YarnException, IOException, ServiceException {
    if (conf == null) {
        conf = new TezConfiguration();
    }
    conf.setBoolean(TezConfiguration.TEZ_IGNORE_LIB_URIS, true);
    conf.setBoolean(TezConfiguration.TEZ_AM_SESSION_MODE, isSession);
    TezClientForTest client = new TezClientForTest("test", conf, lrs, null);
    ApplicationId appId1 = ApplicationId.newInstance(0, 1);
    YarnClient yarnClient = mock(YarnClient.class, RETURNS_DEEP_STUBS);
    when(yarnClient.createApplication().getNewApplicationResponse().getApplicationId()).thenReturn(appId1);
    when(yarnClient.getApplicationReport(appId1).getYarnApplicationState()).thenReturn(YarnApplicationState.NEW);
    when(yarnClient.submitApplication(any(ApplicationSubmissionContext.class))).thenReturn(appId1);
    DAGClientAMProtocolBlockingPB sessionAmProxy = mock(DAGClientAMProtocolBlockingPB.class, RETURNS_DEEP_STUBS);
    when(sessionAmProxy.getAMStatus(any(RpcController.class), any(GetAMStatusRequestProto.class))).thenReturn(GetAMStatusResponseProto.newBuilder().setStatus(TezAppMasterStatusProto.RUNNING).build());
    client.sessionAmProxy = sessionAmProxy;
    client.mockTezYarnClient = new TezYarnClient(yarnClient);
    client.mockYarnClient = yarnClient;
    client.mockAppId = appId1;
    return client;
}
Also used : RpcController(com.google.protobuf.RpcController) DAGClientAMProtocolBlockingPB(org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB) GetAMStatusRequestProto(org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC.GetAMStatusRequestProto) ApplicationSubmissionContext(org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext) ApplicationId(org.apache.hadoop.yarn.api.records.ApplicationId) YarnClient(org.apache.hadoop.yarn.client.api.YarnClient) TezConfiguration(org.apache.tez.dag.api.TezConfiguration)

Aggregations

DAGClientAMProtocolBlockingPB (org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPB)7 ServiceException (com.google.protobuf.ServiceException)3 IOException (java.io.IOException)2 ApplicationId (org.apache.hadoop.yarn.api.records.ApplicationId)2 ApplicationReport (org.apache.hadoop.yarn.api.records.ApplicationReport)2 ApplicationNotFoundException (org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException)2 YarnException (org.apache.hadoop.yarn.exceptions.YarnException)2 TezException (org.apache.tez.dag.api.TezException)2 ThreadFactoryBuilder (com.google.common.util.concurrent.ThreadFactoryBuilder)1 RpcController (com.google.protobuf.RpcController)1 InetSocketAddress (java.net.InetSocketAddress)1 PrivilegedExceptionAction (java.security.PrivilegedExceptionAction)1 Private (org.apache.hadoop.classification.InterfaceAudience.Private)1 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)1 Path (org.apache.hadoop.fs.Path)1 UserGroupInformation (org.apache.hadoop.security.UserGroupInformation)1 ApplicationSubmissionContext (org.apache.hadoop.yarn.api.records.ApplicationSubmissionContext)1 LocalResource (org.apache.hadoop.yarn.api.records.LocalResource)1 YarnClient (org.apache.hadoop.yarn.client.api.YarnClient)1 ClientToAMTokenIdentifier (org.apache.hadoop.yarn.security.client.ClientToAMTokenIdentifier)1