Search in sources :

Example 66 with GFacException

use of org.apache.airavata.gfac.core.GFacException in project airavata by apache.

the class BESJobSubmissionTask method copyOutputFilesToStorage.

private void copyOutputFilesToStorage(TaskContext taskContext, List<OutputDataObjectType> copyOutput) throws GFacException {
    ProcessContext pc = taskContext.getParentProcessContext();
    String remoteFilePath = null, fileName = null, localFilePath = null;
    try {
        authenticationInfo = Factory.getStorageSSHKeyAuthentication(pc);
        ServerInfo serverInfo = pc.getComputeResourceServerInfo();
        Session sshSession = Factory.getSSHSession(authenticationInfo, serverInfo);
        for (OutputDataObjectType output : copyOutput) {
            switch(output.getType()) {
                case STDERR:
                case STDOUT:
                case STRING:
                case URI:
                    localFilePath = output.getValue();
                    if (localFilePath.contains("://")) {
                        localFilePath = localFilePath.substring(localFilePath.indexOf("://") + 2, localFilePath.length());
                    }
                    fileName = localFilePath.substring(localFilePath.lastIndexOf("/") + 1);
                    URI destinationURI = TaskUtils.getDestinationURI(taskContext, hostName, inputPath, fileName);
                    remoteFilePath = destinationURI.getPath();
                    log.info("SCP local file :{} -> from remote :{}", localFilePath, remoteFilePath);
                    SSHUtils.scpTo(localFilePath, remoteFilePath, sshSession);
                    output.setValue(destinationURI.toString());
                    break;
                default:
                    break;
            }
        }
    } catch (IOException | JSchException | SSHApiException | URISyntaxException | CredentialStoreException e) {
        log.error("Error while coping local file " + localFilePath + " to remote " + remoteFilePath, e);
        throw new GFacException("Error while scp output files to remote storage file location", e);
    }
}
Also used : JSchException(com.jcraft.jsch.JSchException) ServerInfo(org.apache.airavata.gfac.core.cluster.ServerInfo) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) CredentialStoreException(org.apache.airavata.credential.store.store.CredentialStoreException) URI(java.net.URI) SSHApiException(org.apache.airavata.gfac.core.SSHApiException) ProcessContext(org.apache.airavata.gfac.core.context.ProcessContext) OutputDataObjectType(org.apache.airavata.model.application.io.OutputDataObjectType) GFacException(org.apache.airavata.gfac.core.GFacException) Session(com.jcraft.jsch.Session)

Example 67 with GFacException

use of org.apache.airavata.gfac.core.GFacException in project airavata by apache.

the class SecurityUtils method addSecurityContext.

public static void addSecurityContext(ProcessContext processContext) throws GFacException {
    if (!processContext.getJobSubmissionProtocol().equals(JobSubmissionProtocol.UNICORE)) {
        logger.error("This is a wrong method to invoke for UNICORE host types,please check your gfac-config.xml");
    } else {
        // set by the framework
        String credentialStoreToken = processContext.getTokenId();
        RequestData requestData;
        try {
            requestData = new RequestData(ServerSettings.getDefaultUserGateway());
        } catch (ApplicationSettingsException e1) {
            throw new GFacException(e1);
        }
        // coming from top tier
        requestData.setTokenId(credentialStoreToken);
        CredentialReader credentialReader = null;
        try {
            credentialReader = GFacUtils.getCredentialReader();
        } catch (Exception e) {
            logger.warn("Cannot get credential reader instance");
        }
        UNICORESecurityContext secCtx = new UNICORESecurityContext(credentialReader, requestData);
    // processContext.addSecurityContext(X509SecurityContext.X509_SECURITY_CONTEXT, secCtx);
    }
}
Also used : ApplicationSettingsException(org.apache.airavata.common.exception.ApplicationSettingsException) GFacException(org.apache.airavata.gfac.core.GFacException) UNICORESecurityContext(org.apache.airavata.gfac.bes.security.UNICORESecurityContext) RequestData(org.apache.airavata.gfac.core.RequestData) CredentialReader(org.apache.airavata.credential.store.store.CredentialReader) GFacException(org.apache.airavata.gfac.core.GFacException) IOException(java.io.IOException) InvalidKeyException(java.security.InvalidKeyException) ApplicationSettingsException(org.apache.airavata.common.exception.ApplicationSettingsException)

Example 68 with GFacException

use of org.apache.airavata.gfac.core.GFacException in project airavata by apache.

the class LocalRemoteCluster method getJobStatus.

@Override
public JobStatus getJobStatus(String jobID) throws GFacException {
    RawCommandInfo monitorCommand = jobManagerConfiguration.getMonitorCommand(jobID);
    LocalCommandOutput localCommandOutput = new LocalCommandOutput();
    try {
        executeCommand(monitorCommand, localCommandOutput);
        return outputParser.parseJobStatus(jobID, localCommandOutput.getStandardErrorString());
    } catch (IOException e) {
        throw new GFacException("Error while getting jobStatus", e);
    }
}
Also used : GFacException(org.apache.airavata.gfac.core.GFacException) IOException(java.io.IOException)

Example 69 with GFacException

use of org.apache.airavata.gfac.core.GFacException in project airavata by apache.

the class HPCRemoteCluster method executeCommand.

private void executeCommand(CommandInfo commandInfo, CommandOutput commandOutput) throws GFacException {
    String command = commandInfo.getCommand();
    int retryCount = 0;
    ChannelExec channelExec = null;
    try {
        while (retryCount < MAX_RETRY_COUNT) {
            retryCount++;
            try {
                Session session = getSshSession();
                channelExec = ((ChannelExec) session.openChannel("exec"));
                channelExec.setCommand(command);
                channelExec.setInputStream(null);
                channelExec.setErrStream(commandOutput.getStandardError());
                channelExec.connect();
                log.info("Executing command {}", commandInfo.getCommand());
                commandOutput.onOutput(channelExec);
                // exit from while loop
                break;
            } catch (JSchException e) {
                if (retryCount == MAX_RETRY_COUNT) {
                    log.error("Retry count " + MAX_RETRY_COUNT + " exceeded for executing command : " + command, e);
                    throw e;
                }
                log.error("Issue with jsch, Retry executing command : " + command, e);
            }
        }
    } catch (JSchException e) {
        throw new GFacException("Unable to execute command - " + command, e);
    } finally {
        // Only disconnecting the channel, session can be reused
        if (channelExec != null) {
            commandOutput.exitCode(channelExec.getExitStatus());
            channelExec.disconnect();
        }
    }
}
Also used : JSchException(com.jcraft.jsch.JSchException) GFacException(org.apache.airavata.gfac.core.GFacException) ChannelExec(com.jcraft.jsch.ChannelExec) Session(com.jcraft.jsch.Session)

Example 70 with GFacException

use of org.apache.airavata.gfac.core.GFacException in project airavata by apache.

the class HPCRemoteCluster method makeDirectory.

@Override
public void makeDirectory(String directoryPath) throws GFacException {
    int retryCount = 0;
    try {
        while (retryCount < MAX_RETRY_COUNT) {
            retryCount++;
            log.info("Creating directory: " + serverInfo.getHost() + ":" + directoryPath);
            try {
                SSHUtils.makeDirectory(directoryPath, getSession());
                // Exit while loop
                break;
            } catch (JSchException e) {
                if (retryCount == MAX_RETRY_COUNT) {
                    log.error("Retry count " + MAX_RETRY_COUNT + " exceeded for creating directory: " + serverInfo.getHost() + ":" + directoryPath, e);
                    throw e;
                }
                log.error("Issue with jsch, Retry creating directory: " + serverInfo.getHost() + ":" + directoryPath);
            }
        }
    } catch (JSchException | IOException e) {
        throw new GFacException("Failed to create directory " + serverInfo.getHost() + ":" + directoryPath, e);
    }
}
Also used : JSchException(com.jcraft.jsch.JSchException) GFacException(org.apache.airavata.gfac.core.GFacException) IOException(java.io.IOException)

Aggregations

GFacException (org.apache.airavata.gfac.core.GFacException)73 IOException (java.io.IOException)24 ApplicationSettingsException (org.apache.airavata.common.exception.ApplicationSettingsException)19 ProcessContext (org.apache.airavata.gfac.core.context.ProcessContext)16 JSchException (com.jcraft.jsch.JSchException)15 RegistryException (org.apache.airavata.registry.cpi.RegistryException)15 AiravataException (org.apache.airavata.common.exception.AiravataException)14 TaskStatus (org.apache.airavata.model.status.TaskStatus)13 URISyntaxException (java.net.URISyntaxException)12 ErrorModel (org.apache.airavata.model.commons.ErrorModel)11 CredentialStoreException (org.apache.airavata.credential.store.store.CredentialStoreException)10 TException (org.apache.thrift.TException)9 AppCatalogException (org.apache.airavata.registry.cpi.AppCatalogException)8 DefaultClientConfiguration (eu.unicore.util.httpclient.DefaultClientConfiguration)7 URI (java.net.URI)7 OutputDataObjectType (org.apache.airavata.model.application.io.OutputDataObjectType)7 Session (com.jcraft.jsch.Session)6 InvalidKeyException (java.security.InvalidKeyException)6 RemoteCluster (org.apache.airavata.gfac.core.cluster.RemoteCluster)6 JobModel (org.apache.airavata.model.job.JobModel)6