Search in sources :

Example 1 with GATObjectCreationException

use of org.gridlab.gat.GATObjectCreationException in project compss by bsc-wdc.

the class GATCopy method doCopy.

private void doCopy(org.gridlab.gat.URI src, org.gridlab.gat.URI dest) throws GATCopyException {
    // Try to copy from each location until successful
    FileInterface f = null;
    LOGGER.debug("RawPath: " + src.getRawPath());
    LOGGER.debug("isLocal: " + src.isLocal());
    if (src.isLocal() && !(new File(src.getRawPath())).exists()) {
        String errorMessage = null;
        if (this.reason instanceof WorkersDebugInfoCopyTransferable) {
            // Only warn, hide error to ErrorManager
            errorMessage = "Workers Debug Information not supported in GAT Adaptor";
            LOGGER.warn(errorMessage);
        } else {
            // Notify error to ErrorManager
            errorMessage = "File '" + src.toString() + "' could not be copied to '" + dest.toString() + "' because it does not exist.";
            ErrorManager.warn(errorMessage);
            LOGGER.warn(errorMessage);
        }
        throw new GATCopyException(errorMessage);
    }
    try {
        f = org.gridlab.gat.GAT.createFile(GATAdaptor.getTransferContext(), src).getFileInterface();
        f.copy(dest);
    } catch (GATObjectCreationException | GATInvocationException e) {
        throw new GATCopyException(e);
    }
}
Also used : FileInterface(org.gridlab.gat.io.FileInterface) GATObjectCreationException(org.gridlab.gat.GATObjectCreationException) GATInvocationException(org.gridlab.gat.GATInvocationException) WorkersDebugInfoCopyTransferable(es.bsc.compss.types.data.transferable.WorkersDebugInfoCopyTransferable) GATCopyException(es.bsc.compss.gat.master.exceptions.GATCopyException) File(java.io.File)

Example 2 with GATObjectCreationException

use of org.gridlab.gat.GATObjectCreationException in project compss by bsc-wdc.

the class LoadLevelerJob method waitForTrigger.

// Wait for the creation of a special file (by the application).
void waitForTrigger(JobState state) throws GATInvocationException {
    if (triggerDirectory == null) {
        return;
    }
    if (jobName == null) {
        return;
    }
    if (waiter == null) {
        try {
            waiter = FileWaiter.createFileWaiter(GAT.createFile(gatContext, triggerDirectory));
        } catch (GATObjectCreationException e) {
            throw new GATInvocationException("Could not create", e);
        }
    }
    String filename = jobName + "." + state.toString().substring(0, 3);
    File file;
    try {
        file = GAT.createFile(gatContext, triggerDirectory + "/" + filename);
    } catch (GATObjectCreationException e) {
        throw new GATInvocationException("Could not create");
    }
    if (logger.isDebugEnabled()) {
        logger.debug("Waiting for " + filename + " in directory " + triggerDirectory);
    }
    waiter.waitFor(filename);
    if (logger.isDebugEnabled()) {
        logger.debug("Finished waiting for " + filename + " in directory " + triggerDirectory);
    }
    synchronized (this.getClass()) {
        if (!file.delete()) {
            if (logger.isDebugEnabled()) {
                logger.debug("Could not remove " + file.toGATURI());
            }
        }
    }
}
Also used : GATObjectCreationException(org.gridlab.gat.GATObjectCreationException) GATInvocationException(org.gridlab.gat.GATInvocationException) File(org.gridlab.gat.io.File)

Example 3 with GATObjectCreationException

use of org.gridlab.gat.GATObjectCreationException in project compss by bsc-wdc.

the class LsfJob method waitForTrigger.

// Wait for the creation of a special file (by the application).
void waitForTrigger(JobState state) throws GATInvocationException {
    if (triggerDirectory == null) {
        return;
    }
    if (jobName == null) {
        return;
    }
    if (waiter == null) {
        try {
            waiter = FileWaiter.createFileWaiter(GAT.createFile(gatContext, triggerDirectory));
        } catch (GATObjectCreationException e) {
            throw new GATInvocationException("Could not create", e);
        }
    }
    String filename = jobName + "." + state.toString().substring(0, 3);
    File file;
    try {
        file = GAT.createFile(gatContext, triggerDirectory + "/" + filename);
    } catch (GATObjectCreationException e) {
        throw new GATInvocationException("Could not create");
    }
    if (logger.isDebugEnabled()) {
        logger.debug("Waiting for " + filename + " in directory " + triggerDirectory);
    }
    waiter.waitFor(filename);
    if (logger.isDebugEnabled()) {
        logger.debug("Finished waiting for " + filename + " in directory " + triggerDirectory);
    }
    synchronized (this.getClass()) {
        if (!file.delete()) {
            if (logger.isDebugEnabled()) {
                logger.debug("Could not remove " + file.toGATURI());
            }
        }
    }
}
Also used : GATObjectCreationException(org.gridlab.gat.GATObjectCreationException) GATInvocationException(org.gridlab.gat.GATInvocationException) File(org.gridlab.gat.io.File)

Example 4 with GATObjectCreationException

use of org.gridlab.gat.GATObjectCreationException in project compss by bsc-wdc.

the class LsfResourceBrokerAdaptor method submitJob.

/*
     * (non-Javadoc)
     * 
     * @see org.gridlab.gat.resources.ResourceBroker#submitJob(org.gridlab.gat.resources.JobDescription)
     */
public Job submitJob(AbstractJobDescription abstractDescription, MetricListener listener, String metricDefinitionName) throws GATInvocationException {
    if (!(abstractDescription instanceof JobDescription)) {
        throw new GATInvocationException("can only handle JobDescriptions: " + abstractDescription.getClass());
    }
    JobDescription description = (JobDescription) abstractDescription;
    SoftwareDescription sd = description.getSoftwareDescription();
    if (sd == null) {
        throw new GATInvocationException("The job description does not contain a software description");
    }
    if (description.getProcessCount() < 1) {
        throw new GATInvocationException("Adaptor cannot handle: process count < 1: " + description.getProcessCount());
    }
    if (description.getResourceCount() != 1) {
        throw new GATInvocationException("Adaptor cannot handle: resource count > 1: " + description.getResourceCount());
    }
    String home = System.getProperty("user.home");
    if (home == null) {
        throw new GATInvocationException("lsf broker could not get user home dir");
    }
    Sandbox sandbox = new Sandbox(gatContext, description, "localhost", home, true, true, false, false);
    LsfJob lsfJob = new LsfJob(gatContext, description, sandbox);
    Job job = null;
    if (description instanceof WrapperJobDescription) {
        WrapperJobCpi tmp = new WrapperJobCpi(gatContext, lsfJob, listener, metricDefinitionName);
        listener = tmp;
        job = tmp;
    } else {
        job = lsfJob;
    }
    if (listener != null && metricDefinitionName != null) {
        Metric metric = lsfJob.getMetricDefinitionByName(metricDefinitionName).createMetric(null);
        lsfJob.addMetricListener(listener, metric);
    }
    lsfJob.setState(Job.JobState.PRE_STAGING);
    lsfJob.waitForTrigger(Job.JobState.PRE_STAGING);
    sandbox.prestage();
    String exe;
    if (sandbox.getResolvedExecutable() != null) {
        exe = sandbox.getResolvedExecutable().getPath();
    // try to set the executable bit, it might be lost
    /* CDIAZ: The command "exe" can be also in a remote host
             * 		  The command must have the right permissions in the remote host
            try {
                new CommandRunner("chmod", "+x", exe);
            } catch (Throwable t) {
                // ignore
            }
            */
    } else {
        exe = getExecutable(description);
    }
    String[] args = getArgumentsArray(description);
    // Directory where the lsf command will be executed
    java.io.File f = new java.io.File(sandbox.getSandboxPath());
    if (!f.exists()) {
        throw new GATInvocationException("Unable to find directory " + f.getAbsolutePath());
    }
    // Check and set the environment for a blaunch command
    Map<String, Object> env = sd.getEnvironment();
    this.prepareBLaunchEnv(env);
    // Encapsulate the original command into a blaunch command
    String host = brokerURI.getHost();
    String blExe = this.getBlaunchCommand();
    String[] blArgs = this.getBlaunchArgs(host, exe, args);
    ProcessBundle bundle = new ProcessBundle(description.getProcessCount(), blExe, blArgs, f, env);
    lsfJob.setSubmissionTime();
    lsfJob.setState(Job.JobState.SCHEDULED);
    try {
        lsfJob.setState(Job.JobState.RUNNING);
        lsfJob.waitForTrigger(Job.JobState.RUNNING);
        lsfJob.setStartTime();
        bundle.startBundle();
        lsfJob.setProcess(bundle);
    } catch (IOException e) {
        throw new CommandNotFoundException("LsfResourceBrokerAdaptor", e);
    }
    if (!sd.streamingStderrEnabled()) {
        try {
            if (sd.getStderr() != null) {
                OutputStream err = GAT.createFileOutputStream(gatContext, sd.getStderr());
                // to file
                StreamForwarder forwarder = new StreamForwarder(bundle.getStderr(), err, sd.getExecutable() + " [stderr]");
                lsfJob.setErrorStream(forwarder);
                if (logger.isDebugEnabled()) {
                    logger.debug("Created stderr forwarder to file " + sd.getStderr());
                }
            } else {
                // or throw it away
                new StreamForwarder(bundle.getStderr(), null, sd.getExecutable() + " [stderr]");
            }
        } catch (GATObjectCreationException e) {
            throw new GATInvocationException("Unable to create file output stream for stderr!", e);
        }
    }
    if (!sd.streamingStdoutEnabled()) {
        // read away the stdout
        try {
            if (sd.getStdout() != null) {
                // to file
                OutputStream out = GAT.createFileOutputStream(gatContext, sd.getStdout());
                StreamForwarder forwarder = new StreamForwarder(bundle.getStdout(), out, sd.getExecutable() + " [stdout]");
                lsfJob.setOutputStream(forwarder);
                if (logger.isDebugEnabled()) {
                    logger.debug("Created stdout forwarder to file " + sd.getStdout());
                }
            } else {
                // or throw it away
                new StreamForwarder(bundle.getStdout(), null, sd.getExecutable() + " [stdout]");
            }
        } catch (GATObjectCreationException e) {
            throw new GATInvocationException("Unable to create file output stream for stdout!", e);
        }
    }
    if (!sd.streamingStdinEnabled() && sd.getStdin() != null) {
        // forward the stdin from file
        try {
            InputStream in = GAT.createFileInputStream(gatContext, sd.getStdin());
            bundle.setStdin(sd.getExecutable(), in);
        } catch (GATObjectCreationException e) {
            throw new GATInvocationException("Unable to create file input stream for stdin!", e);
        }
    }
    lsfJob.monitorState();
    return job;
}
Also used : ProcessBundle(org.gridlab.gat.engine.util.ProcessBundle) GATObjectCreationException(org.gridlab.gat.GATObjectCreationException) InputStream(java.io.InputStream) OutputStream(java.io.OutputStream) StreamForwarder(org.gridlab.gat.engine.util.StreamForwarder) IOException(java.io.IOException) SoftwareDescription(org.gridlab.gat.resources.SoftwareDescription) Sandbox(org.gridlab.gat.resources.cpi.Sandbox) WrapperJobDescription(org.gridlab.gat.resources.WrapperJobDescription) JobDescription(org.gridlab.gat.resources.JobDescription) AbstractJobDescription(org.gridlab.gat.resources.AbstractJobDescription) GATInvocationException(org.gridlab.gat.GATInvocationException) WrapperJobCpi(org.gridlab.gat.resources.cpi.WrapperJobCpi) Metric(org.gridlab.gat.monitoring.Metric) Job(org.gridlab.gat.resources.Job) WrapperJobDescription(org.gridlab.gat.resources.WrapperJobDescription) CommandNotFoundException(org.gridlab.gat.CommandNotFoundException)

Example 5 with GATObjectCreationException

use of org.gridlab.gat.GATObjectCreationException in project compss by bsc-wdc.

the class FileExample method start.

public void start(URI uri1, URI uri2, URI uri3) {
    File file1 = null;
    try {
        file1 = GAT.createFile(uri1);
    } catch (GATObjectCreationException e) {
        System.err.println("failed to create file1 at location '" + uri1 + "': " + e);
        return;
    }
    try {
        file1.copy(uri2);
        System.out.println("file1 at location '" + uri1 + "' copied to file2 at location '" + uri2 + "'");
    } catch (GATInvocationException e) {
        System.err.println("failed to copy file1 at location '" + uri1 + "' to file2 at location '" + uri2 + "': " + e);
        return;
    }
    file1.delete();
    System.out.println("file1 at location '" + uri1 + "' deleted");
    File file2 = null;
    try {
        file2 = GAT.createFile(uri2);
    } catch (GATObjectCreationException e) {
        System.err.println("failed to create file2 at location '" + uri2 + "': " + e);
        return;
    }
    try {
        file2.move(uri3);
        System.out.println("file2 at location '" + uri2 + "' moved to file3 at location '" + uri3 + "'");
    } catch (GATInvocationException e) {
        System.err.println("failed to move file2 at location '" + uri2 + "' to file3 at location '" + uri3 + "': " + e);
        return;
    }
}
Also used : GATObjectCreationException(org.gridlab.gat.GATObjectCreationException) GATInvocationException(org.gridlab.gat.GATInvocationException) File(org.gridlab.gat.io.File)

Aggregations

GATObjectCreationException (org.gridlab.gat.GATObjectCreationException)47 GATInvocationException (org.gridlab.gat.GATInvocationException)39 URI (org.gridlab.gat.URI)18 JobDescription (org.gridlab.gat.resources.JobDescription)18 URISyntaxException (java.net.URISyntaxException)17 SoftwareDescription (org.gridlab.gat.resources.SoftwareDescription)17 ResourceBroker (org.gridlab.gat.resources.ResourceBroker)16 FileInterface (org.gridlab.gat.io.FileInterface)14 Job (org.gridlab.gat.resources.Job)14 IOException (java.io.IOException)11 Preferences (org.gridlab.gat.Preferences)9 BufferedReader (java.io.BufferedReader)5 File (java.io.File)5 InputStreamReader (java.io.InputStreamReader)5 FileOutputStream (java.io.FileOutputStream)4 PrintWriter (java.io.PrintWriter)4 StringWriter (java.io.StringWriter)4 AdaptorNotApplicableException (org.gridlab.gat.AdaptorNotApplicableException)4 GATContext (org.gridlab.gat.GATContext)4 InputStream (java.io.InputStream)3