Search in sources :

Example 1 with ImmediateService

use of org.objectweb.proactive.annotation.ImmediateService in project scheduling by ow2-proactive.

the class RMCore method setNodesAvailable.

/**
 * This method is called periodically by ProActive Nodes to inform the
 * Resource Manager of a possible reconnection. The method is also used by
 * ProActive Nodes to know if they are still known by the Resource Manager.
 * For instance a Node which has been removed by a user from the
 * Resource Manager is no longer known.
 * <p>
 * The method is defined as Immediate Service. This way it is executed in
 * a dedicated Thread. It is essential in order to allow other methods to
 * be executed immediately even if incoming connection to the Nodes is stopped
 * or filtered while a timeout occurs when this method tries to send back a reply.
 * <p>
 * The {@code allNodes} data-structure is written by a single Thread only
 * but read by multiple Threads.
 * <p>
 * Parallel executions of this method must involves different {@code nodeUrl}s.
 * <p>
 * The underlying calls to {@code setBusyNode} and {@code internalSetFree}
 * are writing to the {@code freeNodes} data-structure. It explains why this last
 * is synchronized (thread-safe).
 *
 * @param nodeUrls the URLs of the workers associated to the node that publishes the update.
 * @return The set of worker node URLs that are unknown to the Resource Manager
 * (i.e. have been removed by a user).
 */
@ImmediateService
@Override
public Set<String> setNodesAvailable(Set<String> nodeUrls) {
    waitForRMCoreToBeInitializedIfNeeded();
    if (logger.isTraceEnabled()) {
        logger.trace("Received availability for the following workers: " + nodeUrls);
    }
    ImmutableSet.Builder<String> nodeUrlsNotKnownByTheRM = new ImmutableSet.Builder<>();
    for (String nodeUrl : nodeUrls) {
        RMNode node = this.allNodes.get(nodeUrl);
        if (node == null) {
            logger.warn("Cannot set node as available, the node is unknown: " + nodeUrl);
            if (logger.isDebugEnabled()) {
                logger.debug("Known nodes are: " + Arrays.toString(allNodes.keySet().toArray()));
            }
            nodeUrlsNotKnownByTheRM.add(nodeUrl);
        } else if (node.isDown()) {
            restoreNodeState(nodeUrl, node);
        } else {
            if (logger.isDebugEnabled()) {
                logger.debug("The node identified by " + nodeUrl + " is known and not DOWN, no action performed");
            }
        }
    }
    return nodeUrlsNotKnownByTheRM.build();
}
Also used : RMNode(org.ow2.proactive.resourcemanager.rmnode.RMNode) ImmutableSet(com.google.common.collect.ImmutableSet) ImmediateService(org.objectweb.proactive.annotation.ImmediateService)

Example 2 with ImmediateService

use of org.objectweb.proactive.annotation.ImmediateService in project scheduling by ow2-proactive.

the class SchedulerFrontend method getTaskServerLogs.

@Override
@ImmediateService
public String getTaskServerLogs(String jobId, String taskName) throws UnknownJobException, UnknownTaskException, NotConnectedException, PermissionException {
    JobId id = JobIdImpl.makeJobId(jobId);
    frontendState.checkPermissions("getTaskServerLogs", frontendState.getIdentifiedJob(id), YOU_DO_NOT_HAVE_PERMISSION_TO_GET_THE_TASK_LOGS_OF_THIS_JOB);
    for (TaskId taskId : frontendState.getJobTasks(id)) {
        if (taskId.getReadableName().equals(taskName)) {
            return ServerJobAndTaskLogs.getTaskLog(taskId);
        }
    }
    throw new UnknownTaskException("Unknown task " + taskName + " in job " + jobId);
}
Also used : UnknownTaskException(org.ow2.proactive.scheduler.common.exception.UnknownTaskException) TaskId(org.ow2.proactive.scheduler.common.task.TaskId) JobId(org.ow2.proactive.scheduler.common.job.JobId) ImmediateService(org.objectweb.proactive.annotation.ImmediateService)

Example 3 with ImmediateService

use of org.objectweb.proactive.annotation.ImmediateService in project scheduling by ow2-proactive.

the class SchedulerFrontend method getJobs.

/**
 * {@inheritDoc}
 */
@Override
@ImmediateService
public Page<JobInfo> getJobs(int offset, int limit, JobFilterCriteria filterCriteria, List<SortParameter<JobSortParameter>> sortParameters) throws NotConnectedException, PermissionException {
    UserIdentificationImpl ident = frontendState.checkPermission("getJobs", "You don't have permissions to load jobs");
    boolean myJobsOnly = filterCriteria.isMyJobsOnly();
    String user;
    if (myJobsOnly) {
        user = ident.getUsername();
    } else {
        user = null;
    }
    return dbManager.getJobs(offset, limit, user, filterCriteria.isPending(), filterCriteria.isRunning(), filterCriteria.isFinished(), sortParameters);
}
Also used : UserIdentificationImpl(org.ow2.proactive.scheduler.job.UserIdentificationImpl) ImmediateService(org.objectweb.proactive.annotation.ImmediateService)

Example 4 with ImmediateService

use of org.objectweb.proactive.annotation.ImmediateService in project scheduling by ow2-proactive.

the class SchedulerFrontend method submit.

/**
 * {@inheritDoc}
 */
@Override
@ImmediateService
public JobId submit(Job userJob) throws NotConnectedException, PermissionException, SubmissionClosedException, JobCreationException {
    try {
        if (logger.isDebugEnabled()) {
            logger.debug("New job submission requested : " + userJob.getName());
        }
        // check if the scheduler is stopped
        if (!schedulingService.isSubmitPossible()) {
            String msg = "Scheduler is stopped, cannot submit job";
            logger.info(msg);
            throw new SubmissionClosedException(msg);
        }
        UserIdentificationImpl ident = frontendState.checkPermission("submit", YOU_DO_NOT_HAVE_PERMISSION_TO_SUBMIT_A_JOB);
        InternalJob job = frontendState.createJob(userJob, ident);
        schedulingService.submitJob(job);
        frontendState.jobSubmitted(job, ident);
        return job.getId();
    } catch (Exception e) {
        logger.warn("Error when submitting job.", e);
        throw e;
    }
}
Also used : InternalJob(org.ow2.proactive.scheduler.job.InternalJob) UserIdentificationImpl(org.ow2.proactive.scheduler.job.UserIdentificationImpl) KeyException(java.security.KeyException) UnknownJobException(org.ow2.proactive.scheduler.common.exception.UnknownJobException) TaskCouldNotRestartException(org.ow2.proactive.scheduler.common.exception.TaskCouldNotRestartException) JobCreationException(org.ow2.proactive.scheduler.common.exception.JobCreationException) PermissionException(org.ow2.proactive.scheduler.common.exception.PermissionException) NotConnectedException(org.ow2.proactive.scheduler.common.exception.NotConnectedException) AlreadyConnectedException(org.ow2.proactive.scheduler.common.exception.AlreadyConnectedException) UnknownTaskException(org.ow2.proactive.scheduler.common.exception.UnknownTaskException) TaskCouldNotStartException(org.ow2.proactive.scheduler.common.exception.TaskCouldNotStartException) JobAlreadyFinishedException(org.ow2.proactive.scheduler.common.exception.JobAlreadyFinishedException) SubmissionClosedException(org.ow2.proactive.scheduler.common.exception.SubmissionClosedException) DatabaseManagerException(org.ow2.proactive.db.DatabaseManagerException) TaskSkippedException(org.ow2.proactive.scheduler.common.exception.TaskSkippedException) SubmissionClosedException(org.ow2.proactive.scheduler.common.exception.SubmissionClosedException) ImmediateService(org.objectweb.proactive.annotation.ImmediateService)

Example 5 with ImmediateService

use of org.objectweb.proactive.annotation.ImmediateService in project scheduling by ow2-proactive.

the class SchedulerFrontend method getTaskResultFromIncarnation.

/**
 * {@inheritDoc}
 */
@Override
@ImmediateService
public TaskResult getTaskResultFromIncarnation(JobId jobId, String taskName, int inc) throws NotConnectedException, UnknownJobException, UnknownTaskException, PermissionException {
    // checking permissions
    frontendState.checkPermissions("getTaskResultFromIncarnation", frontendState.getIdentifiedJob(jobId), YOU_DO_NOT_HAVE_PERMISSION_TO_GET_THE_TASK_RESULT_OF_THIS_JOB);
    if (inc < 0) {
        throw new IllegalArgumentException("Incarnation must be 0 or greater.");
    }
    jlogger.debug(jobId, "trying to get the task result, incarnation " + inc);
    if (inc < 0) {
        throw new IllegalArgumentException("Incarnation must be 0 or greater.");
    }
    try {
        TaskResult result = dbManager.loadTaskResult(jobId, taskName, inc);
        // handling special statuses
        TaskState ts = frontendState.getTaskState(jobId, taskName);
        switch(ts.getStatus()) {
            case NOT_STARTED:
                if (result == null) {
                    return new TaskResultImpl(frontendState.getTaskId(jobId, taskName), new TaskCouldNotStartException(), new SimpleTaskLogs("", "The task could not start due to dependency failure"), 0);
                } else {
                    Throwable newException = new TaskCouldNotStartException("The task could not start due to dependency failure", result.getException());
                    ((TaskResultImpl) result).setException(newException);
                }
                break;
            case NOT_RESTARTED:
                if (result == null) {
                    return new TaskResultImpl(frontendState.getTaskId(jobId, taskName), new TaskCouldNotRestartException(), new SimpleTaskLogs("", "The task could not be restarted after an error during the previous execution"), 0);
                } else {
                    Throwable newException = new TaskCouldNotRestartException("The task could not be restarted after an error during the previous execution", result.getException());
                    ((TaskResultImpl) result).setException(newException);
                }
                break;
            case SKIPPED:
                // result should always be null
                return new TaskResultImpl(frontendState.getTaskId(jobId, taskName), new TaskSkippedException(), new SimpleTaskLogs("", "The task was skipped in the workflow"), 0);
        }
        if (result == null) {
            // otherwise the task is not finished
            jlogger.info(jobId, taskName + " is not finished");
            return null;
        } else {
            return result;
        }
    } catch (DatabaseManagerException e) {
        throw new UnknownTaskException("Unknown task " + taskName + ", job: " + jobId);
    }
}
Also used : SimpleTaskLogs(org.ow2.proactive.scheduler.common.task.SimpleTaskLogs) UnknownTaskException(org.ow2.proactive.scheduler.common.exception.UnknownTaskException) TaskCouldNotStartException(org.ow2.proactive.scheduler.common.exception.TaskCouldNotStartException) TaskCouldNotRestartException(org.ow2.proactive.scheduler.common.exception.TaskCouldNotRestartException) TaskResultImpl(org.ow2.proactive.scheduler.task.TaskResultImpl) TaskSkippedException(org.ow2.proactive.scheduler.common.exception.TaskSkippedException) TaskResult(org.ow2.proactive.scheduler.common.task.TaskResult) TaskState(org.ow2.proactive.scheduler.common.task.TaskState) DatabaseManagerException(org.ow2.proactive.db.DatabaseManagerException) ImmediateService(org.objectweb.proactive.annotation.ImmediateService)

Aggregations

ImmediateService (org.objectweb.proactive.annotation.ImmediateService)12 JobId (org.ow2.proactive.scheduler.common.job.JobId)5 UnknownTaskException (org.ow2.proactive.scheduler.common.exception.UnknownTaskException)4 TaskId (org.ow2.proactive.scheduler.common.task.TaskId)3 TaskState (org.ow2.proactive.scheduler.common.task.TaskState)3 DatabaseManagerException (org.ow2.proactive.db.DatabaseManagerException)2 TaskCouldNotRestartException (org.ow2.proactive.scheduler.common.exception.TaskCouldNotRestartException)2 TaskCouldNotStartException (org.ow2.proactive.scheduler.common.exception.TaskCouldNotStartException)2 TaskSkippedException (org.ow2.proactive.scheduler.common.exception.TaskSkippedException)2 UnknownJobException (org.ow2.proactive.scheduler.common.exception.UnknownJobException)2 TaskResult (org.ow2.proactive.scheduler.common.task.TaskResult)2 UserIdentificationImpl (org.ow2.proactive.scheduler.job.UserIdentificationImpl)2 ImmutableSet (com.google.common.collect.ImmutableSet)1 KeyException (java.security.KeyException)1 ArrayList (java.util.ArrayList)1 HashSet (java.util.HashSet)1 LoginException (javax.security.auth.login.LoginException)1 RMNode (org.ow2.proactive.resourcemanager.rmnode.RMNode)1 AlreadyConnectedException (org.ow2.proactive.scheduler.common.exception.AlreadyConnectedException)1 JobAlreadyFinishedException (org.ow2.proactive.scheduler.common.exception.JobAlreadyFinishedException)1