Search in sources :

Example 1 with RMProxyCreationException

use of org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException in project scheduling by ow2-proactive.

the class RMProxiesManager method createRMProxiesManager.

/**
 * Create a RMProxiesManager using RM's URI (example : "rmi://localhost:1099/" ).
 *
 * @param rmURI The URI of a started Resource Manager
 * @return an instance of RMProxiesManager joined to the Resource Manager at the given URI
 */
public static RMProxiesManager createRMProxiesManager(final URI rmURI) throws RMException, RMProxyCreationException, URISyntaxException {
    Credentials schedulerProxyCredentials;
    try {
        schedulerProxyCredentials = Credentials.getCredentials(PASchedulerProperties.getAbsolutePath(PASchedulerProperties.RESOURCE_MANAGER_CREDS.getValueAsString()));
    } catch (Exception e) {
        throw new RMProxyCreationException(e);
    }
    boolean singleConnection = PASchedulerProperties.RESOURCE_MANAGER_SINGLE_CONNECTION.getValueAsBoolean();
    if (singleConnection) {
        return new SingleConnectionRMProxiesManager(rmURI, schedulerProxyCredentials);
    } else {
        return new PerUserConnectionRMProxiesManager(rmURI, schedulerProxyCredentials);
    }
}
Also used : Credentials(org.ow2.proactive.authentication.crypto.Credentials) URISyntaxException(java.net.URISyntaxException) RMException(org.ow2.proactive.resourcemanager.exception.RMException)

Example 2 with RMProxyCreationException

use of org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException in project scheduling by ow2-proactive.

the class TerminationDataTest method testHandleTerminationForTaskToRestart.

@Test
public void testHandleTerminationForTaskToRestart() throws RMProxyCreationException, IOException, ClassNotFoundException {
    JobId jobId = new JobIdImpl(666, "readableName");
    TaskId taskId = TaskIdImpl.createTaskId(jobId, "task-name", 777L);
    terminationData.addRestartData(taskId, 1000L);
    terminationData.handleTermination(service);
    Mockito.verify(schedulingInfrastructure, Mockito.times(1)).schedule(org.mockito.Matchers.any(Runnable.class), org.mockito.Matchers.anyLong());
}
Also used : TaskId(org.ow2.proactive.scheduler.common.task.TaskId) JobIdImpl(org.ow2.proactive.scheduler.job.JobIdImpl) JobId(org.ow2.proactive.scheduler.common.job.JobId) Test(org.junit.Test)

Example 3 with RMProxyCreationException

use of org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException in project scheduling by ow2-proactive.

the class TerminationDataTest method testHandleTerminationForTaskNormalTermination.

@Test
public void testHandleTerminationForTaskNormalTermination() throws RMProxyCreationException, IOException, ClassNotFoundException {
    InternalJob job = new InternalTaskFlowJob("test-name", JobPriority.NORMAL, OnTaskError.CANCEL_JOB, "description");
    JobId jobId = new JobIdImpl(666, "readableName");
    InternalTask internalTask = new InternalScriptTask(job);
    TaskId taskId = TaskIdImpl.createTaskId(jobId, "task-name", 777L);
    internalTask.setId(taskId);
    internalTask.setName("task-name");
    internalTask.setStatus(TaskStatus.RUNNING);
    internalTask.setExecuterInformation(Mockito.mock(ExecuterInformation.class));
    RunningTaskData taskData = new RunningTaskData(internalTask, "user", null, launcher);
    terminationData.addTaskData(null, taskData, TerminationData.TerminationStatus.NORMAL, null);
    terminationData.handleTermination(service);
    Mockito.verify(proxiesManager, Mockito.times(1)).getUserRMProxy("user", null);
    Mockito.verify(rmProxy, Mockito.times(1)).releaseNodes(org.mockito.Matchers.any(NodeSet.class), org.mockito.Matchers.any(org.ow2.proactive.scripting.Script.class), Mockito.any(VariablesMap.class), Mockito.any(HashMap.class), Mockito.any(TaskId.class), Mockito.any(Credentials.class));
}
Also used : ExecuterInformation(org.ow2.proactive.scheduler.task.internal.ExecuterInformation) NodeSet(org.ow2.proactive.utils.NodeSet) InternalJob(org.ow2.proactive.scheduler.job.InternalJob) TaskId(org.ow2.proactive.scheduler.common.task.TaskId) InternalTask(org.ow2.proactive.scheduler.task.internal.InternalTask) HashMap(java.util.HashMap) InternalScriptTask(org.ow2.proactive.scheduler.task.internal.InternalScriptTask) VariablesMap(org.ow2.proactive.scheduler.task.utils.VariablesMap) JobIdImpl(org.ow2.proactive.scheduler.job.JobIdImpl) InternalTaskFlowJob(org.ow2.proactive.scheduler.job.InternalTaskFlowJob) JobId(org.ow2.proactive.scheduler.common.job.JobId) Credentials(org.ow2.proactive.authentication.crypto.Credentials) Test(org.junit.Test)

Example 4 with RMProxyCreationException

use of org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException in project scheduling by ow2-proactive.

the class SchedulingMethodImpl method getRMNodes.

/**
 * Ask to the RM the given number of node resources.<br>
 * If there is a problem with these task selection (such as bad selectionScript) this method
 * will terminate the corresponding tasks and jobs. As the selection scripts contain errors, the task
 * and its surrounding jobs must be stopped.
 *
 * @param neededResourcesNumber the number of resources to ask for (must be &gt; 0).
 * @param tasksToSchedule the task to be scheduled
 * @return A nodeSet that contains at most 'neededResourcesNumber' available compatible resources.
 * 		   An empty nodeSet if no nodes could be found
 * 		   null if the their was an exception when asking for the nodes (ie : selection script has failed)
 */
protected NodeSet getRMNodes(Map<JobId, JobDescriptor> jobMap, int neededResourcesNumber, LinkedList<EligibleTaskDescriptor> tasksToSchedule, Set<String> freeResources) {
    NodeSet nodeSet;
    if (neededResourcesNumber <= 0) {
        throw new IllegalArgumentException("'neededResourcesNumber' must be greater than 0");
    }
    EligibleTaskDescriptor etd = tasksToSchedule.getFirst();
    InternalJob currentJob = ((JobDescriptorImpl) jobMap.get(etd.getJobId())).getInternal();
    InternalTask internalTask0 = currentJob.getIHMTasks().get(etd.getTaskId());
    try {
        TopologyDescriptor descriptor = null;
        boolean bestEffort = true;
        if (internalTask0.isParallel()) {
            descriptor = internalTask0.getParallelEnvironment().getTopologyDescriptor();
            bestEffort = false;
            if (descriptor == null) {
                logger.debug("Topology is not defined for the task " + internalTask0.getName());
            }
        }
        if (descriptor == null) {
            // descriptor is not defined, use default
            descriptor = TopologyDescriptor.ARBITRARY;
        }
        try {
            Criteria criteria = new Criteria(neededResourcesNumber);
            criteria.setTopology(descriptor);
            // resolve script variables (if any) in the list of selection
            // scripts and then set it as the selection criteria.
            criteria.setScripts(resolveScriptVariables(internalTask0.getSelectionScripts(), internalTask0.getRuntimeVariables()));
            criteria.setBlackList(internalTask0.getNodeExclusion());
            criteria.setBestEffort(bestEffort);
            criteria.setAcceptableNodesUrls(freeResources);
            criteria.setBindings(createBindingsForSelectionScripts(currentJob, internalTask0));
            if (internalTask0.getRuntimeGenericInformation().containsKey(SchedulerConstants.NODE_ACCESS_TOKEN)) {
                criteria.setNodeAccessToken(internalTask0.getRuntimeGenericInformation().get(SchedulerConstants.NODE_ACCESS_TOKEN));
            }
            Collection<String> computationDescriptors = new ArrayList<>(tasksToSchedule.size());
            for (EligibleTaskDescriptor task : tasksToSchedule) {
                computationDescriptors.add(TaskLogger.getTaskLogRelativePath(task.getTaskId()));
            }
            criteria.setComputationDescriptors(computationDescriptors);
            nodeSet = getRMProxiesManager().getUserRMProxy(currentJob.getOwner(), currentJob.getCredentials()).getNodes(criteria);
        } catch (TopologyDisabledException tde) {
            jlogger.warn(currentJob.getId(), "will be canceled as the topology is disabled");
            schedulingService.simulateJobStartAndCancelIt(tasksToSchedule, "Topology is disabled");
            return null;
        }
        // the following line is used to unwrap the future, warning when moving or removing
        // it may also throw a ScriptException which is a RuntimeException
        PAFuture.waitFor(nodeSet, true);
        logger.debug("provided nodes " + nodeSet.size());
        return nodeSet;
    } catch (IOException | ClassNotFoundException e) {
        logger.warn("Failed to deserialize previous task variables before selection for task " + internalTask0.getId().toString(), e);
        schedulingService.simulateJobStartAndCancelIt(tasksToSchedule, "Failed to deserialize previous task variables before selection for task " + internalTask0.getId().toString());
        return null;
    } catch (RMProxyCreationException e) {
        logger.warn("Failed to create User RM Proxy", e);
        // simulate jobs starts and cancel it
        schedulingService.simulateJobStartAndCancelIt(tasksToSchedule, "Failed to create User RM Proxy : Authentication Failed to Resource Manager for user '" + currentJob.getOwner() + "'");
        // leave the method by ss failure
        return null;
    }
}
Also used : NodeSet(org.ow2.proactive.utils.NodeSet) InternalJob(org.ow2.proactive.scheduler.job.InternalJob) InternalTask(org.ow2.proactive.scheduler.task.internal.InternalTask) ArrayList(java.util.ArrayList) Criteria(org.ow2.proactive.utils.Criteria) IOException(java.io.IOException) JobDescriptorImpl(org.ow2.proactive.scheduler.descriptor.JobDescriptorImpl) EligibleTaskDescriptor(org.ow2.proactive.scheduler.descriptor.EligibleTaskDescriptor) RMProxyCreationException(org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException) TopologyDisabledException(org.ow2.proactive.resourcemanager.frontend.topology.TopologyDisabledException) TopologyDescriptor(org.ow2.proactive.topology.descriptor.TopologyDescriptor)

Example 5 with RMProxyCreationException

use of org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException in project scheduling by ow2-proactive.

the class RMProxy method init.

public synchronized void init() throws RMException, RMProxyCreationException {
    RMAuthentication auth = RMConnection.join(rmURL.toString());
    proxyActiveObject = RMProxyActiveObject.createAOProxy(auth, creds);
    currentRMConnection = new RMProxiesManager.Connection(rmURL, auth);
}
Also used : RMAuthentication(org.ow2.proactive.resourcemanager.authentication.RMAuthentication)

Aggregations

Test (org.junit.Test)2 Credentials (org.ow2.proactive.authentication.crypto.Credentials)2 JobId (org.ow2.proactive.scheduler.common.job.JobId)2 TaskId (org.ow2.proactive.scheduler.common.task.TaskId)2 InternalJob (org.ow2.proactive.scheduler.job.InternalJob)2 JobIdImpl (org.ow2.proactive.scheduler.job.JobIdImpl)2 InternalTask (org.ow2.proactive.scheduler.task.internal.InternalTask)2 NodeSet (org.ow2.proactive.utils.NodeSet)2 IOException (java.io.IOException)1 URISyntaxException (java.net.URISyntaxException)1 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 RMAuthentication (org.ow2.proactive.resourcemanager.authentication.RMAuthentication)1 RMException (org.ow2.proactive.resourcemanager.exception.RMException)1 TopologyDisabledException (org.ow2.proactive.resourcemanager.frontend.topology.TopologyDisabledException)1 RMProxyCreationException (org.ow2.proactive.scheduler.core.rmproxies.RMProxyCreationException)1 EligibleTaskDescriptor (org.ow2.proactive.scheduler.descriptor.EligibleTaskDescriptor)1 JobDescriptorImpl (org.ow2.proactive.scheduler.descriptor.JobDescriptorImpl)1 InternalTaskFlowJob (org.ow2.proactive.scheduler.job.InternalTaskFlowJob)1 ExecuterInformation (org.ow2.proactive.scheduler.task.internal.ExecuterInformation)1