Search in sources :

Example 16 with TaskResultImpl

use of org.ow2.proactive.scheduler.task.TaskResultImpl in project scheduling by ow2-proactive.

the class InProcessTaskExecutorTest method variablesPropagation_fromParentTask.

@Test
public void variablesPropagation_fromParentTask() throws Throwable {
    TestTaskOutput taskOutput = new TestTaskOutput();
    TaskLauncherInitializer initializer = new TaskLauncherInitializer();
    initializer.setTaskId(TaskIdImpl.createTaskId(new JobIdImpl(1000, "job"), "task", 42L));
    Map<String, Serializable> variablesFromParent = new HashMap<>();
    variablesFromParent.put("var", "parent");
    variablesFromParent.put(SchedulerVars.PA_TASK_ID.toString(), "1234");
    TaskResult[] previousTasksResults = { new TaskResultImpl(null, null, null, null, null, SerializationUtil.serializeVariableMap(variablesFromParent), false) };
    new InProcessTaskExecutor().execute(new TaskContext(new ScriptExecutableContainer(new TaskScript(new SimpleScript("print(variables.get('var'));print(variables.get('PA_TASK_ID'))", "groovy"))), initializer, previousTasksResults, new NodeDataSpacesURIs("", "", "", "", "", ""), "", new NodeInfo("", "", "", "")), taskOutput.outputStream, taskOutput.error);
    assertEquals("parent42", taskOutput.output());
}
Also used : Serializable(java.io.Serializable) TaskContext(org.ow2.proactive.scheduler.task.context.TaskContext) TaskScript(org.ow2.proactive.scripting.TaskScript) InProcessTaskExecutor(org.ow2.proactive.scheduler.task.executors.InProcessTaskExecutor) HashMap(java.util.HashMap) SimpleScript(org.ow2.proactive.scripting.SimpleScript) ScriptExecutableContainer(org.ow2.proactive.scheduler.task.containers.ScriptExecutableContainer) NodeDataSpacesURIs(org.ow2.proactive.scheduler.task.context.NodeDataSpacesURIs) NodeInfo(org.ow2.proactive.scheduler.task.context.NodeInfo) TaskResult(org.ow2.proactive.scheduler.common.task.TaskResult) JobIdImpl(org.ow2.proactive.scheduler.job.JobIdImpl) Test(org.junit.Test)

Example 17 with TaskResultImpl

use of org.ow2.proactive.scheduler.task.TaskResultImpl in project scheduling by ow2-proactive.

the class InProcessTaskExecutorTest method simpleScriptTask.

@Test
public void simpleScriptTask() throws Throwable {
    TestTaskOutput taskOutput = new TestTaskOutput();
    TaskLauncherInitializer initializer = new TaskLauncherInitializer();
    initializer.setPreScript(new SimpleScript("println('pre')", "groovy"));
    initializer.setPostScript(new SimpleScript("println('post')", "groovy"));
    initializer.setTaskId(TaskIdImpl.createTaskId(JobIdImpl.makeJobId("1000"), "job", 1000L));
    TaskResultImpl result = new InProcessTaskExecutor().execute(new TaskContext(new ScriptExecutableContainer(new TaskScript(new SimpleScript("println('hello'); java.lang.Thread.sleep(5); result='hello'", "groovy"))), initializer, null, new NodeDataSpacesURIs("", "", "", "", "", ""), "", new NodeInfo("", "", "", "")), taskOutput.outputStream, taskOutput.error);
    assertEquals(String.format("pre%nhello%npost%n"), taskOutput.output());
    assertEquals("hello", result.value());
    assertTrue("Task duration should be at least 5", result.getTaskDuration() >= 5);
}
Also used : TaskContext(org.ow2.proactive.scheduler.task.context.TaskContext) TaskScript(org.ow2.proactive.scripting.TaskScript) InProcessTaskExecutor(org.ow2.proactive.scheduler.task.executors.InProcessTaskExecutor) NodeInfo(org.ow2.proactive.scheduler.task.context.NodeInfo) SimpleScript(org.ow2.proactive.scripting.SimpleScript) ScriptExecutableContainer(org.ow2.proactive.scheduler.task.containers.ScriptExecutableContainer) NodeDataSpacesURIs(org.ow2.proactive.scheduler.task.context.NodeDataSpacesURIs) Test(org.junit.Test)

Example 18 with TaskResultImpl

use of org.ow2.proactive.scheduler.task.TaskResultImpl in project scheduling by ow2-proactive.

the class ForkedTaskExecutor method createTaskResult.

private TaskResultImpl createTaskResult(TaskContext context, Throwable throwable) {
    TaskResultImpl result = new TaskResultImpl(context.getTaskId(), new ForkedJvmProcessException("Failed to execute task in a forked JVM", throwable));
    result.setPropagatedVariables(new TaskContextVariableExtractor().extractPropagatedVariables(context));
    return result;
}
Also used : ForkedJvmProcessException(org.ow2.proactive.scheduler.task.exceptions.ForkedJvmProcessException) TaskContextVariableExtractor(org.ow2.proactive.scheduler.task.context.TaskContextVariableExtractor) TaskResultImpl(org.ow2.proactive.scheduler.task.TaskResultImpl)

Example 19 with TaskResultImpl

use of org.ow2.proactive.scheduler.task.TaskResultImpl in project scheduling by ow2-proactive.

the class ForkedTaskExecutor method execute.

@Override
public TaskResultImpl execute(TaskContext context, PrintStream outputSink, PrintStream errorSink) {
    CookieBasedProcessTreeKiller taskProcessTreeKiller = null;
    Process process = null;
    ProcessStreamsReader processStreamsReader = null;
    File serializedContext = null;
    try {
        if (!workingDir.exists()) {
            FileUtils.forceMkdir(workingDir);
        }
        serializedContext = taskContextSerializer.serializeContext(context, workingDir);
        OSProcessBuilder processBuilder = forkedJvmProcessBuilderCreator.createForkedProcessBuilder(context, serializedContext, outputSink, errorSink, workingDir);
        TaskId taskId = context.getTaskId();
        if (context.getInitializer().getGenericInformation() == null || !"true".equalsIgnoreCase(context.getInitializer().getGenericInformation().get(SchedulerConstants.DISABLE_PROCESS_TREE_KILLER_GENERIC_INFO))) {
            String cookieNameSuffix = "Job" + taskId.getJobId().value() + "Task" + taskId.value();
            taskProcessTreeKiller = CookieBasedProcessTreeKiller.createProcessChildrenKiller(cookieNameSuffix, processBuilder.environment());
        }
        process = processBuilder.start();
        processStreamsReader = new ProcessStreamsReader(taskId.toString(), process, outputSink, errorSink);
        int exitCode = process.waitFor();
        if (exitCode != 0) {
            try {
                Object error = deserializeTaskResult(serializedContext);
                if (error instanceof TaskContext) {
                    return createTaskResult(context, new IOException("Forked JVM process returned with exit code " + exitCode + ", see task logs for more information"));
                } else {
                    Throwable exception = (Throwable) error;
                    return createTaskResult(context, exception);
                }
            } catch (Throwable cannotDeserializeResult) {
                return createTaskResult(context, cannotDeserializeResult);
            }
        }
        return (TaskResultImpl) deserializeTaskResult(serializedContext);
    } catch (Throwable throwable) {
        return createTaskResult(context, throwable);
    } finally {
        FileUtils.deleteQuietly(serializedContext);
        if (process != null) {
            logger.info("killing forked JVM process");
            process.destroy();
            try {
                // wait for twice the time of the cleanup process
                process.waitFor((new CleanupTimeoutGetter()).getCleanupTimeSeconds(), TimeUnit.SECONDS);
                logger.info("JVM process killed");
            } catch (Exception e) {
                logger.info("Exception while waiting task to finish " + e);
            }
        }
        if (taskProcessTreeKiller != null) {
            taskProcessTreeKiller.kill();
        }
        if (processStreamsReader != null) {
            processStreamsReader.close();
        }
    }
}
Also used : TaskId(org.ow2.proactive.scheduler.common.task.TaskId) TaskContext(org.ow2.proactive.scheduler.task.context.TaskContext) TaskResultImpl(org.ow2.proactive.scheduler.task.TaskResultImpl) CleanupTimeoutGetter(org.ow2.proactive.scheduler.task.utils.task.termination.CleanupTimeoutGetter) IOException(java.io.IOException) IOException(java.io.IOException) ForkedJvmProcessException(org.ow2.proactive.scheduler.task.exceptions.ForkedJvmProcessException) OSProcessBuilder(org.objectweb.proactive.extensions.processbuilder.OSProcessBuilder) ProcessStreamsReader(org.ow2.proactive.scheduler.task.utils.ProcessStreamsReader) CookieBasedProcessTreeKiller(org.ow2.proactive.utils.CookieBasedProcessTreeKiller) File(java.io.File)

Example 20 with TaskResultImpl

use of org.ow2.proactive.scheduler.task.TaskResultImpl in project scheduling by ow2-proactive.

the class SchedulingServiceTest8 method testTaskPreempt.

@Test
public void testTaskPreempt() throws Exception {
    service.submitJob(createJob(createTestJob()));
    listener.assertEvents(SchedulerEvent.JOB_SUBMITTED);
    JobDescriptor jobDesc = startTask();
    try {
        service.preemptTask(jobDesc.getJobId(), "invalid task name", 100);
        Assert.fail();
    } catch (UnknownTaskException e) {
    }
    try {
        service.preemptTask(JobIdImpl.makeJobId("1234567"), "javaTask", 100);
        Assert.fail();
    } catch (UnknownJobException e) {
    }
    service.preemptTask(jobDesc.getJobId(), "javaTask", 100);
    listener.assertEvents(SchedulerEvent.JOB_PENDING_TO_RUNNING, SchedulerEvent.JOB_UPDATED, SchedulerEvent.TASK_PENDING_TO_RUNNING, SchedulerEvent.TASK_WAITING_FOR_RESTART);
    infrastructure.assertRequests(1);
    startTask();
    service.preemptTask(jobDesc.getJobId(), "javaTask", 100);
    listener.assertEvents(SchedulerEvent.TASK_PENDING_TO_RUNNING, SchedulerEvent.TASK_WAITING_FOR_RESTART);
    infrastructure.assertRequests(1);
    startTask();
    TaskId taskId = ((JobDescriptorImpl) jobDesc).getInternal().getTask("javaTask").getId();
    service.taskTerminatedWithResult(taskId, new TaskResultImpl(taskId, "OK", null, 0));
    listener.assertEvents(SchedulerEvent.TASK_PENDING_TO_RUNNING, SchedulerEvent.TASK_RUNNING_TO_FINISHED, SchedulerEvent.JOB_RUNNING_TO_FINISHED, SchedulerEvent.JOB_UPDATED);
    infrastructure.assertRequests(1);
    service.removeJob(jobDesc.getJobId());
    try {
        service.preemptTask(jobDesc.getJobId(), "javaTask", 100);
        Assert.fail();
    } catch (UnknownJobException e) {
    }
}
Also used : UnknownTaskException(org.ow2.proactive.scheduler.common.exception.UnknownTaskException) TaskId(org.ow2.proactive.scheduler.common.task.TaskId) TaskResultImpl(org.ow2.proactive.scheduler.task.TaskResultImpl) JobDescriptor(org.ow2.proactive.scheduler.common.JobDescriptor) UnknownJobException(org.ow2.proactive.scheduler.common.exception.UnknownJobException) JobDescriptorImpl(org.ow2.proactive.scheduler.descriptor.JobDescriptorImpl) Test(org.junit.Test)

Aggregations

TaskResultImpl (org.ow2.proactive.scheduler.task.TaskResultImpl)79 Test (org.junit.Test)77 InternalJob (org.ow2.proactive.scheduler.job.InternalJob)35 InternalTask (org.ow2.proactive.scheduler.task.internal.InternalTask)35 ScriptExecutableContainer (org.ow2.proactive.scheduler.task.containers.ScriptExecutableContainer)30 SimpleScript (org.ow2.proactive.scripting.SimpleScript)30 TaskScript (org.ow2.proactive.scripting.TaskScript)30 TaskContext (org.ow2.proactive.scheduler.task.context.TaskContext)29 TaskId (org.ow2.proactive.scheduler.common.task.TaskId)26 NodeDataSpacesURIs (org.ow2.proactive.scheduler.task.context.NodeDataSpacesURIs)26 NodeInfo (org.ow2.proactive.scheduler.task.context.NodeInfo)26 TaskResult (org.ow2.proactive.scheduler.common.task.TaskResult)22 InProcessTaskExecutor (org.ow2.proactive.scheduler.task.executors.InProcessTaskExecutor)19 HashMap (java.util.HashMap)17 JobId (org.ow2.proactive.scheduler.common.job.JobId)17 JobIdImpl (org.ow2.proactive.scheduler.job.JobIdImpl)16 UnknownJobException (org.ow2.proactive.scheduler.common.exception.UnknownJobException)13 UnknownTaskException (org.ow2.proactive.scheduler.common.exception.UnknownTaskException)13 TaskFlowJob (org.ow2.proactive.scheduler.common.job.TaskFlowJob)13 SimpleTaskLogs (org.ow2.proactive.scheduler.common.task.SimpleTaskLogs)12