Search in sources :

Example 11 with Message

use of edu.iu.dsc.tws.task.api.Message in project twister2 by DSC-SPIDAL.

the class RunnableFixedTask method run.

@Override
public void run() {
    if (executableTask == null) {
        throw new RuntimeException("Task needs to be set to execute");
    }
    LOG.info(String.format("Runnable task %d limit %d", executableTask.getTaskId(), messageProcessLimit));
    Message result;
    if (isMessageBased) {
        // TODO: check if this part needs to be synced
        while (!queueRef.isEmpty()) {
            if (messageProcessCount < messageProcessLimit) {
                result = executableTask.execute(queueRef.poll());
                if (result != null && outQueues != null && !outQueues.isEmpty()) {
                    submitToOutputQueue(result);
                }
                messageProcessCount++;
            } else {
                // Need to make sure the remaining tasks are processed
                LOG.info("Need to run more so resubmitting the task");
                TaskExecutorFixedThread.executorPool.submit(new RunnableFixedTask(executableTask, queueRef, messageProcessLimit, taskExecutor, outQueues));
                return;
            }
        }
        synchronized (ExecutorContext.FIXED_EXECUTOR_LOCK) {
            if (!queueRef.isEmpty()) {
                LOG.info("Need to run more so resubmitting the task");
                TaskExecutorFixedThread.executorPool.submit(new RunnableFixedTask(executableTask, queueRef, messageProcessLimit, taskExecutor, outQueues));
            } else {
                TaskExecutorFixedThread.removeSubmittedTask(executableTask.getTaskId());
            }
        }
    } else {
        result = executableTask.execute();
        if (result != null && outQueues != null && !outQueues.isEmpty()) {
            submitToOutputQueue(result);
        }
        TaskExecutorFixedThread.removeSubmittedTask(executableTask.getTaskId());
    }
}
Also used : Message(edu.iu.dsc.tws.task.api.Message)

Aggregations

Message (edu.iu.dsc.tws.task.api.Message)11 TWSCommunication (edu.iu.dsc.tws.comms.core.TWSCommunication)10 TWSNetwork (edu.iu.dsc.tws.comms.core.TWSNetwork)10 TaskPlan (edu.iu.dsc.tws.comms.core.TaskPlan)10 LinkedQueue (edu.iu.dsc.tws.task.api.LinkedQueue)10 TaskExecutorFixedThread (edu.iu.dsc.tws.task.core.TaskExecutorFixedThread)10 HashMap (java.util.HashMap)10 HashSet (java.util.HashSet)10 DataflowTaskGraphGenerator (edu.iu.dsc.tws.task.taskgraphbuilder.DataflowTaskGraphGenerator)7 ArrayList (java.util.ArrayList)5 SinkTask (edu.iu.dsc.tws.task.api.SinkTask)4 SourceTask (edu.iu.dsc.tws.task.api.SourceTask)4 Task (edu.iu.dsc.tws.task.api.Task)4 DataflowTaskGraphParser (edu.iu.dsc.tws.task.taskgraphbuilder.DataflowTaskGraphParser)4 DataflowOperation (edu.iu.dsc.tws.task.taskgraphbuilder.DataflowOperation)3 MessageReceiver (edu.iu.dsc.tws.comms.api.MessageReceiver)1 Path (edu.iu.dsc.tws.data.fs.Path)1 MemoryManager (edu.iu.dsc.tws.data.memory.MemoryManager)1 LMDBMemoryManager (edu.iu.dsc.tws.data.memory.lmdb.LMDBMemoryManager)1 PipelinedTask (edu.iu.dsc.tws.task.core.PipelinedTask)1