Search in sources :

Example 6 with TaskEventScheduleTask

use of org.apache.tez.dag.app.dag.event.TaskEventScheduleTask in project tez by apache.

the class VertexImpl method scheduleTasks.

@Override
public void scheduleTasks(List<ScheduleTaskRequest> tasksToSchedule) {
    try {
        unsetTasksNotYetScheduled();
        // update state under write lock
        writeLock.lock();
        try {
            for (ScheduleTaskRequest task : tasksToSchedule) {
                if (numTasks <= task.getTaskIndex()) {
                    throw new TezUncheckedException("Invalid taskId: " + task.getTaskIndex() + " for vertex: " + logIdentifier);
                }
                TaskLocationHint locationHint = task.getTaskLocationHint();
                if (locationHint != null) {
                    if (taskLocationHints == null) {
                        taskLocationHints = new TaskLocationHint[numTasks];
                    }
                    taskLocationHints[task.getTaskIndex()] = locationHint;
                }
            }
        } finally {
            writeLock.unlock();
        }
        /**
         * read lock is not needed here. For e.g after starting task
         * scheduling on the vertex, it would not change numTasks. Rest of
         * the methods creating remote task specs have their
         * own locking mechanisms. Ref: TEZ-3297
         */
        for (ScheduleTaskRequest task : tasksToSchedule) {
            TezTaskID taskId = TezTaskID.getInstance(vertexId, task.getTaskIndex());
            TaskSpec baseTaskSpec = createRemoteTaskSpec(taskId.getId());
            boolean fromRecovery = recoveryData == null ? false : recoveryData.getTaskRecoveryData(taskId) != null;
            eventHandler.handle(new TaskEventScheduleTask(taskId, baseTaskSpec, getTaskLocationHint(taskId), fromRecovery));
        }
    } catch (AMUserCodeException e) {
        String msg = "Exception in " + e.getSource() + ", vertex=" + getLogIdentifier();
        LOG.error(msg, e);
        // send event to fail the vertex
        eventHandler.handle(new VertexEventManagerUserCodeError(getVertexId(), e));
        // throw an unchecked exception to stop the vertex manager that invoked this.
        throw new TezUncheckedException(e);
    }
}
Also used : TaskLocationHint(org.apache.tez.dag.api.TaskLocationHint) TezUncheckedException(org.apache.tez.dag.api.TezUncheckedException) VertexEventManagerUserCodeError(org.apache.tez.dag.app.dag.event.VertexEventManagerUserCodeError) TaskSpec(org.apache.tez.runtime.api.impl.TaskSpec) TaskEventScheduleTask(org.apache.tez.dag.app.dag.event.TaskEventScheduleTask) ScheduleTaskRequest(org.apache.tez.dag.api.VertexManagerPluginContext.ScheduleTaskRequest) TezTaskID(org.apache.tez.dag.records.TezTaskID)

Aggregations

TaskEventScheduleTask (org.apache.tez.dag.app.dag.event.TaskEventScheduleTask)6 TaskLocationHint (org.apache.tez.dag.api.TaskLocationHint)4 TezTaskID (org.apache.tez.dag.records.TezTaskID)4 PlanTaskLocationHint (org.apache.tez.dag.api.records.DAGProtos.PlanTaskLocationHint)3 StateChangeNotifierForTest (org.apache.tez.dag.app.dag.TestStateChangeNotifier.StateChangeNotifierForTest)3 TaskSpec (org.apache.tez.runtime.api.impl.TaskSpec)3 EdgeManagerForTest (org.apache.tez.test.EdgeManagerForTest)3 GraceShuffleVertexManagerForTest (org.apache.tez.test.GraceShuffleVertexManagerForTest)3 VertexManagerPluginForTest (org.apache.tez.test.VertexManagerPluginForTest)3 Test (org.junit.Test)3 ProcessorDescriptor (org.apache.tez.dag.api.ProcessorDescriptor)2 TaskEventTAFailed (org.apache.tez.dag.app.dag.event.TaskEventTAFailed)2 TaskEventTASucceeded (org.apache.tez.dag.app.dag.event.TaskEventTASucceeded)2 TezTaskAttemptID (org.apache.tez.dag.records.TezTaskAttemptID)2 GroupInputSpec (org.apache.tez.runtime.api.impl.GroupInputSpec)2 InputSpec (org.apache.tez.runtime.api.impl.InputSpec)2 OutputSpec (org.apache.tez.runtime.api.impl.OutputSpec)2 TezUncheckedException (org.apache.tez.dag.api.TezUncheckedException)1 ScheduleTaskRequest (org.apache.tez.dag.api.VertexManagerPluginContext.ScheduleTaskRequest)1 Task (org.apache.tez.dag.app.dag.Task)1