Examples with TaskTracker org.apache.hadoop.hive.ql.exec.repl.util.TaskTracker used on opensource projects

Example 6 with TaskTracker

use of org.apache.hadoop.hive.ql.exec.repl.util.TaskTracker in project hive by apache.

the class ReplLoadTask method updateDatabaseLastReplID.

/**
 * There was a database update done before and we want to make sure we update the last repl
 * id on this database as we are now going to switch to processing a new database.
 * This has to be last task in the graph since if there are intermediate tasks and the last.repl.id
 * is a root level task then in the execution phase the root level tasks will get executed first,
 * however if any of the child tasks of the bootstrap load failed then even though the bootstrap has failed
 * the last repl status of the target database will return a valid value, which will not represent
 * the state of the database.
 */
private TaskTracker updateDatabaseLastReplID(int maxTasks, Context context, Scope scope) throws SemanticException {
    /*
    we don't want to put any limits on this task as this is essential before we start
    processing new database events.
   */
    TaskTracker taskTracker = new AlterDatabase(context, work.databaseEvent(context.hiveConf), work.dbNameToLoadIn, new TaskTracker(maxTasks), work.getMetricCollector()).tasks();
    AddDependencyToLeaves function = new AddDependencyToLeaves(taskTracker.tasks());
    DAGTraversal.traverse(scope.rootTasks, function);
    return taskTracker;
}

Example 7 with TaskTracker

use of org.apache.hadoop.hive.ql.exec.repl.util.TaskTracker in project hive by apache.

the class ReplLoadTask method addLoadConstraintsTasks.

private TaskTracker addLoadConstraintsTasks(Context loadContext, BootstrapEvent next, TaskTracker dbTracker, Scope scope) throws IOException, SemanticException {
    LoadConstraint loadConstraint = new LoadConstraint(loadContext, (ConstraintEvent) next, work.dbNameToLoadIn, dbTracker, (new Path(work.dumpDirectory)).getParent().toString(), work.getMetricCollector());
    TaskTracker constraintTracker = loadConstraint.tasks();
    scope.rootTasks.addAll(constraintTracker.tasks());
    constraintTracker.debugLog("constraints");
    return constraintTracker;
}

Example 8 with TaskTracker

use of org.apache.hadoop.hive.ql.exec.repl.util.TaskTracker in project hive by apache.

the class TestTaskTracker method taskTrackerCompositionInitializesTheMaxTasksCorrectly.

@Test
public void taskTrackerCompositionInitializesTheMaxTasksCorrectly() {
    TaskTracker taskTracker = new TaskTracker(1);
    assertTrue(taskTracker.canAddMoreTasks());
    taskTracker.addTask(task);
    assertFalse(taskTracker.canAddMoreTasks());
    TaskTracker taskTracker2 = new TaskTracker(taskTracker);
    assertFalse(taskTracker2.canAddMoreTasks());
}