Examples with MapTaskCompletionEventsUpdate org.apache.hadoop.mapred.MapTaskCompletionEventsUpdate used on opensource projects

Example 1 with MapTaskCompletionEventsUpdate

use of org.apache.hadoop.mapred.MapTaskCompletionEventsUpdate in project hadoop by apache.

the class EventFetcher method getMapCompletionEvents.

/** 
   * Queries the {@link TaskTracker} for a set of map-completion events 
   * from a given event ID.
   * @throws IOException
   */
protected int getMapCompletionEvents() throws IOException, InterruptedException {
    int numNewMaps = 0;
    TaskCompletionEvent[] events = null;
    do {
        MapTaskCompletionEventsUpdate update = umbilical.getMapCompletionEvents((org.apache.hadoop.mapred.JobID) reduce.getJobID(), fromEventIdx, maxEventsToFetch, (org.apache.hadoop.mapred.TaskAttemptID) reduce);
        events = update.getMapTaskCompletionEvents();
        LOG.debug("Got " + events.length + " map completion events from " + fromEventIdx);
        assert !update.shouldReset() : "Unexpected legacy state";
        // Update the last seen event ID
        fromEventIdx += events.length;
        //    outputs at all.
        for (TaskCompletionEvent event : events) {
            scheduler.resolve(event);
            if (TaskCompletionEvent.Status.SUCCEEDED == event.getTaskStatus()) {
                ++numNewMaps;
            }
        }
    } while (events.length == maxEventsToFetch);
    return numNewMaps;
}

Example 2 with MapTaskCompletionEventsUpdate

use of org.apache.hadoop.mapred.MapTaskCompletionEventsUpdate in project hadoop by apache.

the class TestEventFetcher method getMockedCompletionEventsUpdate.

private MapTaskCompletionEventsUpdate getMockedCompletionEventsUpdate(int startIdx, int numEvents) {
    ArrayList<TaskCompletionEvent> tceList = new ArrayList<TaskCompletionEvent>(numEvents);
    for (int i = 0; i < numEvents; ++i) {
        int eventIdx = startIdx + i;
        TaskCompletionEvent tce = new TaskCompletionEvent(eventIdx, new TaskAttemptID("12345", 1, TaskType.MAP, eventIdx, 0), eventIdx, true, TaskCompletionEvent.Status.SUCCEEDED, "http://somehost:8888");
        tceList.add(tce);
    }
    TaskCompletionEvent[] events = {};
    return new MapTaskCompletionEventsUpdate(tceList.toArray(events), false);
}