Search in sources :

Example 21 with OutputCommitter

use of org.apache.hadoop.mapreduce.OutputCommitter in project cdap by caskdata.

the class MultipleOutputsCommitter method abortTask.

@Override
public void abortTask(TaskAttemptContext taskContext) throws IOException {
    for (Map.Entry<String, OutputCommitter> committer : committers.entrySet()) {
        TaskAttemptContext namedTaskContext = MultipleOutputs.getNamedTaskContext(taskContext, committer.getKey());
        committer.getValue().abortTask(namedTaskContext);
    }
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) TaskAttemptContext(org.apache.hadoop.mapreduce.TaskAttemptContext) Map(java.util.Map)

Example 22 with OutputCommitter

use of org.apache.hadoop.mapreduce.OutputCommitter in project cdap by caskdata.

the class MultipleOutputsCommitter method recoverTask.

@Override
public void recoverTask(TaskAttemptContext taskContext) throws IOException {
    for (Map.Entry<String, OutputCommitter> committer : committers.entrySet()) {
        TaskAttemptContext namedTaskContext = MultipleOutputs.getNamedTaskContext(taskContext, committer.getKey());
        committer.getValue().recoverTask(namedTaskContext);
    }
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) TaskAttemptContext(org.apache.hadoop.mapreduce.TaskAttemptContext) Map(java.util.Map)

Example 23 with OutputCommitter

use of org.apache.hadoop.mapreduce.OutputCommitter in project cdap by caskdata.

the class MultipleOutputsCommitter method abortJob.

@Override
public void abortJob(JobContext jobContext, JobStatus.State state) throws IOException {
    for (Map.Entry<String, OutputCommitter> committer : committers.entrySet()) {
        JobContext namedJobContext = MultipleOutputs.getNamedJobContext(jobContext, committer.getKey());
        committer.getValue().abortJob(namedJobContext, state);
    }
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) JobContext(org.apache.hadoop.mapreduce.JobContext) Map(java.util.Map)

Example 24 with OutputCommitter

use of org.apache.hadoop.mapreduce.OutputCommitter in project cdap by caskdata.

the class MultipleOutputsCommitter method commitJob.

@Override
public void commitJob(JobContext jobContext) throws IOException {
    for (Map.Entry<String, OutputCommitter> committer : committers.entrySet()) {
        JobContext namedJobContext = MultipleOutputs.getNamedJobContext(jobContext, committer.getKey());
        committer.getValue().commitJob(namedJobContext);
    }
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) JobContext(org.apache.hadoop.mapreduce.JobContext) Map(java.util.Map)

Example 25 with OutputCommitter

use of org.apache.hadoop.mapreduce.OutputCommitter in project cdap by caskdata.

the class MultipleOutputsCommitter method setupTask.

@Override
public void setupTask(TaskAttemptContext taskContext) throws IOException {
    for (Map.Entry<String, OutputCommitter> committer : committers.entrySet()) {
        TaskAttemptContext namedTaskContext = MultipleOutputs.getNamedTaskContext(taskContext, committer.getKey());
        committer.getValue().setupTask(namedTaskContext);
    }
}
Also used : OutputCommitter(org.apache.hadoop.mapreduce.OutputCommitter) TaskAttemptContext(org.apache.hadoop.mapreduce.TaskAttemptContext) Map(java.util.Map)

Aggregations

OutputCommitter (org.apache.hadoop.mapreduce.OutputCommitter)47 Test (org.junit.Test)29 Configuration (org.apache.hadoop.conf.Configuration)23 TaskAttemptContext (org.apache.hadoop.mapreduce.TaskAttemptContext)18 JobContext (org.apache.hadoop.mapreduce.JobContext)13 CommitterEventHandler (org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler)13 JobEvent (org.apache.hadoop.mapreduce.v2.app.job.event.JobEvent)11 AsyncDispatcher (org.apache.hadoop.yarn.event.AsyncDispatcher)11 TaskAttemptID (org.apache.hadoop.mapreduce.TaskAttemptID)10 IOException (java.io.IOException)8 JobTaskEvent (org.apache.hadoop.mapreduce.v2.app.job.event.JobTaskEvent)8 HashMap (java.util.HashMap)7 JobId (org.apache.hadoop.mapreduce.v2.api.records.JobId)7 ArrayList (java.util.ArrayList)6 Map (java.util.Map)6 NullWritable (org.apache.hadoop.io.NullWritable)6 TaskId (org.apache.hadoop.mapreduce.v2.api.records.TaskId)6 AppContext (org.apache.hadoop.mapreduce.v2.app.AppContext)6 JobStartEvent (org.apache.hadoop.mapreduce.v2.app.job.event.JobStartEvent)6 TaskAttemptEvent (org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEvent)6