Search in sources :

Example 11 with IOInterruptedException

use of org.apache.tez.runtime.library.api.IOInterruptedException in project tez by apache.

the class MROutput method getWriter.

/**
 * Get a key value write to write Map Reduce compatible output
 */
@Override
public KeyValueWriter getWriter() throws IOException {
    return new KeyValueWriter() {

        private final boolean useNewWriter = useNewApi;

        @SuppressWarnings("unchecked")
        @Override
        public void write(Object key, Object value) throws IOException {
            if (useNewWriter) {
                try {
                    newRecordWriter.write(key, value);
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    throw new IOInterruptedException("Interrupted while writing next key-value", e);
                }
            } else {
                oldRecordWriter.write(key, value);
            }
            outputRecordCounter.increment(1);
            getContext().notifyProgress();
        }
    };
}
Also used : KeyValueWriter(org.apache.tez.runtime.library.api.KeyValueWriter) IOInterruptedException(org.apache.tez.runtime.library.api.IOInterruptedException) IOInterruptedException(org.apache.tez.runtime.library.api.IOInterruptedException)

Example 12 with IOInterruptedException

use of org.apache.tez.runtime.library.api.IOInterruptedException in project tez by apache.

the class MRReaderMapReduce method next.

@Override
public boolean next() throws IOException {
    boolean hasNext;
    try {
        hasNext = recordReader.nextKeyValue();
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
        throw new IOInterruptedException("Interrupted while checking for next key-value", e);
    }
    if (hasNext) {
        inputRecordCounter.increment(1);
        notifyProgress();
    } else {
        hasCompletedProcessing();
        completedProcessing = true;
        notifyDone();
    }
    return hasNext;
}
Also used : IOInterruptedException(org.apache.tez.runtime.library.api.IOInterruptedException) IOInterruptedException(org.apache.tez.runtime.library.api.IOInterruptedException)

Aggregations

IOInterruptedException (org.apache.tez.runtime.library.api.IOInterruptedException)12 FSDataOutputStream (org.apache.hadoop.fs.FSDataOutputStream)2 Path (org.apache.hadoop.fs.Path)2 InputContext (org.apache.tez.runtime.api.InputContext)2 Writer (org.apache.tez.runtime.library.common.sort.impl.IFile.Writer)2 Test (org.junit.Test)2 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Text (org.apache.hadoop.io.Text)1 TezCounter (org.apache.tez.common.counters.TezCounter)1 TezCounters (org.apache.tez.common.counters.TezCounters)1 Event (org.apache.tez.runtime.api.Event)1 KeyValueWriter (org.apache.tez.runtime.library.api.KeyValueWriter)1 KeyValueWriterWithBasePath (org.apache.tez.runtime.library.api.KeyValueWriterWithBasePath)1 KeyValuesReader (org.apache.tez.runtime.library.api.KeyValuesReader)1 ValuesIterator (org.apache.tez.runtime.library.common.ValuesIterator)1 ShuffleManager (org.apache.tez.runtime.library.common.shuffle.impl.ShuffleManager)1 DiskSegment (org.apache.tez.runtime.library.common.sort.impl.TezMerger.DiskSegment)1 Segment (org.apache.tez.runtime.library.common.sort.impl.TezMerger.Segment)1 TezRawKeyValueIterator (org.apache.tez.runtime.library.common.sort.impl.TezRawKeyValueIterator)1