Examples with Shuffle org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle used on opensource projects

Example 1 with Shuffle

use of org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle in project tez by apache.

the class OrderedGroupedKVInput method waitForInputReady.

/**
 * Waits for the input to become ready for consumption
 * @throws IOException
 * @throws InterruptedException
 */
public void waitForInputReady() throws IOException, InterruptedException, TezException {
    // Cannot synchronize entire method since this is called form user code and can block.
    Shuffle localShuffleCopy = null;
    synchronized (this) {
        Preconditions.checkState(isStarted.get(), "Must start input before invoking this method");
        if (getNumPhysicalInputs() == 0) {
            return;
        }
        localShuffleCopy = shuffle;
    }
    TezRawKeyValueIterator localRawIter = localShuffleCopy.waitForInput();
    synchronized (this) {
        rawIter = localRawIter;
        createValuesIterator();
    }
}

Example 2 with Shuffle

use of org.apache.tez.runtime.library.common.shuffle.orderedgrouped.Shuffle in project tez by apache.

the class OrderedGroupedKVInput method handleEvents.

@Override
public void handleEvents(List<Event> inputEvents) throws IOException {
    Shuffle shuffleLocalRef;
    synchronized (this) {
        if (getNumPhysicalInputs() == 0) {
            throw new RuntimeException("No input events expected as numInputs is 0");
        }
        if (!isStarted.get()) {
            if (firstEventReceivedTime == -1) {
                firstEventReceivedTime = System.currentTimeMillis();
            }
            pendingEvents.addAll(inputEvents);
            return;
        }
        shuffleLocalRef = shuffle;
    }
    shuffleLocalRef.handleEvents(inputEvents);
}