Examples with Unstable org.apache.hadoop.classification.InterfaceStability.Unstable used on opensource projects

Example 6 with Unstable

use of org.apache.hadoop.classification.InterfaceStability.Unstable in project tez by apache.

the class TezClient method preWarm.

/**
 * API to help pre-allocate containers in session mode. In non-session mode
 * this is ignored. The pre-allocated containers may be re-used by subsequent
 * job DAGs to improve performance.
 * The preWarm vertex should be configured and setup exactly
 * like the other vertices in the job DAGs so that the pre-allocated
 * containers may be re-used by the subsequent DAGs to improve performance.
 * The processor for the preWarmVertex may be used to pre-warm the containers
 * by pre-loading classes etc. It should be short-running so that pre-warming
 * does not block real execution. Users can specify their custom processors or
 * use the PreWarmProcessor from the runtime library.
 * The parallelism of the preWarmVertex will determine the number of preWarmed
 * containers.
 * Pre-warming is best efforts and among other factors is limited by the free
 * resources on the cluster. Based on the specified timeout value it returns
 * false if the status is not READY after the wait period.
 * @param preWarmVertex
 * @param timeout
 * @param unit
 * @throws TezException
 * @throws IOException
 */
@Unstable
public synchronized void preWarm(PreWarmVertex preWarmVertex, long timeout, TimeUnit unit) throws TezException, IOException {
    if (!isSession) {
        // do nothing for non session mode. This is there to let the code
        // work correctly in both modes
        LOG.warn("preWarm is not supported in non-session mode," + "please use session-mode of TezClient");
        return;
    }
    verifySessionStateForSubmission();
    DAG dag = org.apache.tez.dag.api.DAG.create(TezConstants.TEZ_PREWARM_DAG_NAME_PREFIX + "_" + preWarmDAGCounter++);
    dag.addVertex(preWarmVertex);
    boolean isReady;
    try {
        isReady = waitTillReady(timeout, unit);
    } catch (InterruptedException e) {
        throw new IOException("Interrupted while waiting for AM to become " + "available", e);
    }
    if (isReady) {
        submitDAG(dag);
    } else {
        throw new SessionNotReady("Tez AM not ready, could not submit DAG");
    }
}