Search in sources :

Example 41 with DAG

use of org.apache.tez.dag.app.dag.DAG in project tez by apache.

the class DAGClientHandler method getDAG.

DAG getDAG(String dagIdStr) throws TezException {
    TezDAGID dagId;
    try {
        dagId = TezDAGID.fromString(dagIdStr);
    } catch (IllegalArgumentException e) {
        throw new TezException("Bad dagId: " + dagIdStr, e);
    }
    DAG currentDAG = getCurrentDAG();
    if (currentDAG == null) {
        throw new TezException("No running dag at present");
    }
    final String currentDAGIdStr = currentDAG.getID().toString();
    if (!currentDAGIdStr.equals(dagIdStr)) {
        if (getAllDagIDs().contains(dagIdStr)) {
            if (LOG.isDebugEnabled()) {
                LOG.debug("Looking for finished dagId " + dagIdStr + " current dag is " + currentDAGIdStr);
            }
            throw new DAGNotRunningException("DAG " + dagIdStr + " Not running, current dag is " + currentDAGIdStr);
        } else {
            LOG.warn("Current DAGID : " + currentDAGIdStr + ", Looking for string (not found): " + dagIdStr + ", dagIdObj: " + dagId);
            throw new TezException("Unknown dagId: " + dagIdStr);
        }
    }
    return currentDAG;
}
Also used : TezException(org.apache.tez.dag.api.TezException) DAGNotRunningException(org.apache.tez.dag.api.DAGNotRunningException) TezDAGID(org.apache.tez.dag.records.TezDAGID) DAG(org.apache.tez.dag.app.dag.DAG)

Example 42 with DAG

use of org.apache.tez.dag.app.dag.DAG in project tez by apache.

the class TaskCommunicatorContextImpl method getInputVertexNames.

@Override
public Iterable<String> getInputVertexNames(String vertexName) {
    Preconditions.checkNotNull(vertexName, "VertexName cannot be null: " + vertexName);
    DAG dag = getDag();
    Vertex vertex = dag.getVertex(vertexName);
    Set<Vertex> sources = vertex.getInputVertices().keySet();
    return Iterables.transform(sources, new Function<Vertex, String>() {

        @Override
        public String apply(Vertex input) {
            return input.getName();
        }
    });
}
Also used : Vertex(org.apache.tez.dag.app.dag.Vertex) DAG(org.apache.tez.dag.app.dag.DAG)

Example 43 with DAG

use of org.apache.tez.dag.app.dag.DAG in project tez by apache.

the class TaskCommunicatorContextImpl method getVertexRunningTaskCount.

@Override
public int getVertexRunningTaskCount(String vertexName) {
    Preconditions.checkArgument(vertexName != null, "VertexName must be specified");
    DAG dag = getDag();
    Vertex vertex = dag.getVertex(vertexName);
    return vertex.getRunningTasks();
}
Also used : Vertex(org.apache.tez.dag.app.dag.Vertex) DAG(org.apache.tez.dag.app.dag.DAG)

Example 44 with DAG

use of org.apache.tez.dag.app.dag.DAG in project tez by apache.

the class TaskCommunicatorContextImpl method getVertexTotalTaskCount.

@Override
public int getVertexTotalTaskCount(String vertexName) {
    Preconditions.checkArgument(vertexName != null, "VertexName must be specified");
    DAG dag = getDag();
    Vertex vertex = dag.getVertex(vertexName);
    return vertex.getTotalTasks();
}
Also used : Vertex(org.apache.tez.dag.app.dag.Vertex) DAG(org.apache.tez.dag.app.dag.DAG)

Example 45 with DAG

use of org.apache.tez.dag.app.dag.DAG in project tez by apache.

the class TaskCommunicatorContextImpl method registerForVertexStateUpdates.

@Override
public void registerForVertexStateUpdates(String vertexName, @Nullable Set<VertexState> stateSet) {
    Preconditions.checkNotNull(vertexName, "VertexName cannot be null: " + vertexName);
    DAG dag = getDag();
    dag.getStateChangeNotifier().registerForVertexUpdates(vertexName, stateSet, this);
}
Also used : DAG(org.apache.tez.dag.app.dag.DAG)

Aggregations

DAG (org.apache.tez.dag.app.dag.DAG)46 Vertex (org.apache.tez.dag.app.dag.Vertex)22 Test (org.junit.Test)16 HashMap (java.util.HashMap)11 TezVertexID (org.apache.tez.dag.records.TezVertexID)11 EventHandler (org.apache.hadoop.yarn.event.EventHandler)10 Set (java.util.Set)9 TreeMap (java.util.TreeMap)8 Event (org.apache.hadoop.yarn.event.Event)8 AppContext (org.apache.tez.dag.app.AppContext)8 TezDAGID (org.apache.tez.dag.records.TezDAGID)7 Configuration (org.apache.hadoop.conf.Configuration)6 Matchers.anyString (org.mockito.Matchers.anyString)6 ImmutableMap (com.google.common.collect.ImmutableMap)5 Map (java.util.Map)5 ContainerId (org.apache.hadoop.yarn.api.records.ContainerId)5 TaskAttempt (org.apache.tez.dag.app.dag.TaskAttempt)5 IOException (java.io.IOException)4 ArrayList (java.util.ArrayList)4 LinkedList (java.util.LinkedList)4