Search in sources :

Example 6 with TezEdgeProperty

use of org.apache.hadoop.hive.ql.plan.TezEdgeProperty in project hive by apache.

the class GenTezWork method getFollowingWorkIndex.

private int getFollowingWorkIndex(TezWork tezWork, UnionWork unionWork, ReduceSinkOperator rs) throws SemanticException {
    int index = 0;
    for (BaseWork baseWork : tezWork.getChildren(unionWork)) {
        TezEdgeProperty edgeProperty = tezWork.getEdgeProperty(unionWork, baseWork);
        if (edgeProperty.getEdgeType() != TezEdgeProperty.EdgeType.CONTAINS) {
            return index;
        }
        index++;
    }
    throw new SemanticException("Following work not found for the reduce sink: " + rs.getName());
}
Also used : TezEdgeProperty(org.apache.hadoop.hive.ql.plan.TezEdgeProperty) BaseWork(org.apache.hadoop.hive.ql.plan.BaseWork)

Example 7 with TezEdgeProperty

use of org.apache.hadoop.hive.ql.plan.TezEdgeProperty in project hive by apache.

the class GenTezWork method connectUnionWorkWithWork.

private void connectUnionWorkWithWork(UnionWork unionWork, BaseWork work, TezWork tezWork, GenTezProcContext context) {
    LOG.debug("Connecting union work (" + unionWork + ") with work (" + work + ")");
    TezEdgeProperty edgeProp = new TezEdgeProperty(EdgeType.CONTAINS);
    tezWork.connect(unionWork, work, edgeProp);
    unionWork.addUnionOperators(context.currentUnionOperators);
    context.workWithUnionOperators.add(work);
}
Also used : TezEdgeProperty(org.apache.hadoop.hive.ql.plan.TezEdgeProperty)

Aggregations

TezEdgeProperty (org.apache.hadoop.hive.ql.plan.TezEdgeProperty)7 BaseWork (org.apache.hadoop.hive.ql.plan.BaseWork)6 Operator (org.apache.hadoop.hive.ql.exec.Operator)4 TezWork (org.apache.hadoop.hive.ql.plan.TezWork)4 ArrayList (java.util.ArrayList)3 ReduceSinkOperator (org.apache.hadoop.hive.ql.exec.ReduceSinkOperator)3 OperatorDesc (org.apache.hadoop.hive.ql.plan.OperatorDesc)3 EdgeType (org.apache.hadoop.hive.ql.plan.TezEdgeProperty.EdgeType)3 LinkedHashMap (java.util.LinkedHashMap)2 LinkedList (java.util.LinkedList)2 List (java.util.List)2 FileSystem (org.apache.hadoop.fs.FileSystem)2 CommonMergeJoinOperator (org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator)2 DummyStoreOperator (org.apache.hadoop.hive.ql.exec.DummyStoreOperator)2 HashTableDummyOperator (org.apache.hadoop.hive.ql.exec.HashTableDummyOperator)2 MapJoinOperator (org.apache.hadoop.hive.ql.exec.MapJoinOperator)2 RowSchema (org.apache.hadoop.hive.ql.exec.RowSchema)2 MergeJoinWork (org.apache.hadoop.hive.ql.plan.MergeJoinWork)2 ReduceWork (org.apache.hadoop.hive.ql.plan.ReduceWork)2 UnionWork (org.apache.hadoop.hive.ql.plan.UnionWork)2