Search in sources :

Example 6 with OptimizeSparkProcContext

use of org.apache.hadoop.hive.ql.parse.spark.OptimizeSparkProcContext in project hive by apache.

the class SparkJoinOptimizer method process.

@Override
public Object process(Node nd, Stack<Node> stack, NodeProcessorCtx procCtx, Object... nodeOutputs) throws SemanticException {
    OptimizeSparkProcContext context = (OptimizeSparkProcContext) procCtx;
    HiveConf conf = context.getConf();
    if (conf.getBoolVar(HiveConf.ConfVars.HIVE_AUTO_SORTMERGE_JOIN_TOMAPJOIN)) {
        Object mapJoinOp = mapJoinOptimizer.process(nd, stack, procCtx, nodeOutputs);
        if (mapJoinOp == null) {
            smbJoinOptimizer.process(nd, stack, procCtx, nodeOutputs);
        }
    } else {
        Object sortMergeJoinOp = smbJoinOptimizer.process(nd, stack, procCtx, nodeOutputs);
        if (sortMergeJoinOp == null) {
            mapJoinOptimizer.process(nd, stack, procCtx, nodeOutputs);
        }
    }
    return null;
}
Also used : OptimizeSparkProcContext(org.apache.hadoop.hive.ql.parse.spark.OptimizeSparkProcContext) HiveConf(org.apache.hadoop.hive.conf.HiveConf)

Aggregations

OptimizeSparkProcContext (org.apache.hadoop.hive.ql.parse.spark.OptimizeSparkProcContext)6 HiveConf (org.apache.hadoop.hive.conf.HiveConf)3 ArrayList (java.util.ArrayList)2 List (java.util.List)1 Set (java.util.Set)1 FileSinkOperator (org.apache.hadoop.hive.ql.exec.FileSinkOperator)1 JoinOperator (org.apache.hadoop.hive.ql.exec.JoinOperator)1 LimitOperator (org.apache.hadoop.hive.ql.exec.LimitOperator)1 MapJoinOperator (org.apache.hadoop.hive.ql.exec.MapJoinOperator)1 Operator (org.apache.hadoop.hive.ql.exec.Operator)1 ReduceSinkOperator (org.apache.hadoop.hive.ql.exec.ReduceSinkOperator)1 TableScanOperator (org.apache.hadoop.hive.ql.exec.TableScanOperator)1 Partition (org.apache.hadoop.hive.ql.metadata.Partition)1 Table (org.apache.hadoop.hive.ql.metadata.Table)1 BucketJoinProcCtx (org.apache.hadoop.hive.ql.optimizer.BucketJoinProcCtx)1 SortBucketJoinProcCtx (org.apache.hadoop.hive.ql.optimizer.SortBucketJoinProcCtx)1 SparkPartitionPruningSinkDesc (org.apache.hadoop.hive.ql.optimizer.spark.SparkPartitionPruningSinkDesc)1 OptimizeTezProcContext (org.apache.hadoop.hive.ql.parse.OptimizeTezProcContext)1 ParseContext (org.apache.hadoop.hive.ql.parse.ParseContext)1 PrunedPartitionList (org.apache.hadoop.hive.ql.parse.PrunedPartitionList)1