Search in sources :

Example 11 with InvalidRelException

use of org.apache.calcite.rel.InvalidRelException in project drill by apache.

the class UnionAllPrule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final DrillUnionRel union = (DrillUnionRel) call.rel(0);
    final List<RelNode> inputs = union.getInputs();
    List<RelNode> convertedInputList = Lists.newArrayList();
    PlannerSettings settings = PrelUtil.getPlannerSettings(call.getPlanner());
    boolean allHashDistributed = true;
    for (int i = 0; i < inputs.size(); i++) {
        RelNode child = inputs.get(i);
        List<DistributionField> childDistFields = Lists.newArrayList();
        RelNode convertedChild;
        for (RelDataTypeField f : child.getRowType().getFieldList()) {
            childDistFields.add(new DistributionField(f.getIndex()));
        }
        if (settings.isUnionAllDistributeEnabled()) {
            /*
         * Strictly speaking, union-all does not need re-distribution of data; but in Drill's execution
         * model, the data distribution and parallelism operators are the same. Here, we insert a
         * hash distribution operator to allow parallelism to be determined independently for the parent
         * and children. (See DRILL-4833).
         * Note that a round robin distribution would have sufficed but we don't have one.
         */
            DrillDistributionTrait hashChild = new DrillDistributionTrait(DrillDistributionTrait.DistributionType.HASH_DISTRIBUTED, ImmutableList.copyOf(childDistFields));
            RelTraitSet traitsChild = call.getPlanner().emptyTraitSet().plus(Prel.DRILL_PHYSICAL).plus(hashChild);
            convertedChild = convert(child, PrelUtil.fixTraits(call, traitsChild));
        } else {
            RelTraitSet traitsChild = call.getPlanner().emptyTraitSet().plus(Prel.DRILL_PHYSICAL);
            convertedChild = convert(child, PrelUtil.fixTraits(call, traitsChild));
            allHashDistributed = false;
        }
        convertedInputList.add(convertedChild);
    }
    try {
        RelTraitSet traits;
        if (allHashDistributed) {
            // since all children of union-all are hash distributed, propagate the traits of the left child
            traits = convertedInputList.get(0).getTraitSet();
        } else {
            // output distribution trait is set to ANY since union-all inputs may be distributed in different ways
            // and unlike a join there are no join keys that allow determining how the output would be distributed.
            // Note that a downstream operator may impose a required distribution which would be satisfied by
            // inserting an Exchange after the Union-All.
            traits = call.getPlanner().emptyTraitSet().plus(Prel.DRILL_PHYSICAL).plus(DrillDistributionTrait.ANY);
        }
        Preconditions.checkArgument(convertedInputList.size() >= 2, "Union list must be at least two items.");
        RelNode left = convertedInputList.get(0);
        for (int i = 1; i < convertedInputList.size(); i++) {
            left = new UnionAllPrel(union.getCluster(), traits, ImmutableList.of(left, convertedInputList.get(i)), false);
        }
        call.transformTo(left);
    } catch (InvalidRelException e) {
        tracer.warning(e.toString());
    }
}
Also used : InvalidRelException(org.apache.calcite.rel.InvalidRelException) RelTraitSet(org.apache.calcite.plan.RelTraitSet) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RelNode(org.apache.calcite.rel.RelNode) DrillUnionRel(org.apache.drill.exec.planner.logical.DrillUnionRel) DistributionField(org.apache.drill.exec.planner.physical.DrillDistributionTrait.DistributionField)

Example 12 with InvalidRelException

use of org.apache.calcite.rel.InvalidRelException in project drill by apache.

the class UnionDistinctPrule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final DrillUnionRel union = (DrillUnionRel) call.rel(0);
    final List<RelNode> inputs = union.getInputs();
    List<RelNode> convertedInputList = Lists.newArrayList();
    RelTraitSet traits = call.getPlanner().emptyTraitSet().plus(Prel.DRILL_PHYSICAL);
    try {
        for (int i = 0; i < inputs.size(); i++) {
            RelNode convertedInput = convert(inputs.get(i), PrelUtil.fixTraits(call, traits));
            convertedInputList.add(convertedInput);
        }
        traits = call.getPlanner().emptyTraitSet().plus(Prel.DRILL_PHYSICAL).plus(DrillDistributionTrait.SINGLETON);
        UnionDistinctPrel unionDistinct = new UnionDistinctPrel(union.getCluster(), traits, convertedInputList, false);
        call.transformTo(unionDistinct);
    } catch (InvalidRelException e) {
        tracer.warning(e.toString());
    }
}
Also used : InvalidRelException(org.apache.calcite.rel.InvalidRelException) RelNode(org.apache.calcite.rel.RelNode) DrillUnionRel(org.apache.drill.exec.planner.logical.DrillUnionRel) RelTraitSet(org.apache.calcite.plan.RelTraitSet)

Example 13 with InvalidRelException

use of org.apache.calcite.rel.InvalidRelException in project drill by apache.

the class NestedLoopJoinPrule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    PlannerSettings settings = PrelUtil.getPlannerSettings(call.getPlanner());
    if (!settings.isNestedLoopJoinEnabled()) {
        return;
    }
    final DrillJoinRel join = (DrillJoinRel) call.rel(0);
    final RelNode left = join.getLeft();
    final RelNode right = join.getRight();
    if (!checkPreconditions(join, left, right, settings)) {
        return;
    }
    try {
        if (checkBroadcastConditions(call.getPlanner(), join, left, right)) {
            createBroadcastPlan(call, join, join.getCondition(), PhysicalJoinType.NESTEDLOOP_JOIN, left, right, null, /* left collation */
            null);
        }
    } catch (InvalidRelException e) {
        tracer.warning(e.toString());
    }
}
Also used : InvalidRelException(org.apache.calcite.rel.InvalidRelException) DrillJoinRel(org.apache.drill.exec.planner.logical.DrillJoinRel) RelNode(org.apache.calcite.rel.RelNode)

Aggregations

InvalidRelException (org.apache.calcite.rel.InvalidRelException)13 RelNode (org.apache.calcite.rel.RelNode)11 RelTraitSet (org.apache.calcite.plan.RelTraitSet)8 DrillJoinRel (org.apache.drill.exec.planner.logical.DrillJoinRel)4 RelCollation (org.apache.calcite.rel.RelCollation)2 DrillAggregateRel (org.apache.drill.exec.planner.logical.DrillAggregateRel)2 DrillUnionRel (org.apache.drill.exec.planner.logical.DrillUnionRel)2 CalciteSemanticException (org.apache.hadoop.hive.ql.optimizer.calcite.CalciteSemanticException)2 HiveRulesRegistry (org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRulesRegistry)2 ArrayList (java.util.ArrayList)1 JoinInfo (org.apache.calcite.rel.core.JoinInfo)1 LogicalAggregate (org.apache.calcite.rel.logical.LogicalAggregate)1 LogicalJoin (org.apache.calcite.rel.logical.LogicalJoin)1 LogicalUnion (org.apache.calcite.rel.logical.LogicalUnion)1 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)1 RexNode (org.apache.calcite.rex.RexNode)1 DistributionField (org.apache.drill.exec.planner.physical.DrillDistributionTrait.DistributionField)1