Search in sources :

Example 86 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveJoinCommuteRule method onMatch.

public void onMatch(final RelOptRuleCall call) {
    Project topProject = call.rel(0);
    Join join = call.rel(1);
    // 1. We check if it is a permutation project. If it is
    // not, or this is the identity, the rule will do nothing
    final Permutation topPermutation = topProject.getPermutation();
    if (topPermutation == null) {
        return;
    }
    if (topPermutation.isIdentity()) {
        return;
    }
    // 2. We swap the join
    final RelNode swapped = JoinCommuteRule.swap(join, true);
    if (swapped == null) {
        return;
    }
    // bail out.
    if (swapped instanceof Join) {
        return;
    }
    // 4. We check if it is a permutation project. If it is
    // not, or this is the identity, the rule will do nothing
    final Project bottomProject = (Project) swapped;
    final Permutation bottomPermutation = bottomProject.getPermutation();
    if (bottomPermutation == null) {
        return;
    }
    if (bottomPermutation.isIdentity()) {
        return;
    }
    // 5. If the product of the topPermutation and bottomPermutation yields
    // the identity, then we can swap the join and remove the project on
    // top.
    final Permutation product = topPermutation.product(bottomPermutation);
    if (!product.isIdentity()) {
        return;
    }
    // 6. Return the new join as a replacement
    final Join swappedJoin = (Join) bottomProject.getInput(0);
    call.transformTo(swappedJoin);
}
Also used : Project(org.apache.calcite.rel.core.Project) HiveProject(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject) RelNode(org.apache.calcite.rel.RelNode) Permutation(org.apache.calcite.util.Permutation) HiveJoin(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveJoin) Join(org.apache.calcite.rel.core.Join)

Example 87 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveProjectJoinTransposeRule method matches.

@Override
public boolean matches(RelOptRuleCall call) {
    final RelNode leftInput = call.rel(2);
    final RelNode rightInput = call.rel(3);
    if (leftInput instanceof Project && rightInput instanceof Project) {
        return false;
    }
    return true;
}
Also used : Project(org.apache.calcite.rel.core.Project) RelNode(org.apache.calcite.rel.RelNode)

Example 88 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveProjectJoinTransposeRule method onMatch.

// ~ Methods ----------------------------------------------------------------
// implement RelOptRule
public void onMatch(RelOptRuleCall call) {
    Project origProj = call.rel(0);
    final Join join = call.rel(1);
    if (join.getJoinType() == JoinRelType.SEMI || join.getJoinType() == JoinRelType.ANTI) {
        // TODO: support SemiJoin
        return;
    }
    // locate all fields referenced in the projection and join condition;
    // determine which inputs are referenced in the projection and
    // join condition; if all fields are being referenced and there are no
    // special expressions, no point in proceeding any further
    PushProjector pushProject = new PushProjector(origProj, join.getCondition(), join, preserveExprCondition, call.builder());
    if (pushProject.locateAllRefs()) {
        return;
    }
    // create left and right projections, projecting only those
    // fields referenced on each side
    RelNode leftProjRel = pushProject.createProjectRefsAndExprs(join.getLeft(), true, false);
    RelNode rightProjRel = pushProject.createProjectRefsAndExprs(join.getRight(), true, true);
    // convert the join condition to reference the projected columns
    RexNode newJoinFilter = null;
    int[] adjustments = pushProject.getAdjustments();
    if (join.getCondition() != null) {
        List<RelDataTypeField> projJoinFieldList = new ArrayList<>();
        projJoinFieldList.addAll(join.getSystemFieldList());
        projJoinFieldList.addAll(leftProjRel.getRowType().getFieldList());
        projJoinFieldList.addAll(rightProjRel.getRowType().getFieldList());
        newJoinFilter = pushProject.convertRefsAndExprs(join.getCondition(), projJoinFieldList, adjustments);
    }
    // create a new join with the projected children
    Join newJoinRel = join.copy(join.getTraitSet(), newJoinFilter, leftProjRel, rightProjRel, join.getJoinType(), join.isSemiJoinDone());
    // put the original project on top of the join, converting it to
    // reference the modified projection list
    RelNode topProject = pushProject.createNewProject(newJoinRel, adjustments);
    call.transformTo(topProject);
}
Also used : Project(org.apache.calcite.rel.core.Project) PushProjector(org.apache.calcite.rel.rules.PushProjector) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RelNode(org.apache.calcite.rel.RelNode) ArrayList(java.util.ArrayList) Join(org.apache.calcite.rel.core.Join) RexNode(org.apache.calcite.rex.RexNode)

Example 89 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveProjectOverIntersectRemoveRule method matches.

// ~ Methods ----------------------------------------------------------------
@Override
public boolean matches(RelOptRuleCall call) {
    Project project = call.rel(0);
    Intersect intersect = call.rel(1);
    return isTrivial(project, intersect);
}
Also used : Project(org.apache.calcite.rel.core.Project) HiveProject(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject) HiveIntersect(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveIntersect) Intersect(org.apache.calcite.rel.core.Intersect)

Example 90 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveSemiJoinProjectTransposeRule method onMatch.

// ~ Methods ----------------------------------------------------------------
public void onMatch(RelOptRuleCall call) {
    Join semiJoin = call.rel(0);
    Project project = call.rel(1);
    // Convert the LHS semi-join keys to reference the child projection
    // expression; all projection expressions must be RexInputRefs,
    // otherwise, we wouldn't have created this semi-join.
    // convert the semijoin condition to reflect the LHS with the project
    // pulled up
    RexNode newCondition = adjustCondition(project, semiJoin);
    Join newSemiJoin = HiveSemiJoin.getSemiJoin(project.getCluster(), project.getTraitSet(), project.getInput(), semiJoin.getRight(), newCondition);
    // Create the new projection.  Note that the projection expressions
    // are the same as the original because they only reference the LHS
    // of the semijoin and the semijoin only projects out the LHS
    final RelBuilder relBuilder = call.builder();
    relBuilder.push(newSemiJoin);
    relBuilder.project(project.getProjects(), project.getRowType().getFieldNames());
    call.transformTo(relBuilder.build());
}
Also used : Project(org.apache.calcite.rel.core.Project) RelBuilder(org.apache.calcite.tools.RelBuilder) HiveSemiJoin(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSemiJoin) Join(org.apache.calcite.rel.core.Join) RexNode(org.apache.calcite.rex.RexNode)

Aggregations

Project (org.apache.calcite.rel.core.Project)143 RexNode (org.apache.calcite.rex.RexNode)77 RelNode (org.apache.calcite.rel.RelNode)71 ArrayList (java.util.ArrayList)46 LogicalProject (org.apache.calcite.rel.logical.LogicalProject)35 RexBuilder (org.apache.calcite.rex.RexBuilder)28 RelDataType (org.apache.calcite.rel.type.RelDataType)26 Aggregate (org.apache.calcite.rel.core.Aggregate)22 Filter (org.apache.calcite.rel.core.Filter)22 Join (org.apache.calcite.rel.core.Join)22 List (java.util.List)19 RexLiteral (org.apache.calcite.rex.RexLiteral)19 AggregateCall (org.apache.calcite.rel.core.AggregateCall)18 Sort (org.apache.calcite.rel.core.Sort)18 Test (org.junit.Test)18 RelBuilder (org.apache.calcite.tools.RelBuilder)17 ImmutableBitSet (org.apache.calcite.util.ImmutableBitSet)16 HiveProject (org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject)16 Collectors (java.util.stream.Collectors)15 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)15