Search in sources :

Example 1 with RelNode

use of org.apache.calcite.rel.RelNode in project hive by apache.

the class HiveAggregateProjectMergeRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final HiveAggregate aggregate = call.rel(0);
    final HiveProject project = call.rel(1);
    RelNode x = apply(aggregate, project);
    if (x != null) {
        call.transformTo(x);
    }
}
Also used : HiveAggregate(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveAggregate) RelNode(org.apache.calcite.rel.RelNode) HiveProject(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject)

Example 2 with RelNode

use of org.apache.calcite.rel.RelNode in project hive by apache.

the class HiveSemiJoinRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    LOG.debug("Matched HiveSemiJoinRule");
    final Project project = call.rel(0);
    final Join join = call.rel(1);
    final RelNode left = call.rel(2);
    final Aggregate aggregate = call.rel(3);
    final RelOptCluster cluster = join.getCluster();
    final RexBuilder rexBuilder = cluster.getRexBuilder();
    final ImmutableBitSet bits = RelOptUtil.InputFinder.bits(project.getProjects(), null);
    final ImmutableBitSet rightBits = ImmutableBitSet.range(left.getRowType().getFieldCount(), join.getRowType().getFieldCount());
    if (bits.intersects(rightBits)) {
        return;
    }
    final JoinInfo joinInfo = join.analyzeCondition();
    if (!joinInfo.rightSet().equals(ImmutableBitSet.range(aggregate.getGroupCount()))) {
        // By the way, neither a super-set nor a sub-set would work.
        return;
    }
    if (join.getJoinType() == JoinRelType.LEFT) {
        // since for LEFT join we are only interested in rows from LEFT we can get rid of right side
        call.transformTo(call.builder().push(left).project(project.getProjects(), project.getRowType().getFieldNames()).build());
        return;
    }
    if (join.getJoinType() != JoinRelType.INNER) {
        return;
    }
    if (!joinInfo.isEqui()) {
        return;
    }
    LOG.debug("All conditions matched for HiveSemiJoinRule. Going to apply transformation.");
    final List<Integer> newRightKeyBuilder = Lists.newArrayList();
    final List<Integer> aggregateKeys = aggregate.getGroupSet().asList();
    for (int key : joinInfo.rightKeys) {
        newRightKeyBuilder.add(aggregateKeys.get(key));
    }
    final ImmutableIntList newRightKeys = ImmutableIntList.copyOf(newRightKeyBuilder);
    final RelNode newRight = aggregate.getInput();
    final RexNode newCondition = RelOptUtil.createEquiJoinCondition(left, joinInfo.leftKeys, newRight, newRightKeys, rexBuilder);
    RelNode semi = null;
    // is not expected further down the pipeline. see jira for more details
    if (aggregate.getInput() instanceof HepRelVertex && ((HepRelVertex) aggregate.getInput()).getCurrentRel() instanceof Join) {
        Join rightJoin = (Join) (((HepRelVertex) aggregate.getInput()).getCurrentRel());
        List<RexNode> projects = new ArrayList<>();
        for (int i = 0; i < rightJoin.getRowType().getFieldCount(); i++) {
            projects.add(rexBuilder.makeInputRef(rightJoin, i));
        }
        RelNode topProject = call.builder().push(rightJoin).project(projects, rightJoin.getRowType().getFieldNames(), true).build();
        semi = call.builder().push(left).push(topProject).semiJoin(newCondition).build();
    } else {
        semi = call.builder().push(left).push(aggregate.getInput()).semiJoin(newCondition).build();
    }
    call.transformTo(call.builder().push(semi).project(project.getProjects(), project.getRowType().getFieldNames()).build());
}
Also used : RelOptCluster(org.apache.calcite.plan.RelOptCluster) ImmutableBitSet(org.apache.calcite.util.ImmutableBitSet) ArrayList(java.util.ArrayList) Join(org.apache.calcite.rel.core.Join) JoinInfo(org.apache.calcite.rel.core.JoinInfo) Project(org.apache.calcite.rel.core.Project) HepRelVertex(org.apache.calcite.plan.hep.HepRelVertex) RelNode(org.apache.calcite.rel.RelNode) RexBuilder(org.apache.calcite.rex.RexBuilder) ImmutableIntList(org.apache.calcite.util.ImmutableIntList) Aggregate(org.apache.calcite.rel.core.Aggregate) RexNode(org.apache.calcite.rex.RexNode)

Example 3 with RelNode

use of org.apache.calcite.rel.RelNode in project hive by apache.

the class HivePointLookupOptimizerRule method onMatch.

public void onMatch(RelOptRuleCall call) {
    final Filter filter = call.rel(0);
    final RexBuilder rexBuilder = filter.getCluster().getRexBuilder();
    final RexNode condition = RexUtil.pullFactors(rexBuilder, filter.getCondition());
    // 1. We try to transform possible candidates
    RexTransformIntoInClause transformIntoInClause = new RexTransformIntoInClause(rexBuilder, filter, minNumORClauses);
    RexNode newCondition = transformIntoInClause.apply(condition);
    // 2. We merge IN expressions
    RexMergeInClause mergeInClause = new RexMergeInClause(rexBuilder);
    newCondition = mergeInClause.apply(newCondition);
    // 3. If we could not transform anything, we bail out
    if (newCondition.toString().equals(condition.toString())) {
        return;
    }
    // 4. We create the filter with the new condition
    RelNode newFilter = filter.copy(filter.getTraitSet(), filter.getInput(), newCondition);
    call.transformTo(newFilter);
}
Also used : RelNode(org.apache.calcite.rel.RelNode) Filter(org.apache.calcite.rel.core.Filter) RexBuilder(org.apache.calcite.rex.RexBuilder) RexNode(org.apache.calcite.rex.RexNode)

Example 4 with RelNode

use of org.apache.calcite.rel.RelNode in project hive by apache.

the class HiveMaterializedViewFilterScanRule method apply.

protected void apply(RelOptRuleCall call, Project project, Filter filter, TableScan scan) {
    RelOptPlanner planner = call.getPlanner();
    List<RelOptMaterialization> materializations = (planner instanceof VolcanoPlanner) ? ((VolcanoPlanner) planner).getMaterializations() : ImmutableList.<RelOptMaterialization>of();
    if (!materializations.isEmpty()) {
        RelNode root = project.copy(project.getTraitSet(), Collections.singletonList(filter.copy(filter.getTraitSet(), Collections.singletonList((RelNode) scan))));
        // Costing is done in transformTo(), so we call it repeatedly with all applicable
        // materialized views and cheapest one will be picked
        List<RelOptMaterialization> applicableMaterializations = VolcanoPlanner.getApplicableMaterializations(root, materializations);
        for (RelOptMaterialization materialization : applicableMaterializations) {
            List<RelNode> subs = new MaterializedViewSubstitutionVisitor(materialization.queryRel, root, relBuilderFactory).go(materialization.tableRel);
            for (RelNode s : subs) {
                call.transformTo(s);
            }
        }
    }
}
Also used : RelNode(org.apache.calcite.rel.RelNode) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) VolcanoPlanner(org.apache.calcite.plan.volcano.VolcanoPlanner) RelOptPlanner(org.apache.calcite.plan.RelOptPlanner)

Example 5 with RelNode

use of org.apache.calcite.rel.RelNode in project hive by apache.

the class ASTConverter method convert.

public static ASTNode convert(final RelNode relNode, List<FieldSchema> resultSchema, boolean alignColumns) throws CalciteSemanticException {
    RelNode root = PlanModifierForASTConv.convertOpTree(relNode, resultSchema, alignColumns);
    ASTConverter c = new ASTConverter(root, 0);
    return c.convert();
}
Also used : RelNode(org.apache.calcite.rel.RelNode)

Aggregations

RelNode (org.apache.calcite.rel.RelNode)1133 RexNode (org.apache.calcite.rex.RexNode)353 ArrayList (java.util.ArrayList)269 Test (org.junit.Test)239 RelBuilder (org.apache.calcite.tools.RelBuilder)161 RelDataType (org.apache.calcite.rel.type.RelDataType)156 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)142 RexBuilder (org.apache.calcite.rex.RexBuilder)129 ImmutableBitSet (org.apache.calcite.util.ImmutableBitSet)121 RelTraitSet (org.apache.calcite.plan.RelTraitSet)114 RelMetadataQuery (org.apache.calcite.rel.metadata.RelMetadataQuery)97 RexInputRef (org.apache.calcite.rex.RexInputRef)85 HashMap (java.util.HashMap)82 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)77 Project (org.apache.calcite.rel.core.Project)74 RelOptCluster (org.apache.calcite.plan.RelOptCluster)71 List (java.util.List)67 Pair (org.apache.calcite.util.Pair)67 AggregateCall (org.apache.calcite.rel.core.AggregateCall)63 SqlNode (org.apache.calcite.sql.SqlNode)61