Search in sources :

Example 1 with RelVisitor

use of org.apache.calcite.rel.RelVisitor in project calcite by apache.

the class VolcanoPlanner method provenance.

/**
 * Returns a multi-line string describing the provenance of a tree of
 * relational expressions. For each node in the tree, prints the rule that
 * created the node, if any. Recursively describes the provenance of the
 * relational expressions that are the arguments to that rule.
 *
 * <p>Thus, every relational expression and rule invocation that affected
 * the final outcome is described in the provenance. This can be useful
 * when finding the root cause of "mistakes" in a query plan.</p>
 *
 * @param root Root relational expression in a tree
 * @return Multi-line string describing the rules that created the tree
 */
private String provenance(RelNode root) {
    final StringWriter sw = new StringWriter();
    final PrintWriter pw = new PrintWriter(sw);
    final List<RelNode> nodes = new ArrayList<>();
    new RelVisitor() {

        public void visit(RelNode node, int ordinal, RelNode parent) {
            nodes.add(node);
            super.visit(node, ordinal, parent);
        }
    }.go(root);
    final Set<RelNode> visited = new HashSet<>();
    for (RelNode node : nodes) {
        provenanceRecurse(pw, node, 0, visited);
    }
    pw.flush();
    return sw.toString();
}
Also used : StringWriter(java.io.StringWriter) RelNode(org.apache.calcite.rel.RelNode) ArrayList(java.util.ArrayList) RelVisitor(org.apache.calcite.rel.RelVisitor) PrintWriter(java.io.PrintWriter) HashSet(java.util.HashSet)

Example 2 with RelVisitor

use of org.apache.calcite.rel.RelVisitor in project druid by druid-io.

the class RelParameterizerShuttle method bindRel.

private RelNode bindRel(RelNode node, RexBuilder builder, RelDataTypeFactory typeFactory) {
    final RexShuttle binder = new RexShuttle() {

        @Override
        public RexNode visitDynamicParam(RexDynamicParam dynamicParam) {
            return bind(dynamicParam, builder, typeFactory);
        }
    };
    node = node.accept(binder);
    node.childrenAccept(new RelVisitor() {

        @Override
        public void visit(RelNode node, int ordinal, RelNode parent) {
            super.visit(node, ordinal, parent);
            RelNode transformed = node.accept(binder);
            if (!node.equals(transformed)) {
                parent.replaceInput(ordinal, transformed);
            }
        }
    });
    return node;
}
Also used : RexShuttle(org.apache.calcite.rex.RexShuttle) RelNode(org.apache.calcite.rel.RelNode) RelVisitor(org.apache.calcite.rel.RelVisitor) RexDynamicParam(org.apache.calcite.rex.RexDynamicParam)

Example 3 with RelVisitor

use of org.apache.calcite.rel.RelVisitor in project hive by apache.

the class MaterializedViewRewritingRelVisitor method check.

private void check(Union union) {
    // We found the Union
    if (union.getInputs().size() != 2) {
        // Bail out
        throw new ReturnedValue(false);
    }
    // First branch should have the query (with write ID filter conditions)
    new RelVisitor() {

        @Override
        public void visit(RelNode node, int ordinal, RelNode parent) {
            if (node instanceof TableScan || node instanceof Filter || node instanceof Project || node instanceof Join) {
                // We can continue
                super.visit(node, ordinal, parent);
            } else if (node instanceof Aggregate && containsAggregate) {
                Aggregate aggregate = (Aggregate) node;
                for (int i = 0; i < aggregate.getAggCallList().size(); ++i) {
                    AggregateCall aggregateCall = aggregate.getAggCallList().get(i);
                    if (aggregateCall.getAggregation().getKind() == SqlKind.COUNT && aggregateCall.getArgList().size() == 0) {
                        countIndex = i + aggregate.getGroupCount();
                        break;
                    }
                }
                // We can continue
                super.visit(node, ordinal, parent);
            } else {
                throw new ReturnedValue(false);
            }
        }
    }.go(union.getInput(0));
    // Second branch should only have the MV
    new RelVisitor() {

        @Override
        public void visit(RelNode node, int ordinal, RelNode parent) {
            if (node instanceof TableScan) {
                // We can continue
                // TODO: Need to check that this is the same MV that we are rebuilding
                RelOptHiveTable hiveTable = (RelOptHiveTable) node.getTable();
                if (!hiveTable.getHiveTableMD().isMaterializedView()) {
                    // If it is not a materialized view, we do not rewrite it
                    throw new ReturnedValue(false);
                }
                if (containsAggregate && !AcidUtils.isFullAcidTable(hiveTable.getHiveTableMD())) {
                    // we do not rewrite it (we need MERGE support)
                    throw new ReturnedValue(false);
                }
            } else if (node instanceof Project) {
                // We can continue
                super.visit(node, ordinal, parent);
            } else {
                throw new ReturnedValue(false);
            }
        }
    }.go(union.getInput(1));
    // We pass all the checks, we can rewrite
    throw new ReturnedValue(true);
}
Also used : AggregateCall(org.apache.calcite.rel.core.AggregateCall) TableScan(org.apache.calcite.rel.core.TableScan) Project(org.apache.calcite.rel.core.Project) RelOptHiveTable(org.apache.hadoop.hive.ql.optimizer.calcite.RelOptHiveTable) RelNode(org.apache.calcite.rel.RelNode) Filter(org.apache.calcite.rel.core.Filter) Join(org.apache.calcite.rel.core.Join) RelVisitor(org.apache.calcite.rel.RelVisitor) Aggregate(org.apache.calcite.rel.core.Aggregate)

Example 4 with RelVisitor

use of org.apache.calcite.rel.RelVisitor in project hive by apache.

the class HiveMaterializedViewUtils method augmentMaterializationWithTimeInformation.

/**
 * Method to enrich the materialization query contained in the input with
 * its invalidation.
 */
public static HiveRelOptMaterialization augmentMaterializationWithTimeInformation(HiveRelOptMaterialization materialization, String validTxnsList, ValidTxnWriteIdList materializationTxnList) throws LockException {
    // Extract tables used by the query which will in turn be used to generate
    // the corresponding txn write ids
    List<String> tablesUsed = new ArrayList<>();
    new RelVisitor() {

        @Override
        public void visit(RelNode node, int ordinal, RelNode parent) {
            if (node instanceof TableScan) {
                TableScan ts = (TableScan) node;
                tablesUsed.add(((RelOptHiveTable) ts.getTable()).getHiveTableMD().getFullyQualifiedName());
            }
            super.visit(node, ordinal, parent);
        }
    }.go(materialization.queryRel);
    ValidTxnWriteIdList currentTxnList = SessionState.get().getTxnMgr().getValidWriteIds(tablesUsed, validTxnsList);
    // Augment
    final RexBuilder rexBuilder = materialization.queryRel.getCluster().getRexBuilder();
    final HepProgramBuilder augmentMaterializationProgram = new HepProgramBuilder().addRuleInstance(new HiveAugmentMaterializationRule(rexBuilder, currentTxnList, materializationTxnList));
    final HepPlanner augmentMaterializationPlanner = new HepPlanner(augmentMaterializationProgram.build());
    augmentMaterializationPlanner.setRoot(materialization.queryRel);
    final RelNode modifiedQueryRel = augmentMaterializationPlanner.findBestExp();
    return new HiveRelOptMaterialization(materialization.tableRel, modifiedQueryRel, null, materialization.qualifiedTableName, materialization.getScope(), materialization.getRebuildMode());
}
Also used : HiveTableScan(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveTableScan) TableScan(org.apache.calcite.rel.core.TableScan) ArrayList(java.util.ArrayList) HepProgramBuilder(org.apache.calcite.plan.hep.HepProgramBuilder) HepPlanner(org.apache.calcite.plan.hep.HepPlanner) HiveRelNode(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode) RelNode(org.apache.calcite.rel.RelNode) RexBuilder(org.apache.calcite.rex.RexBuilder) RelVisitor(org.apache.calcite.rel.RelVisitor) ValidTxnWriteIdList(org.apache.hadoop.hive.common.ValidTxnWriteIdList) HiveRelOptMaterialization(org.apache.hadoop.hive.ql.metadata.HiveRelOptMaterialization)

Example 5 with RelVisitor

use of org.apache.calcite.rel.RelVisitor in project hive by apache.

the class HiveJdbcConverter method getTableScan.

public JdbcHiveTableScan getTableScan() {
    final JdbcHiveTableScan[] tmpJdbcHiveTableScan = new JdbcHiveTableScan[1];
    new RelVisitor() {

        public void visit(RelNode node, int ordinal, RelNode parent) {
            if (node instanceof JdbcHiveTableScan && tmpJdbcHiveTableScan[0] == null) {
                tmpJdbcHiveTableScan[0] = (JdbcHiveTableScan) node;
            } else {
                super.visit(node, ordinal, parent);
            }
        }
    }.go(this);
    JdbcHiveTableScan jdbcHiveTableScan = tmpJdbcHiveTableScan[0];
    assert jdbcHiveTableScan != null;
    return jdbcHiveTableScan;
}
Also used : HiveRelNode(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode) RelNode(org.apache.calcite.rel.RelNode) RelVisitor(org.apache.calcite.rel.RelVisitor)

Aggregations

RelNode (org.apache.calcite.rel.RelNode)8 RelVisitor (org.apache.calcite.rel.RelVisitor)8 ArrayList (java.util.ArrayList)3 TableScan (org.apache.calcite.rel.core.TableScan)3 HashSet (java.util.HashSet)2 RelTraitSet (org.apache.calcite.plan.RelTraitSet)2 HiveRelNode (org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode)2 PrintWriter (java.io.PrintWriter)1 StringWriter (java.io.StringWriter)1 Set (java.util.Set)1 Nullable (javax.annotation.Nullable)1 DataContext (org.apache.calcite.DataContext)1 JavaTypeFactory (org.apache.calcite.adapter.java.JavaTypeFactory)1 CalciteSchema (org.apache.calcite.jdbc.CalciteSchema)1 LatticeEntry (org.apache.calcite.jdbc.CalciteSchema.LatticeEntry)1 RelOptCluster (org.apache.calcite.plan.RelOptCluster)1 RelOptLattice (org.apache.calcite.plan.RelOptLattice)1 RelOptMaterialization (org.apache.calcite.plan.RelOptMaterialization)1 RelOptPlanner (org.apache.calcite.plan.RelOptPlanner)1 RelOptTable (org.apache.calcite.plan.RelOptTable)1