Search in sources :

Example 1 with RelOptMaterialization

use of org.apache.calcite.plan.RelOptMaterialization in project hive by apache.

the class HiveMaterializedViewFilterScanRule method apply.

protected void apply(RelOptRuleCall call, Project project, Filter filter, TableScan scan) {
    RelOptPlanner planner = call.getPlanner();
    List<RelOptMaterialization> materializations = (planner instanceof VolcanoPlanner) ? ((VolcanoPlanner) planner).getMaterializations() : ImmutableList.<RelOptMaterialization>of();
    if (!materializations.isEmpty()) {
        RelNode root = project.copy(project.getTraitSet(), Collections.singletonList(filter.copy(filter.getTraitSet(), Collections.singletonList((RelNode) scan))));
        // Costing is done in transformTo(), so we call it repeatedly with all applicable
        // materialized views and cheapest one will be picked
        List<RelOptMaterialization> applicableMaterializations = VolcanoPlanner.getApplicableMaterializations(root, materializations);
        for (RelOptMaterialization materialization : applicableMaterializations) {
            List<RelNode> subs = new MaterializedViewSubstitutionVisitor(materialization.queryRel, root, relBuilderFactory).go(materialization.tableRel);
            for (RelNode s : subs) {
                call.transformTo(s);
            }
        }
    }
}
Also used : RelNode(org.apache.calcite.rel.RelNode) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) VolcanoPlanner(org.apache.calcite.plan.volcano.VolcanoPlanner) RelOptPlanner(org.apache.calcite.plan.RelOptPlanner)

Example 2 with RelOptMaterialization

use of org.apache.calcite.plan.RelOptMaterialization in project hive by apache.

the class Hive method getRewritingMaterializedViews.

/**
   * Get the materialized views that have been enabled for rewriting from the
   * metastore. If the materialized view is in the cache, we do not need to
   * parse it to generate a logical plan for the rewriting. Instead, we
   * return the version present in the cache.
   *
   * @return the list of materialized views available for rewriting
   * @throws HiveException
   */
public List<RelOptMaterialization> getRewritingMaterializedViews() throws HiveException {
    try {
        // Final result
        List<RelOptMaterialization> result = new ArrayList<>();
        for (String dbName : getMSC().getAllDatabases()) {
            // From metastore (for security)
            List<String> tables = getMSC().getAllTables(dbName);
            // Cached views (includes all)
            Collection<RelOptMaterialization> cachedViews = HiveMaterializedViewsRegistry.get().getRewritingMaterializedViews(dbName);
            if (cachedViews.isEmpty()) {
                // Bail out: empty list
                continue;
            }
            Map<String, RelOptMaterialization> qualifiedNameToView = new HashMap<String, RelOptMaterialization>();
            for (RelOptMaterialization materialization : cachedViews) {
                qualifiedNameToView.put(materialization.table.getQualifiedName().get(0), materialization);
            }
            for (String table : tables) {
                // Compose qualified name
                String fullyQualifiedName = dbName;
                if (fullyQualifiedName != null && !fullyQualifiedName.isEmpty()) {
                    fullyQualifiedName = fullyQualifiedName + "." + table;
                } else {
                    fullyQualifiedName = table;
                }
                RelOptMaterialization materialization = qualifiedNameToView.get(fullyQualifiedName);
                if (materialization != null) {
                    // Add to final result set
                    result.add(materialization);
                }
            }
        }
        return result;
    } catch (Exception e) {
        throw new HiveException(e);
    }
}
Also used : LinkedHashMap(java.util.LinkedHashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) HashMap(java.util.HashMap) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) ArrayList(java.util.ArrayList) AlreadyExistsException(org.apache.hadoop.hive.metastore.api.AlreadyExistsException) InvalidOperationException(org.apache.hadoop.hive.metastore.api.InvalidOperationException) TException(org.apache.thrift.TException) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) SerDeException(org.apache.hadoop.hive.serde2.SerDeException) NoSuchObjectException(org.apache.hadoop.hive.metastore.api.NoSuchObjectException) MetaException(org.apache.hadoop.hive.metastore.api.MetaException) HiveMetaException(org.apache.hadoop.hive.metastore.HiveMetaException) FileNotFoundException(java.io.FileNotFoundException) JDODataStoreException(javax.jdo.JDODataStoreException)

Example 3 with RelOptMaterialization

use of org.apache.calcite.plan.RelOptMaterialization in project calcite by apache.

the class AbstractMaterializedViewRule method perform.

/**
 * Rewriting logic is based on "Optimizing Queries Using Materialized Views:
 * A Practical, Scalable Solution" by Goldstein and Larson.
 *
 * <p>On the query side, rules matches a Project-node chain or node, where node
 * is either an Aggregate or a Join. Subplan rooted at the node operator must
 * be composed of one or more of the following operators: TableScan, Project,
 * Filter, and Join.
 *
 * <p>For each join MV, we need to check the following:
 * <ol>
 * <li> The plan rooted at the Join operator in the view produces all rows
 * needed by the plan rooted at the Join operator in the query.</li>
 * <li> All columns required by compensating predicates, i.e., predicates that
 * need to be enforced over the view, are available at the view output.</li>
 * <li> All output expressions can be computed from the output of the view.</li>
 * <li> All output rows occur with the correct duplication factor. We might
 * rely on existing Unique-Key - Foreign-Key relationships to extract that
 * information.</li>
 * </ol>
 *
 * <p>In turn, for each aggregate MV, we need to check the following:
 * <ol>
 * <li> The plan rooted at the Aggregate operator in the view produces all rows
 * needed by the plan rooted at the Aggregate operator in the query.</li>
 * <li> All columns required by compensating predicates, i.e., predicates that
 * need to be enforced over the view, are available at the view output.</li>
 * <li> The grouping columns in the query are a subset of the grouping columns
 * in the view.</li>
 * <li> All columns required to perform further grouping are available in the
 * view output.</li>
 * <li> All columns required to compute output expressions are available in the
 * view output.</li>
 * </ol>
 *
 * <p>The rule contains multiple extensions compared to the original paper. One of
 * them is the possibility of creating rewritings using Union operators, e.g., if
 * the result of a query is partially contained in the materialized view.
 */
protected void perform(RelOptRuleCall call, Project topProject, RelNode node) {
    final RexBuilder rexBuilder = node.getCluster().getRexBuilder();
    final RelMetadataQuery mq = RelMetadataQuery.instance();
    final RelOptPlanner planner = call.getPlanner();
    final RexExecutor executor = Util.first(planner.getExecutor(), RexUtil.EXECUTOR);
    final RelOptPredicateList predicates = RelOptPredicateList.EMPTY;
    final RexSimplify simplify = new RexSimplify(rexBuilder, predicates, true, executor);
    final List<RelOptMaterialization> materializations = (planner instanceof VolcanoPlanner) ? ((VolcanoPlanner) planner).getMaterializations() : ImmutableList.<RelOptMaterialization>of();
    if (!materializations.isEmpty()) {
        // try to generate a rewriting are met
        if (!isValidPlan(topProject, node, mq)) {
            return;
        }
        // Obtain applicable (filtered) materializations
        // TODO: Filtering of relevant materializations needs to be
        // improved so we gather only materializations that might
        // actually generate a valid rewriting.
        final List<RelOptMaterialization> applicableMaterializations = RelOptMaterializations.getApplicableMaterializations(node, materializations);
        if (!applicableMaterializations.isEmpty()) {
            // 2. Initialize all query related auxiliary data structures
            // that will be used throughout query rewriting process
            // Generate query table references
            final Set<RelTableRef> queryTableRefs = mq.getTableReferences(node);
            if (queryTableRefs == null) {
                // Bail out
                return;
            }
            // Extract query predicates
            final RelOptPredicateList queryPredicateList = mq.getAllPredicates(node);
            if (queryPredicateList == null) {
                // Bail out
                return;
            }
            final RexNode pred = simplify.simplify(RexUtil.composeConjunction(rexBuilder, queryPredicateList.pulledUpPredicates, false));
            final Triple<RexNode, RexNode, RexNode> queryPreds = splitPredicates(rexBuilder, pred);
            // Extract query equivalence classes. An equivalence class is a set
            // of columns in the query output that are known to be equal.
            final EquivalenceClasses qEC = new EquivalenceClasses();
            for (RexNode conj : RelOptUtil.conjunctions(queryPreds.getLeft())) {
                assert conj.isA(SqlKind.EQUALS);
                RexCall equiCond = (RexCall) conj;
                qEC.addEquivalenceClass((RexTableInputRef) equiCond.getOperands().get(0), (RexTableInputRef) equiCond.getOperands().get(1));
            }
            // rewrite the given query
            for (RelOptMaterialization materialization : applicableMaterializations) {
                RelNode view = materialization.tableRel;
                Project topViewProject;
                RelNode viewNode;
                if (materialization.queryRel instanceof Project) {
                    topViewProject = (Project) materialization.queryRel;
                    viewNode = topViewProject.getInput();
                } else {
                    topViewProject = null;
                    viewNode = materialization.queryRel;
                }
                // 3.1. View checks before proceeding
                if (!isValidPlan(topViewProject, viewNode, mq)) {
                    // Skip it
                    continue;
                }
                // 3.2. Initialize all query related auxiliary data structures
                // that will be used throughout query rewriting process
                // Extract view predicates
                final RelOptPredicateList viewPredicateList = mq.getAllPredicates(viewNode);
                if (viewPredicateList == null) {
                    // Skip it
                    continue;
                }
                final RexNode viewPred = simplify.simplify(RexUtil.composeConjunction(rexBuilder, viewPredicateList.pulledUpPredicates, false));
                final Triple<RexNode, RexNode, RexNode> viewPreds = splitPredicates(rexBuilder, viewPred);
                // Extract view table references
                final Set<RelTableRef> viewTableRefs = mq.getTableReferences(viewNode);
                if (viewTableRefs == null) {
                    // Bail out
                    return;
                }
                // Extract view tables
                MatchModality matchModality;
                Multimap<RexTableInputRef, RexTableInputRef> compensationEquiColumns = ArrayListMultimap.create();
                if (!queryTableRefs.equals(viewTableRefs)) {
                    // subset of query tables (add additional tables through joins if possible)
                    if (viewTableRefs.containsAll(queryTableRefs)) {
                        matchModality = MatchModality.QUERY_PARTIAL;
                        final EquivalenceClasses vEC = new EquivalenceClasses();
                        for (RexNode conj : RelOptUtil.conjunctions(viewPreds.getLeft())) {
                            assert conj.isA(SqlKind.EQUALS);
                            RexCall equiCond = (RexCall) conj;
                            vEC.addEquivalenceClass((RexTableInputRef) equiCond.getOperands().get(0), (RexTableInputRef) equiCond.getOperands().get(1));
                        }
                        if (!compensatePartial(viewTableRefs, vEC, queryTableRefs, compensationEquiColumns)) {
                            // Cannot rewrite, skip it
                            continue;
                        }
                    } else if (queryTableRefs.containsAll(viewTableRefs)) {
                        matchModality = MatchModality.VIEW_PARTIAL;
                        ViewPartialRewriting partialRewritingResult = compensateViewPartial(call.builder(), rexBuilder, mq, view, topProject, node, queryTableRefs, qEC, topViewProject, viewNode, viewTableRefs);
                        if (partialRewritingResult == null) {
                            // Cannot rewrite, skip it
                            continue;
                        }
                        // Rewrite succeeded
                        view = partialRewritingResult.newView;
                        topViewProject = partialRewritingResult.newTopViewProject;
                        viewNode = partialRewritingResult.newViewNode;
                    } else {
                        // Skip it
                        continue;
                    }
                } else {
                    matchModality = MatchModality.COMPLETE;
                }
                // 4. We map every table in the query to a table with the same qualified
                // name (all query tables are contained in the view, thus this is equivalent
                // to mapping every table in the query to a view table).
                final Multimap<RelTableRef, RelTableRef> multiMapTables = ArrayListMultimap.create();
                for (RelTableRef queryTableRef1 : queryTableRefs) {
                    for (RelTableRef queryTableRef2 : queryTableRefs) {
                        if (queryTableRef1.getQualifiedName().equals(queryTableRef2.getQualifiedName())) {
                            multiMapTables.put(queryTableRef1, queryTableRef2);
                        }
                    }
                }
                // If a table is used multiple times, we will create multiple mappings,
                // and we will try to rewrite the query using each of the mappings.
                // Then, we will try to map every source table (query) to a target
                // table (view), and if we are successful, we will try to create
                // compensation predicates to filter the view results further
                // (if needed).
                final List<BiMap<RelTableRef, RelTableRef>> flatListMappings = generateTableMappings(multiMapTables);
                for (BiMap<RelTableRef, RelTableRef> queryToViewTableMapping : flatListMappings) {
                    // TableMapping : mapping query tables -> view tables
                    // 4.0. If compensation equivalence classes exist, we need to add
                    // the mapping to the query mapping
                    final EquivalenceClasses currQEC = EquivalenceClasses.copy(qEC);
                    if (matchModality == MatchModality.QUERY_PARTIAL) {
                        for (Entry<RexTableInputRef, RexTableInputRef> e : compensationEquiColumns.entries()) {
                            // Copy origin
                            RelTableRef queryTableRef = queryToViewTableMapping.inverse().get(e.getKey().getTableRef());
                            RexTableInputRef queryColumnRef = RexTableInputRef.of(queryTableRef, e.getKey().getIndex(), e.getKey().getType());
                            // Add to query equivalence classes and table mapping
                            currQEC.addEquivalenceClass(queryColumnRef, e.getValue());
                            queryToViewTableMapping.put(e.getValue().getTableRef(), // identity
                            e.getValue().getTableRef());
                        }
                    }
                    // 4.1. Compute compensation predicates, i.e., predicates that need to be
                    // enforced over the view to retain query semantics. The resulting predicates
                    // are expressed using {@link RexTableInputRef} over the query.
                    // First, to establish relationship, we swap column references of the view
                    // predicates to point to query tables and compute equivalence classes.
                    final RexNode viewColumnsEquiPred = RexUtil.swapTableReferences(rexBuilder, viewPreds.getLeft(), queryToViewTableMapping.inverse());
                    final EquivalenceClasses queryBasedVEC = new EquivalenceClasses();
                    for (RexNode conj : RelOptUtil.conjunctions(viewColumnsEquiPred)) {
                        assert conj.isA(SqlKind.EQUALS);
                        RexCall equiCond = (RexCall) conj;
                        queryBasedVEC.addEquivalenceClass((RexTableInputRef) equiCond.getOperands().get(0), (RexTableInputRef) equiCond.getOperands().get(1));
                    }
                    Triple<RexNode, RexNode, RexNode> compensationPreds = computeCompensationPredicates(rexBuilder, simplify, currQEC, queryPreds, queryBasedVEC, viewPreds, queryToViewTableMapping);
                    if (compensationPreds == null && generateUnionRewriting) {
                        // Attempt partial rewriting using union operator. This rewriting
                        // will read some data from the view and the rest of the data from
                        // the query computation. The resulting predicates are expressed
                        // using {@link RexTableInputRef} over the view.
                        compensationPreds = computeCompensationPredicates(rexBuilder, simplify, queryBasedVEC, viewPreds, currQEC, queryPreds, queryToViewTableMapping.inverse());
                        if (compensationPreds == null) {
                            // This was our last chance to use the view, skip it
                            continue;
                        }
                        RexNode compensationColumnsEquiPred = compensationPreds.getLeft();
                        RexNode otherCompensationPred = RexUtil.composeConjunction(rexBuilder, ImmutableList.of(compensationPreds.getMiddle(), compensationPreds.getRight()), false);
                        assert !compensationColumnsEquiPred.isAlwaysTrue() || !otherCompensationPred.isAlwaysTrue();
                        // b. Generate union branch (query).
                        final RelNode unionInputQuery = rewriteQuery(call.builder(), rexBuilder, simplify, mq, compensationColumnsEquiPred, otherCompensationPred, topProject, node, queryToViewTableMapping, queryBasedVEC, currQEC);
                        if (unionInputQuery == null) {
                            // Skip it
                            continue;
                        }
                        // c. Generate union branch (view).
                        // We trigger the unifying method. This method will either create a Project
                        // or an Aggregate operator on top of the view. It will also compute the
                        // output expressions for the query.
                        final RelNode unionInputView = rewriteView(call.builder(), rexBuilder, simplify, mq, matchModality, true, view, topProject, node, topViewProject, viewNode, queryToViewTableMapping, currQEC);
                        if (unionInputView == null) {
                            // Skip it
                            continue;
                        }
                        // d. Generate final rewriting (union).
                        final RelNode result = createUnion(call.builder(), rexBuilder, topProject, unionInputQuery, unionInputView);
                        if (result == null) {
                            // Skip it
                            continue;
                        }
                        call.transformTo(result);
                    } else if (compensationPreds != null) {
                        RexNode compensationColumnsEquiPred = compensationPreds.getLeft();
                        RexNode otherCompensationPred = RexUtil.composeConjunction(rexBuilder, ImmutableList.of(compensationPreds.getMiddle(), compensationPreds.getRight()), false);
                        // a. Compute final compensation predicate.
                        if (!compensationColumnsEquiPred.isAlwaysTrue() || !otherCompensationPred.isAlwaysTrue()) {
                            // All columns required by compensating predicates must be contained
                            // in the view output (condition 2).
                            List<RexNode> viewExprs = topViewProject == null ? extractReferences(rexBuilder, view) : topViewProject.getChildExps();
                            // since we want to enforce the rest
                            if (!compensationColumnsEquiPred.isAlwaysTrue()) {
                                compensationColumnsEquiPred = rewriteExpression(rexBuilder, mq, view, viewNode, viewExprs, queryToViewTableMapping.inverse(), queryBasedVEC, false, compensationColumnsEquiPred);
                                if (compensationColumnsEquiPred == null) {
                                    // Skip it
                                    continue;
                                }
                            }
                            // For the rest, we use the query equivalence classes
                            if (!otherCompensationPred.isAlwaysTrue()) {
                                otherCompensationPred = rewriteExpression(rexBuilder, mq, view, viewNode, viewExprs, queryToViewTableMapping.inverse(), currQEC, true, otherCompensationPred);
                                if (otherCompensationPred == null) {
                                    // Skip it
                                    continue;
                                }
                            }
                        }
                        final RexNode viewCompensationPred = RexUtil.composeConjunction(rexBuilder, ImmutableList.of(compensationColumnsEquiPred, otherCompensationPred), false);
                        // b. Generate final rewriting if possible.
                        // First, we add the compensation predicate (if any) on top of the view.
                        // Then, we trigger the unifying method. This method will either create a
                        // Project or an Aggregate operator on top of the view. It will also compute
                        // the output expressions for the query.
                        RelBuilder builder = call.builder();
                        RelNode viewWithFilter;
                        if (!viewCompensationPred.isAlwaysTrue()) {
                            RexNode newPred = simplify.simplify(viewCompensationPred);
                            viewWithFilter = builder.push(view).filter(newPred).build();
                            // We add (and push) the filter to the view plan before triggering the rewriting.
                            // This is useful in case some of the columns can be folded to same value after
                            // filter is added.
                            Pair<RelNode, RelNode> pushedNodes = pushFilterToOriginalViewPlan(builder, topViewProject, viewNode, newPred);
                            topViewProject = (Project) pushedNodes.left;
                            viewNode = pushedNodes.right;
                        } else {
                            viewWithFilter = builder.push(view).build();
                        }
                        final RelNode result = rewriteView(builder, rexBuilder, simplify, mq, matchModality, false, viewWithFilter, topProject, node, topViewProject, viewNode, queryToViewTableMapping, currQEC);
                        if (result == null) {
                            // Skip it
                            continue;
                        }
                        call.transformTo(result);
                    }
                // end else
                }
            }
        }
    }
}
Also used : RelMetadataQuery(org.apache.calcite.rel.metadata.RelMetadataQuery) RelOptPlanner(org.apache.calcite.plan.RelOptPlanner) RexCall(org.apache.calcite.rex.RexCall) RexExecutor(org.apache.calcite.rex.RexExecutor) RelOptPredicateList(org.apache.calcite.plan.RelOptPredicateList) RexBuilder(org.apache.calcite.rex.RexBuilder) RelOptPredicateList(org.apache.calcite.plan.RelOptPredicateList) List(java.util.List) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) Pair(org.apache.calcite.util.Pair) RelBuilder(org.apache.calcite.tools.RelBuilder) BiMap(com.google.common.collect.BiMap) HashBiMap(com.google.common.collect.HashBiMap) RelTableRef(org.apache.calcite.rex.RexTableInputRef.RelTableRef) RexTableInputRef(org.apache.calcite.rex.RexTableInputRef) Project(org.apache.calcite.rel.core.Project) RelNode(org.apache.calcite.rel.RelNode) RexSimplify(org.apache.calcite.rex.RexSimplify) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) VolcanoPlanner(org.apache.calcite.plan.volcano.VolcanoPlanner) RexNode(org.apache.calcite.rex.RexNode)

Example 4 with RelOptMaterialization

use of org.apache.calcite.plan.RelOptMaterialization in project calcite by apache.

the class VolcanoPlanner method registerMaterializations.

private void registerMaterializations() {
    // Avoid using materializations while populating materializations!
    final CalciteConnectionConfig config = context.unwrap(CalciteConnectionConfig.class);
    if (config == null || !config.materializationsEnabled()) {
        return;
    }
    // Register rels using materialized views.
    final List<Pair<RelNode, List<RelOptMaterialization>>> materializationUses = RelOptMaterializations.useMaterializedViews(originalRoot, materializations);
    for (Pair<RelNode, List<RelOptMaterialization>> use : materializationUses) {
        RelNode rel = use.left;
        Hook.SUB.run(rel);
        registerImpl(rel, root.set);
    }
    // Register table rels of materialized views that cannot find a substitution
    // in root rel transformation but can potentially be useful.
    final Set<RelOptMaterialization> applicableMaterializations = new HashSet<>(RelOptMaterializations.getApplicableMaterializations(originalRoot, materializations));
    for (Pair<RelNode, List<RelOptMaterialization>> use : materializationUses) {
        applicableMaterializations.removeAll(use.right);
    }
    for (RelOptMaterialization materialization : applicableMaterializations) {
        RelSubset subset = registerImpl(materialization.queryRel, null);
        RelNode tableRel2 = RelOptUtil.createCastRel(materialization.tableRel, materialization.queryRel.getRowType(), true);
        registerImpl(tableRel2, subset.set);
    }
    // Register rels using lattices.
    final List<Pair<RelNode, RelOptLattice>> latticeUses = RelOptMaterializations.useLattices(originalRoot, ImmutableList.copyOf(latticeByName.values()));
    if (!latticeUses.isEmpty()) {
        RelNode rel = latticeUses.get(0).left;
        Hook.SUB.run(rel);
        registerImpl(rel, root.set);
    }
}
Also used : RelNode(org.apache.calcite.rel.RelNode) CalciteConnectionConfig(org.apache.calcite.config.CalciteConnectionConfig) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) List(java.util.List) ArrayList(java.util.ArrayList) ImmutableList(com.google.common.collect.ImmutableList) Pair(org.apache.calcite.util.Pair) HashSet(java.util.HashSet)

Example 5 with RelOptMaterialization

use of org.apache.calcite.plan.RelOptMaterialization in project hive by apache.

the class HiveMaterializedViewsRegistry method addMaterializedView.

/**
   * Adds the materialized view to the cache.
   *
   * @param materializedViewTable the materialized view
   */
public RelOptMaterialization addMaterializedView(Table materializedViewTable) {
    // Bail out if it is not enabled for rewriting
    if (!materializedViewTable.isRewriteEnabled()) {
        return null;
    }
    ConcurrentMap<ViewKey, RelOptMaterialization> cq = new ConcurrentHashMap<ViewKey, RelOptMaterialization>();
    final ConcurrentMap<ViewKey, RelOptMaterialization> prevCq = materializedViews.putIfAbsent(materializedViewTable.getDbName(), cq);
    if (prevCq != null) {
        cq = prevCq;
    }
    // Bail out if it already exists
    final ViewKey vk = new ViewKey(materializedViewTable.getTableName(), materializedViewTable.getCreateTime());
    if (cq.containsKey(vk)) {
        return null;
    }
    // Add to cache
    final String viewQuery = materializedViewTable.getViewOriginalText();
    final RelNode tableRel = createTableScan(materializedViewTable);
    if (tableRel == null) {
        LOG.warn("Materialized view " + materializedViewTable.getCompleteName() + " ignored; error creating view replacement");
        return null;
    }
    final RelNode queryRel = parseQuery(viewQuery);
    if (queryRel == null) {
        LOG.warn("Materialized view " + materializedViewTable.getCompleteName() + " ignored; error parsing original query");
        return null;
    }
    RelOptMaterialization materialization = new RelOptMaterialization(tableRel, queryRel, null);
    cq.put(vk, materialization);
    if (LOG.isDebugEnabled()) {
        LOG.debug("Cached materialized view for rewriting: " + tableRel.getTable().getQualifiedName());
    }
    return materialization;
}
Also used : HiveRelNode(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode) RelNode(org.apache.calcite.rel.RelNode) RelOptMaterialization(org.apache.calcite.plan.RelOptMaterialization) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap)

Aggregations

RelOptMaterialization (org.apache.calcite.plan.RelOptMaterialization)11 RelNode (org.apache.calcite.rel.RelNode)8 ArrayList (java.util.ArrayList)5 RelOptPlanner (org.apache.calcite.plan.RelOptPlanner)5 ImmutableList (com.google.common.collect.ImmutableList)3 List (java.util.List)3 ConcurrentHashMap (java.util.concurrent.ConcurrentHashMap)3 VolcanoPlanner (org.apache.calcite.plan.volcano.VolcanoPlanner)3 FileNotFoundException (java.io.FileNotFoundException)2 IOException (java.io.IOException)2 ExecutionException (java.util.concurrent.ExecutionException)2 JDODataStoreException (javax.jdo.JDODataStoreException)2 RelOptLattice (org.apache.calcite.plan.RelOptLattice)2 RelTraitSet (org.apache.calcite.plan.RelTraitSet)2 Pair (org.apache.calcite.util.Pair)2 HiveMetaException (org.apache.hadoop.hive.metastore.HiveMetaException)2 AlreadyExistsException (org.apache.hadoop.hive.metastore.api.AlreadyExistsException)2 InvalidOperationException (org.apache.hadoop.hive.metastore.api.InvalidOperationException)2 MetaException (org.apache.hadoop.hive.metastore.api.MetaException)2 NoSuchObjectException (org.apache.hadoop.hive.metastore.api.NoSuchObjectException)2