Search in sources :

Example 91 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveProjectMergeRule method matches.

@Override
public boolean matches(RelOptRuleCall call) {
    // Currently we do not support merging windowing functions with other
    // windowing functions i.e. embedding windowing functions within each
    // other
    final Project topProject = call.rel(0);
    final Project bottomProject = call.rel(1);
    for (RexNode expr : topProject.getProjects()) {
        if (expr instanceof RexOver) {
            Set<Integer> positions = HiveCalciteUtil.getInputRefs(expr);
            for (int pos : positions) {
                if (bottomProject.getProjects().get(pos) instanceof RexOver) {
                    return false;
                }
            }
        }
    }
    return super.matches(call);
}
Also used : RexOver(org.apache.calcite.rex.RexOver) Project(org.apache.calcite.rel.core.Project) RexNode(org.apache.calcite.rex.RexNode)

Example 92 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveFilterProjectTSTransposeRule method onMatch.

// ~ Methods ----------------------------------------------------------------
// implement RelOptRule
public void onMatch(RelOptRuleCall call) {
    final Filter filter = call.rel(0);
    final Project project = call.rel(1);
    if (RexOver.containsOver(project.getProjects(), null)) {
        // it can be pushed down. For now we don't support this.
        return;
    }
    if (RexUtil.containsCorrelation(filter.getCondition())) {
        // Correlate from being de-correlated.
        return;
    }
    // convert the filter to one that references the child of the project
    RexNode newCondition = RelOptUtil.pushPastProject(filter.getCondition(), project);
    // Remove cast of BOOLEAN NOT NULL to BOOLEAN or vice versa. Filter accepts
    // nullable and not-nullable conditions, but a CAST might get in the way of
    // other rewrites.
    final RelDataTypeFactory typeFactory = filter.getCluster().getTypeFactory();
    if (RexUtil.isNullabilityCast(typeFactory, newCondition)) {
        newCondition = ((RexCall) newCondition).getOperands().get(0);
    }
    RelNode newFilterRel = filterFactory == null ? filter.copy(filter.getTraitSet(), project.getInput(), newCondition) : filterFactory.createFilter(project.getInput(), newCondition);
    RelNode newProjRel = projectFactory == null ? project.copy(project.getTraitSet(), newFilterRel, project.getProjects(), project.getRowType()) : projectFactory.createProject(newFilterRel, Collections.emptyList(), project.getProjects(), project.getRowType().getFieldNames());
    call.transformTo(newProjRel);
}
Also used : RexCall(org.apache.calcite.rex.RexCall) Project(org.apache.calcite.rel.core.Project) HiveProject(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject) RelNode(org.apache.calcite.rel.RelNode) Filter(org.apache.calcite.rel.core.Filter) RelDataTypeFactory(org.apache.calcite.rel.type.RelDataTypeFactory) RexNode(org.apache.calcite.rex.RexNode)

Example 93 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveCalciteUtil method getExprNodes.

public static List<ExprNodeDesc> getExprNodes(List<Integer> inputRefs, RelNode inputRel, String inputTabAlias) {
    List<ExprNodeDesc> exprNodes = new ArrayList<ExprNodeDesc>();
    List<RexNode> rexInputRefs = getInputRef(inputRefs, inputRel);
    List<RexNode> exprs = inputRel instanceof Project ? ((Project) inputRel).getProjects() : null;
    // TODO: Change ExprNodeConverter to be independent of Partition Expr
    ExprNodeConverter exprConv = new ExprNodeConverter(inputTabAlias, inputRel.getRowType(), new HashSet<Integer>(), inputRel.getCluster().getTypeFactory());
    for (int index = 0; index < rexInputRefs.size(); index++) {
        // check the corresponding expression in exprs to see if it is literal
        if (exprs != null && index < exprs.size() && exprs.get(inputRefs.get(index)) instanceof RexLiteral) {
            // because rexInputRefs represent ref expr corresponding to value in inputRefs it is used to get
            // corresponding index
            ExprNodeDesc exprNodeDesc = exprConv.visitLiteral((RexLiteral) exprs.get(inputRefs.get(index)));
            exprNodes.add(exprNodeDesc);
        } else {
            RexNode iRef = rexInputRefs.get(index);
            exprNodes.add(iRef.accept(exprConv));
        }
    }
    return exprNodes;
}
Also used : HiveProject(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject) Project(org.apache.calcite.rel.core.Project) RexLiteral(org.apache.calcite.rex.RexLiteral) ArrayList(java.util.ArrayList) ExprNodeDesc(org.apache.hadoop.hive.ql.plan.ExprNodeDesc) RexNode(org.apache.calcite.rex.RexNode) ExprNodeConverter(org.apache.hadoop.hive.ql.optimizer.calcite.translator.ExprNodeConverter)

Example 94 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class CorrelateProjectExtractor method findCorrelationDependentCalls.

/**
 * Traverses a plan and finds all simply correlated row expressions with the specified id.
 */
private static Set<RexNode> findCorrelationDependentCalls(CorrelationId corrId, RelNode plan) {
    SimpleCorrelationCollector finder = new SimpleCorrelationCollector(corrId);
    plan.accept(new RelHomogeneousShuttle() {

        @Override
        public RelNode visit(RelNode other) {
            if (other instanceof Project || other instanceof Filter) {
                other.accept(finder);
            }
            return super.visit(other);
        }
    });
    return finder.correlations;
}
Also used : Project(org.apache.calcite.rel.core.Project) RelNode(org.apache.calcite.rel.RelNode) Filter(org.apache.calcite.rel.core.Filter) RelHomogeneousShuttle(org.apache.calcite.rel.RelHomogeneousShuttle)

Example 95 with Project

use of org.apache.calcite.rel.core.Project in project hive by apache.

the class HiveJdbcConverter method generateSql.

public String generateSql() {
    SqlDialect dialect = getJdbcDialect();
    final HiveJdbcImplementor jdbcImplementor = new HiveJdbcImplementor(dialect, (JavaTypeFactory) getCluster().getTypeFactory());
    Project topProject;
    if (getInput() instanceof Project) {
        topProject = (Project) getInput();
    } else {
        // If it is not a project operator, we add it on top of the input
        // to force generating the column names instead of * while
        // translating to SQL
        RelNode nodeToTranslate = getInput();
        RexBuilder builder = getCluster().getRexBuilder();
        List<RexNode> projects = new ArrayList<>(nodeToTranslate.getRowType().getFieldList().size());
        for (int i = 0; i < nodeToTranslate.getRowType().getFieldCount(); i++) {
            projects.add(builder.makeInputRef(nodeToTranslate, i));
        }
        topProject = new JdbcProject(nodeToTranslate.getCluster(), nodeToTranslate.getTraitSet(), nodeToTranslate, projects, nodeToTranslate.getRowType());
    }
    final HiveJdbcImplementor.Result result = jdbcImplementor.visitRoot(topProject);
    return result.asStatement().toSqlString(dialect).getSql();
}
Also used : Project(org.apache.calcite.rel.core.Project) JdbcProject(org.apache.calcite.adapter.jdbc.JdbcRules.JdbcProject) HiveJdbcImplementor(org.apache.hadoop.hive.ql.optimizer.calcite.rules.jdbc.HiveJdbcImplementor) HiveRelNode(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode) RelNode(org.apache.calcite.rel.RelNode) JdbcProject(org.apache.calcite.adapter.jdbc.JdbcRules.JdbcProject) ArrayList(java.util.ArrayList) SqlDialect(org.apache.calcite.sql.SqlDialect) RexBuilder(org.apache.calcite.rex.RexBuilder) RexNode(org.apache.calcite.rex.RexNode)

Aggregations

Project (org.apache.calcite.rel.core.Project)143 RexNode (org.apache.calcite.rex.RexNode)77 RelNode (org.apache.calcite.rel.RelNode)71 ArrayList (java.util.ArrayList)46 LogicalProject (org.apache.calcite.rel.logical.LogicalProject)35 RexBuilder (org.apache.calcite.rex.RexBuilder)28 RelDataType (org.apache.calcite.rel.type.RelDataType)26 Aggregate (org.apache.calcite.rel.core.Aggregate)22 Filter (org.apache.calcite.rel.core.Filter)22 Join (org.apache.calcite.rel.core.Join)22 List (java.util.List)19 RexLiteral (org.apache.calcite.rex.RexLiteral)19 AggregateCall (org.apache.calcite.rel.core.AggregateCall)18 Sort (org.apache.calcite.rel.core.Sort)18 Test (org.junit.Test)18 RelBuilder (org.apache.calcite.tools.RelBuilder)17 ImmutableBitSet (org.apache.calcite.util.ImmutableBitSet)16 HiveProject (org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveProject)16 Collectors (java.util.stream.Collectors)15 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)15