Search in sources :

Example 11 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project druid by druid-io.

the class Expressions method toRowExtraction.

/**
   * Translate a Calcite row-expression to a Druid row extraction. Note that this signature will probably need to
   * change once we support extractions from multiple columns.
   *
   * @param plannerContext SQL planner context
   * @param rowOrder       order of fields in the Druid rows to be extracted from
   * @param expression     expression meant to be applied on top of the rows
   *
   * @return RowExtraction or null if not possible
   */
public static RowExtraction toRowExtraction(final DruidOperatorTable operatorTable, final PlannerContext plannerContext, final List<String> rowOrder, final RexNode expression) {
    if (expression.getKind() == SqlKind.INPUT_REF) {
        final RexInputRef ref = (RexInputRef) expression;
        final String columnName = rowOrder.get(ref.getIndex());
        if (columnName == null) {
            throw new ISE("WTF?! Expression referred to nonexistent index[%d]", ref.getIndex());
        }
        return RowExtraction.of(columnName, null);
    } else if (expression.getKind() == SqlKind.CAST) {
        final RexNode operand = ((RexCall) expression).getOperands().get(0);
        if (expression.getType().getSqlTypeName() == SqlTypeName.DATE && operand.getType().getSqlTypeName() == SqlTypeName.TIMESTAMP) {
            // Handling casting TIMESTAMP to DATE by flooring to DAY.
            return FloorExtractionOperator.applyTimestampFloor(toRowExtraction(operatorTable, plannerContext, rowOrder, operand), TimeUnits.toQueryGranularity(TimeUnitRange.DAY, plannerContext.getTimeZone()));
        } else {
            // TODO(gianm): Probably not a good idea to ignore other CASTs like this.
            return toRowExtraction(operatorTable, plannerContext, rowOrder, ((RexCall) expression).getOperands().get(0));
        }
    } else {
        // Try conversion using a SqlExtractionOperator.
        final RowExtraction retVal;
        if (expression instanceof RexCall) {
            final SqlExtractionOperator extractionOperator = operatorTable.lookupExtractionOperator(expression.getKind(), ((RexCall) expression).getOperator().getName());
            retVal = extractionOperator != null ? extractionOperator.convert(operatorTable, plannerContext, rowOrder, expression) : null;
        } else {
            retVal = null;
        }
        return retVal;
    }
}
Also used : RexCall(org.apache.calcite.rex.RexCall) RexInputRef(org.apache.calcite.rex.RexInputRef) ISE(io.druid.java.util.common.ISE) RexNode(org.apache.calcite.rex.RexNode)

Example 12 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class FlinkRelDecorrelator method decorrelateRel.

/**
	 * Rewrite Correlator into a left outer join.
	 *
	 * @param rel Correlator
	 */
public Frame decorrelateRel(LogicalCorrelate rel) {
    //
    // Rewrite logic:
    //
    // The original left input will be joined with the new right input that
    // has generated correlated variables propagated up. For any generated
    // cor vars that are not used in the join key, pass them along to be
    // joined later with the CorrelatorRels that produce them.
    //
    // the right input to Correlator should produce correlated variables
    final RelNode oldLeft = rel.getInput(0);
    final RelNode oldRight = rel.getInput(1);
    final Frame leftFrame = getInvoke(oldLeft, rel);
    final Frame rightFrame = getInvoke(oldRight, rel);
    if (leftFrame == null || rightFrame == null) {
        // If any input has not been rewritten, do not rewrite this rel.
        return null;
    }
    if (rightFrame.corVarOutputPos.isEmpty()) {
        return null;
    }
    assert rel.getRequiredColumns().cardinality() <= rightFrame.corVarOutputPos.keySet().size();
    // Change correlator rel into a join.
    // Join all the correlated variables produced by this correlator rel
    // with the values generated and propagated from the right input
    final SortedMap<Correlation, Integer> corVarOutputPos = new TreeMap<>(rightFrame.corVarOutputPos);
    final List<RexNode> conditions = new ArrayList<>();
    final List<RelDataTypeField> newLeftOutput = leftFrame.r.getRowType().getFieldList();
    int newLeftFieldCount = newLeftOutput.size();
    final List<RelDataTypeField> newRightOutput = rightFrame.r.getRowType().getFieldList();
    for (Map.Entry<Correlation, Integer> rightOutputPos : Lists.newArrayList(corVarOutputPos.entrySet())) {
        final Correlation corVar = rightOutputPos.getKey();
        if (!corVar.corr.equals(rel.getCorrelationId())) {
            continue;
        }
        final int newLeftPos = leftFrame.oldToNewOutputPos.get(corVar.field);
        final int newRightPos = rightOutputPos.getValue();
        conditions.add(rexBuilder.makeCall(SqlStdOperatorTable.EQUALS, RexInputRef.of(newLeftPos, newLeftOutput), new RexInputRef(newLeftFieldCount + newRightPos, newRightOutput.get(newRightPos).getType())));
        // remove this cor var from output position mapping
        corVarOutputPos.remove(corVar);
    }
    // vars that are not used in the join key.
    for (Correlation corVar : corVarOutputPos.keySet()) {
        int newPos = corVarOutputPos.get(corVar) + newLeftFieldCount;
        corVarOutputPos.put(corVar, newPos);
    }
    // then add any cor var from the left input. Do not need to change
    // output positions.
    corVarOutputPos.putAll(leftFrame.corVarOutputPos);
    // Create the mapping between the output of the old correlation rel
    // and the new join rel
    final Map<Integer, Integer> mapOldToNewOutputPos = Maps.newHashMap();
    int oldLeftFieldCount = oldLeft.getRowType().getFieldCount();
    int oldRightFieldCount = oldRight.getRowType().getFieldCount();
    assert rel.getRowType().getFieldCount() == oldLeftFieldCount + oldRightFieldCount;
    // Left input positions are not changed.
    mapOldToNewOutputPos.putAll(leftFrame.oldToNewOutputPos);
    // Right input positions are shifted by newLeftFieldCount.
    for (int i = 0; i < oldRightFieldCount; i++) {
        mapOldToNewOutputPos.put(i + oldLeftFieldCount, rightFrame.oldToNewOutputPos.get(i) + newLeftFieldCount);
    }
    final RexNode condition = RexUtil.composeConjunction(rexBuilder, conditions, false);
    RelNode newJoin = LogicalJoin.create(leftFrame.r, rightFrame.r, condition, ImmutableSet.<CorrelationId>of(), rel.getJoinType().toJoinType());
    return register(rel, newJoin, mapOldToNewOutputPos, corVarOutputPos);
}
Also used : ArrayList(java.util.ArrayList) TreeMap(java.util.TreeMap) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RelNode(org.apache.calcite.rel.RelNode) RexInputRef(org.apache.calcite.rex.RexInputRef) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) NavigableMap(java.util.NavigableMap) SortedMap(java.util.SortedMap) HashMap(java.util.HashMap) ImmutableSortedMap(com.google.common.collect.ImmutableSortedMap) TreeMap(java.util.TreeMap) RexNode(org.apache.calcite.rex.RexNode)

Example 13 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project hive by apache.

the class HiveRelDecorrelator method projectJoinOutputWithNullability.

/**
 * Pulls project above the join from its RHS input. Enforces nullability
 * for join output.
 *
 * @param join          Join
 * @param project       Original project as the right-hand input of the join
 * @param nullIndicatorPos Position of null indicator
 * @return the subtree with the new LogicalProject at the root
 */
private RelNode projectJoinOutputWithNullability(LogicalJoin join, LogicalProject project, int nullIndicatorPos) {
    final RelDataTypeFactory typeFactory = join.getCluster().getTypeFactory();
    final RelNode left = join.getLeft();
    final JoinRelType joinType = join.getJoinType();
    RexInputRef nullIndicator = new RexInputRef(nullIndicatorPos, typeFactory.createTypeWithNullability(join.getRowType().getFieldList().get(nullIndicatorPos).getType(), true));
    // now create the new project
    List<Pair<RexNode, String>> newProjExprs = Lists.newArrayList();
    // project everything from the LHS and then those from the original
    // projRel
    List<RelDataTypeField> leftInputFields = left.getRowType().getFieldList();
    for (int i = 0; i < leftInputFields.size(); i++) {
        newProjExprs.add(RexInputRef.of2(i, leftInputFields));
    }
    // Marked where the projected expr is coming from so that the types will
    // become nullable for the original projections which are now coming out
    // of the nullable side of the OJ.
    boolean projectPulledAboveLeftCorrelator = joinType.generatesNullsOnRight();
    for (Pair<RexNode, String> pair : project.getNamedProjects()) {
        RexNode newProjExpr = removeCorrelationExpr(pair.left, projectPulledAboveLeftCorrelator, nullIndicator);
        newProjExprs.add(Pair.of(newProjExpr, pair.right));
    }
    return RelOptUtil.createProject(join, newProjExprs, false);
}
Also used : JoinRelType(org.apache.calcite.rel.core.JoinRelType) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) HiveRelNode(org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveRelNode) RelNode(org.apache.calcite.rel.RelNode) RelDataTypeFactory(org.apache.calcite.rel.type.RelDataTypeFactory) RexInputRef(org.apache.calcite.rex.RexInputRef) Pair(org.apache.calcite.util.Pair) RexNode(org.apache.calcite.rex.RexNode)

Example 14 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project hive by apache.

the class HiveSubQRemoveRelBuilder method field.

/**
 * As {@link #field(int, int, int)}, but if {@code alias} is true, the method
 * may apply an alias to make sure that the field has the same name as in the
 * input frame. If no alias is applied the expression is definitely a
 * {@link RexInputRef}.
 */
private RexNode field(int inputCount, int inputOrdinal, int fieldOrdinal, boolean alias) {
    final Frame frame = peek_(inputCount, inputOrdinal);
    final RelNode input = frame.rel;
    final RelDataType rowType = input.getRowType();
    if (fieldOrdinal < 0 || fieldOrdinal > rowType.getFieldCount()) {
        throw new IllegalArgumentException("field ordinal [" + fieldOrdinal + "] out of range; input fields are: " + rowType.getFieldNames());
    }
    final RelDataTypeField field = rowType.getFieldList().get(fieldOrdinal);
    final int offset = inputOffset(inputCount, inputOrdinal);
    final RexInputRef ref = cluster.getRexBuilder().makeInputRef(field.getType(), offset + fieldOrdinal);
    final RelDataTypeField aliasField = frame.fields().get(fieldOrdinal);
    if (!alias || field.getName().equals(aliasField.getName())) {
        return ref;
    } else {
        return alias(ref, aliasField.getName());
    }
}
Also used : RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RelNode(org.apache.calcite.rel.RelNode) RexInputRef(org.apache.calcite.rex.RexInputRef) RelDataType(org.apache.calcite.rel.type.RelDataType)

Example 15 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project herddb by diennea.

the class CalcitePlanner method planBindableTableScan.

private PlannerOp planBindableTableScan(BindableTableScan scan, RelDataType rowType) {
    if (rowType == null) {
        rowType = scan.getRowType();
    }
    final String tableSpace = scan.getTable().getQualifiedName().get(0);
    final TableImpl tableImpl = (TableImpl) scan.getTable().unwrap(org.apache.calcite.schema.Table.class);
    Table table = tableImpl.tableManager.getTable();
    SQLRecordPredicate predicate = null;
    if (!scan.filters.isEmpty()) {
        CompiledSQLExpression where = null;
        if (scan.filters.size() == 1) {
            RexNode expr = scan.filters.get(0);
            where = SQLExpressionCompiler.compileExpression(expr);
        } else {
            CompiledSQLExpression[] operands = new CompiledSQLExpression[scan.filters.size()];
            int i = 0;
            for (RexNode expr : scan.filters) {
                CompiledSQLExpression condition = SQLExpressionCompiler.compileExpression(expr);
                operands[i++] = condition;
            }
            where = new CompiledMultiAndExpression(operands);
        }
        predicate = new SQLRecordPredicate(table, null, where);
        TableSpaceManager tableSpaceManager = manager.getTableSpaceManager(tableSpace);
        IndexOperation op = scanForIndexAccess(where, table, tableSpaceManager);
        predicate.setIndexOperation(op);
        CompiledSQLExpression filterPk = findFiltersOnPrimaryKey(table, where);
        if (filterPk != null) {
            filterPk = remapPositionalAccessToToPrimaryKeyAccessor(filterPk, table, scan);
        }
        predicate.setPrimaryKeyFilter(filterPk);
    }
    List<RexNode> projections = new ArrayList<>(scan.projects.size());
    int i = 0;
    for (int fieldpos : scan.projects) {
        projections.add(new RexInputRef(fieldpos, rowType.getFieldList().get(i++).getType()));
    }
    Projection projection = buildProjection(projections, rowType, true, table.columns);
    ScanStatement scanStatement = new ScanStatement(tableSpace, table.name, projection, predicate, null, null);
    scanStatement.setTableDef(table);
    return new BindableTableScanOp(scanStatement);
}
Also used : CompiledMultiAndExpression(herddb.sql.expressions.CompiledMultiAndExpression) Table(herddb.model.Table) RelOptTable(org.apache.calcite.plan.RelOptTable) ProjectableFilterableTable(org.apache.calcite.schema.ProjectableFilterableTable) ScannableTable(org.apache.calcite.schema.ScannableTable) AbstractTable(org.apache.calcite.schema.impl.AbstractTable) ModifiableTable(org.apache.calcite.schema.ModifiableTable) ArrayList(java.util.ArrayList) Projection(herddb.model.Projection) CompiledSQLExpression(herddb.sql.expressions.CompiledSQLExpression) IndexOperation(herddb.index.IndexOperation) TableSpaceManager(herddb.core.TableSpaceManager) RexInputRef(org.apache.calcite.rex.RexInputRef) RexNode(org.apache.calcite.rex.RexNode) ScanStatement(herddb.model.commands.ScanStatement) BindableTableScanOp(herddb.model.planner.BindableTableScanOp)

Aggregations

RexInputRef (org.apache.calcite.rex.RexInputRef)241 RexNode (org.apache.calcite.rex.RexNode)200 ArrayList (java.util.ArrayList)103 RelNode (org.apache.calcite.rel.RelNode)85 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)80 RexCall (org.apache.calcite.rex.RexCall)67 RelDataType (org.apache.calcite.rel.type.RelDataType)63 RexBuilder (org.apache.calcite.rex.RexBuilder)54 ImmutableBitSet (org.apache.calcite.util.ImmutableBitSet)52 HashMap (java.util.HashMap)47 AggregateCall (org.apache.calcite.rel.core.AggregateCall)36 List (java.util.List)35 HashSet (java.util.HashSet)32 Pair (org.apache.calcite.util.Pair)32 RexLiteral (org.apache.calcite.rex.RexLiteral)29 Map (java.util.Map)24 RelOptUtil (org.apache.calcite.plan.RelOptUtil)24 Set (java.util.Set)20 ImmutableList (com.google.common.collect.ImmutableList)19 LinkedHashMap (java.util.LinkedHashMap)19