Search in sources :

Example 86 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class WatermarkAssignerChangelogNormalizeTransposeRule method buildMapping.

private Mappings.TargetMapping buildMapping(RexProgram program) {
    final Map<Integer, Integer> mapInToOutPos = new HashMap<>();
    final List<RexLocalRef> projects = program.getProjectList();
    for (int idx = 0; idx < projects.size(); idx++) {
        RexNode rexNode = program.expandLocalRef(projects.get(idx));
        if (rexNode instanceof RexInputRef) {
            mapInToOutPos.put(((RexInputRef) rexNode).getIndex(), idx);
        }
    }
    return Mappings.target(mapInToOutPos, program.getInputRowType().getFieldCount(), program.getOutputRowType().getFieldCount());
}
Also used : HashMap(java.util.HashMap) RexLocalRef(org.apache.calcite.rex.RexLocalRef) RexInputRef(org.apache.calcite.rex.RexInputRef) RexNode(org.apache.calcite.rex.RexNode)

Example 87 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class PythonCorrelateSplitRule method createTopCalc.

private FlinkLogicalCalc createTopCalc(int primitiveLeftFieldCount, RexBuilder rexBuilder, ArrayBuffer<RexNode> extractedRexNodes, RelDataType calcRowType, FlinkLogicalCorrelate newCorrelate) {
    RexProgram rexProgram = new RexProgramBuilder(newCorrelate.getRowType(), rexBuilder).getProgram();
    int offset = extractedRexNodes.size() + primitiveLeftFieldCount;
    // extract correlate output RexNode.
    List<RexNode> newTopCalcProjects = rexProgram.getExprList().stream().filter(x -> x instanceof RexInputRef).filter(x -> {
        int index = ((RexInputRef) x).getIndex();
        return index < primitiveLeftFieldCount || index >= offset;
    }).collect(Collectors.toList());
    return new FlinkLogicalCalc(newCorrelate.getCluster(), newCorrelate.getTraitSet(), newCorrelate, RexProgram.create(newCorrelate.getRowType(), newTopCalcProjects, null, calcRowType, rexBuilder));
}
Also used : RexFieldAccess(org.apache.calcite.rex.RexFieldAccess) RexProgram(org.apache.calcite.rex.RexProgram) RexUtil(org.apache.calcite.rex.RexUtil) SqlValidatorUtil(org.apache.calcite.sql.validate.SqlValidatorUtil) RexNode(org.apache.calcite.rex.RexNode) LinkedList(java.util.LinkedList) ArrayBuffer(scala.collection.mutable.ArrayBuffer) PythonUtil(org.apache.flink.table.planner.plan.utils.PythonUtil) RelDataType(org.apache.calcite.rel.type.RelDataType) RexDefaultVisitor(org.apache.flink.table.planner.plan.utils.RexDefaultVisitor) RexBuilder(org.apache.calcite.rex.RexBuilder) Iterator(scala.collection.Iterator) FlinkLogicalTableFunctionScan(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalTableFunctionScan) FlinkLogicalCalc(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCalc) RelNode(org.apache.calcite.rel.RelNode) Collectors(java.util.stream.Collectors) RelOptRuleCall(org.apache.calcite.plan.RelOptRuleCall) RexInputRef(org.apache.calcite.rex.RexInputRef) RelOptRule(org.apache.calcite.plan.RelOptRule) RexProgramBuilder(org.apache.calcite.rex.RexProgramBuilder) List(java.util.List) StreamPhysicalCorrelateRule(org.apache.flink.table.planner.plan.rules.physical.stream.StreamPhysicalCorrelateRule) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RexCorrelVariable(org.apache.calcite.rex.RexCorrelVariable) HepRelVertex(org.apache.calcite.plan.hep.HepRelVertex) FlinkLogicalCorrelate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCorrelate) RexCall(org.apache.calcite.rex.RexCall) FlinkLogicalCalc(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCalc) RexProgram(org.apache.calcite.rex.RexProgram) RexInputRef(org.apache.calcite.rex.RexInputRef) RexProgramBuilder(org.apache.calcite.rex.RexProgramBuilder) RexNode(org.apache.calcite.rex.RexNode)

Example 88 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class FlinkSemiAntiJoinProjectTransposeRule method matches.

@Override
public boolean matches(RelOptRuleCall call) {
    LogicalJoin join = call.rel(0);
    LogicalProject project = call.rel(1);
    // only accept SEMI/ANTI join
    JoinRelType joinType = join.getJoinType();
    if (joinType != JoinRelType.SEMI && joinType != JoinRelType.ANTI) {
        return false;
    }
    // all expressions in Project should be RexInputRef
    for (RexNode p : project.getProjects()) {
        if (!(p instanceof RexInputRef)) {
            return false;
        }
    }
    return true;
}
Also used : JoinRelType(org.apache.calcite.rel.core.JoinRelType) LogicalJoin(org.apache.calcite.rel.logical.LogicalJoin) RexInputRef(org.apache.calcite.rex.RexInputRef) LogicalProject(org.apache.calcite.rel.logical.LogicalProject) RexNode(org.apache.calcite.rex.RexNode)

Example 89 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class RelTimeIndicatorConverter method createCalcToMaterializeTimeIndicators.

private RelNode createCalcToMaterializeTimeIndicators(RelNode input, Set<Integer> refIndices) {
    // create new calc
    List<RexNode> projects = input.getRowType().getFieldList().stream().map(field -> {
        RexNode project = new RexInputRef(field.getIndex(), field.getType());
        if (refIndices.contains(field.getIndex())) {
            project = materializeTimeIndicators(project);
        }
        return project;
    }).collect(Collectors.toList());
    RexProgram newProgram = RexProgram.create(input.getRowType(), projects, null, input.getRowType().getFieldNames(), rexBuilder);
    return FlinkLogicalCalc.create(input, newProgram);
}
Also used : WindowUtil.groupingContainsWindowStartEnd(org.apache.flink.table.planner.plan.utils.WindowUtil.groupingContainsWindowStartEnd) RexProgram(org.apache.calcite.rex.RexProgram) Tuple2(org.apache.flink.api.java.tuple.Tuple2) FlinkLogicalWindowAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWindowAggregate) LogicalTableModify(org.apache.calcite.rel.logical.LogicalTableModify) FlinkLogicalLegacySink(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalLegacySink) FlinkLogicalDistribution(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalDistribution) FlinkLogicalIntersect(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalIntersect) TemporalJoinUtil(org.apache.flink.table.planner.plan.utils.TemporalJoinUtil) RexPatternFieldRef(org.apache.calcite.rex.RexPatternFieldRef) RexNode(org.apache.calcite.rex.RexNode) Map(java.util.Map) FlinkLogicalSnapshot(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSnapshot) LocalZonedTimestampType(org.apache.flink.table.types.logical.LocalZonedTimestampType) FlinkLogicalMinus(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalMinus) Predicate(java.util.function.Predicate) FlinkLogicalTableFunctionScan(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalTableFunctionScan) FlinkLogicalRank(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalRank) Set(java.util.Set) FlinkLogicalJoin(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalJoin) Preconditions(org.apache.flink.util.Preconditions) Collectors(java.util.stream.Collectors) FlinkLogicalValues(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalValues) RexInputRef(org.apache.calcite.rex.RexInputRef) List(java.util.List) FlinkTypeFactory.isTimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isTimeIndicatorType) LogicalType(org.apache.flink.table.types.logical.LogicalType) SqlStdOperatorTable(org.apache.calcite.sql.fun.SqlStdOperatorTable) FlinkLogicalSink(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSink) RexLocalRef(org.apache.calcite.rex.RexLocalRef) ValidationException(org.apache.flink.table.api.ValidationException) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) FlinkTypeFactory.isRowtimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isRowtimeIndicatorType) FlinkLogicalSort(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSort) FlinkLogicalOverAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalOverAggregate) RexCall(org.apache.calcite.rex.RexCall) IntStream(java.util.stream.IntStream) TableScan(org.apache.calcite.rel.core.TableScan) LogicalCalc(org.apache.calcite.rel.logical.LogicalCalc) FlinkLogicalMatch(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalMatch) FlinkLogicalUnion(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalUnion) SetOp(org.apache.calcite.rel.core.SetOp) RelWindowProperties(org.apache.flink.table.planner.plan.trait.RelWindowProperties) RelOptUtil(org.apache.calcite.plan.RelOptUtil) Function(java.util.function.Function) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) LinkedHashMap(java.util.LinkedHashMap) MatchUtil.isFinalOnMatchTimeIndicator(org.apache.flink.table.planner.plan.utils.MatchUtil.isFinalOnMatchTimeIndicator) TimestampType(org.apache.flink.table.types.logical.TimestampType) Pair(org.apache.calcite.util.Pair) FlinkLogicalAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalAggregate) TimeIndicatorRelDataType(org.apache.flink.table.planner.plan.schema.TimeIndicatorRelDataType) SqlOperator(org.apache.calcite.sql.SqlOperator) SingleRel(org.apache.calcite.rel.SingleRel) RelCollations(org.apache.calcite.rel.RelCollations) RelDataType(org.apache.calcite.rel.type.RelDataType) SqlTypeName(org.apache.calcite.sql.type.SqlTypeName) FlinkRelMetadataQuery(org.apache.flink.table.planner.plan.metadata.FlinkRelMetadataQuery) RexBuilder(org.apache.calcite.rex.RexBuilder) FlinkTypeFactory.isProctimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isProctimeIndicatorType) TableException(org.apache.flink.table.api.TableException) FlinkLogicalTableAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalTableAggregate) FlinkLogicalCalc(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCalc) FlinkLogicalWindowTableAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWindowTableAggregate) FlinkLogicalExpand(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalExpand) RelNode(org.apache.calcite.rel.RelNode) Aggregate(org.apache.calcite.rel.core.Aggregate) JoinUtil(org.apache.flink.table.planner.plan.utils.JoinUtil) RelHomogeneousShuttle(org.apache.calcite.rel.RelHomogeneousShuttle) RexProgramBuilder(org.apache.calcite.rex.RexProgramBuilder) FlinkSqlOperatorTable(org.apache.flink.table.planner.functions.sql.FlinkSqlOperatorTable) FlinkLogicalWatermarkAssigner(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWatermarkAssigner) AggregateCall(org.apache.calcite.rel.core.AggregateCall) RexShuttle(org.apache.calcite.rex.RexShuttle) FlinkLogicalCorrelate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCorrelate) Collections(java.util.Collections) RexProgram(org.apache.calcite.rex.RexProgram) RexInputRef(org.apache.calcite.rex.RexInputRef) RexNode(org.apache.calcite.rex.RexNode)

Example 90 with RexInputRef

use of org.apache.calcite.rex.RexInputRef in project flink by apache.

the class RelTimeIndicatorConverter method visitJoin.

private RelNode visitJoin(FlinkLogicalJoin join) {
    RelNode newLeft = join.getLeft().accept(this);
    RelNode newRight = join.getRight().accept(this);
    int leftFieldCount = newLeft.getRowType().getFieldCount();
    // temporal table join
    if (TemporalJoinUtil.satisfyTemporalJoin(join, newLeft, newRight)) {
        RelNode rewrittenTemporalJoin = join.copy(join.getTraitSet(), join.getCondition(), newLeft, newRight, join.getJoinType(), join.isSemiJoinDone());
        // Materialize all of the time attributes from the right side of temporal join
        Set<Integer> rightIndices = IntStream.range(0, newRight.getRowType().getFieldCount()).mapToObj(startIdx -> leftFieldCount + startIdx).collect(Collectors.toSet());
        return createCalcToMaterializeTimeIndicators(rewrittenTemporalJoin, rightIndices);
    } else {
        if (JoinUtil.satisfyRegularJoin(join, newLeft, newRight)) {
            // materialize time attribute fields of regular join's inputs
            newLeft = materializeTimeIndicators(newLeft);
            newRight = materializeTimeIndicators(newRight);
        }
        List<RelDataTypeField> leftRightFields = new ArrayList<>();
        leftRightFields.addAll(newLeft.getRowType().getFieldList());
        leftRightFields.addAll(newRight.getRowType().getFieldList());
        RexNode newCondition = join.getCondition().accept(new RexShuttle() {

            @Override
            public RexNode visitInputRef(RexInputRef inputRef) {
                if (isTimeIndicatorType(inputRef.getType())) {
                    return RexInputRef.of(inputRef.getIndex(), leftRightFields);
                } else {
                    return super.visitInputRef(inputRef);
                }
            }
        });
        return FlinkLogicalJoin.create(newLeft, newRight, newCondition, join.getJoinType());
    }
}
Also used : WindowUtil.groupingContainsWindowStartEnd(org.apache.flink.table.planner.plan.utils.WindowUtil.groupingContainsWindowStartEnd) RexProgram(org.apache.calcite.rex.RexProgram) Tuple2(org.apache.flink.api.java.tuple.Tuple2) FlinkLogicalWindowAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWindowAggregate) LogicalTableModify(org.apache.calcite.rel.logical.LogicalTableModify) FlinkLogicalLegacySink(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalLegacySink) FlinkLogicalDistribution(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalDistribution) FlinkLogicalIntersect(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalIntersect) TemporalJoinUtil(org.apache.flink.table.planner.plan.utils.TemporalJoinUtil) RexPatternFieldRef(org.apache.calcite.rex.RexPatternFieldRef) RexNode(org.apache.calcite.rex.RexNode) Map(java.util.Map) FlinkLogicalSnapshot(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSnapshot) LocalZonedTimestampType(org.apache.flink.table.types.logical.LocalZonedTimestampType) FlinkLogicalMinus(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalMinus) Predicate(java.util.function.Predicate) FlinkLogicalTableFunctionScan(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalTableFunctionScan) FlinkLogicalRank(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalRank) Set(java.util.Set) FlinkLogicalJoin(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalJoin) Preconditions(org.apache.flink.util.Preconditions) Collectors(java.util.stream.Collectors) FlinkLogicalValues(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalValues) RexInputRef(org.apache.calcite.rex.RexInputRef) List(java.util.List) FlinkTypeFactory.isTimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isTimeIndicatorType) LogicalType(org.apache.flink.table.types.logical.LogicalType) SqlStdOperatorTable(org.apache.calcite.sql.fun.SqlStdOperatorTable) FlinkLogicalSink(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSink) RexLocalRef(org.apache.calcite.rex.RexLocalRef) ValidationException(org.apache.flink.table.api.ValidationException) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) FlinkTypeFactory.isRowtimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isRowtimeIndicatorType) FlinkLogicalSort(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalSort) FlinkLogicalOverAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalOverAggregate) RexCall(org.apache.calcite.rex.RexCall) IntStream(java.util.stream.IntStream) TableScan(org.apache.calcite.rel.core.TableScan) LogicalCalc(org.apache.calcite.rel.logical.LogicalCalc) FlinkLogicalMatch(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalMatch) FlinkLogicalUnion(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalUnion) SetOp(org.apache.calcite.rel.core.SetOp) RelWindowProperties(org.apache.flink.table.planner.plan.trait.RelWindowProperties) RelOptUtil(org.apache.calcite.plan.RelOptUtil) Function(java.util.function.Function) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) LinkedHashMap(java.util.LinkedHashMap) MatchUtil.isFinalOnMatchTimeIndicator(org.apache.flink.table.planner.plan.utils.MatchUtil.isFinalOnMatchTimeIndicator) TimestampType(org.apache.flink.table.types.logical.TimestampType) Pair(org.apache.calcite.util.Pair) FlinkLogicalAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalAggregate) TimeIndicatorRelDataType(org.apache.flink.table.planner.plan.schema.TimeIndicatorRelDataType) SqlOperator(org.apache.calcite.sql.SqlOperator) SingleRel(org.apache.calcite.rel.SingleRel) RelCollations(org.apache.calcite.rel.RelCollations) RelDataType(org.apache.calcite.rel.type.RelDataType) SqlTypeName(org.apache.calcite.sql.type.SqlTypeName) FlinkRelMetadataQuery(org.apache.flink.table.planner.plan.metadata.FlinkRelMetadataQuery) RexBuilder(org.apache.calcite.rex.RexBuilder) FlinkTypeFactory.isProctimeIndicatorType(org.apache.flink.table.planner.calcite.FlinkTypeFactory.isProctimeIndicatorType) TableException(org.apache.flink.table.api.TableException) FlinkLogicalTableAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalTableAggregate) FlinkLogicalCalc(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCalc) FlinkLogicalWindowTableAggregate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWindowTableAggregate) FlinkLogicalExpand(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalExpand) RelNode(org.apache.calcite.rel.RelNode) Aggregate(org.apache.calcite.rel.core.Aggregate) JoinUtil(org.apache.flink.table.planner.plan.utils.JoinUtil) RelHomogeneousShuttle(org.apache.calcite.rel.RelHomogeneousShuttle) RexProgramBuilder(org.apache.calcite.rex.RexProgramBuilder) FlinkSqlOperatorTable(org.apache.flink.table.planner.functions.sql.FlinkSqlOperatorTable) FlinkLogicalWatermarkAssigner(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalWatermarkAssigner) AggregateCall(org.apache.calcite.rel.core.AggregateCall) RexShuttle(org.apache.calcite.rex.RexShuttle) FlinkLogicalCorrelate(org.apache.flink.table.planner.plan.nodes.logical.FlinkLogicalCorrelate) Collections(java.util.Collections) RelDataTypeField(org.apache.calcite.rel.type.RelDataTypeField) RexShuttle(org.apache.calcite.rex.RexShuttle) RelNode(org.apache.calcite.rel.RelNode) ArrayList(java.util.ArrayList) RexInputRef(org.apache.calcite.rex.RexInputRef) RexNode(org.apache.calcite.rex.RexNode)

Aggregations

RexInputRef (org.apache.calcite.rex.RexInputRef)241 RexNode (org.apache.calcite.rex.RexNode)200 ArrayList (java.util.ArrayList)103 RelNode (org.apache.calcite.rel.RelNode)85 RelDataTypeField (org.apache.calcite.rel.type.RelDataTypeField)80 RexCall (org.apache.calcite.rex.RexCall)67 RelDataType (org.apache.calcite.rel.type.RelDataType)63 RexBuilder (org.apache.calcite.rex.RexBuilder)54 ImmutableBitSet (org.apache.calcite.util.ImmutableBitSet)52 HashMap (java.util.HashMap)47 AggregateCall (org.apache.calcite.rel.core.AggregateCall)36 List (java.util.List)35 HashSet (java.util.HashSet)32 Pair (org.apache.calcite.util.Pair)32 RexLiteral (org.apache.calcite.rex.RexLiteral)29 Map (java.util.Map)24 RelOptUtil (org.apache.calcite.plan.RelOptUtil)24 Set (java.util.Set)20 ImmutableList (com.google.common.collect.ImmutableList)19 LinkedHashMap (java.util.LinkedHashMap)19