Search in sources :

Example 6 with SqlMonotonicity

use of org.apache.calcite.sql.validate.SqlMonotonicity in project flink by apache.

the class RankProcessStrategy method analyzeRankProcessStrategies.

/**
 * Gets {@link RankProcessStrategy} based on input, partitionKey and orderKey.
 */
static List<RankProcessStrategy> analyzeRankProcessStrategies(StreamPhysicalRel rank, ImmutableBitSet partitionKey, RelCollation orderKey) {
    FlinkRelMetadataQuery mq = (FlinkRelMetadataQuery) rank.getCluster().getMetadataQuery();
    List<RelFieldCollation> fieldCollations = orderKey.getFieldCollations();
    boolean isUpdateStream = !ChangelogPlanUtils.inputInsertOnly(rank);
    RelNode input = rank.getInput(0);
    if (isUpdateStream) {
        Set<ImmutableBitSet> upsertKeys = mq.getUpsertKeysInKeyGroupRange(input, partitionKey.toArray());
        if (upsertKeys == null || upsertKeys.isEmpty() || // upsert key should contains partition key
        upsertKeys.stream().noneMatch(k -> k.contains(partitionKey))) {
            // and we fall back to using retract rank
            return Collections.singletonList(RETRACT_STRATEGY);
        } else {
            FlinkRelMetadataQuery fmq = FlinkRelMetadataQuery.reuseOrCreate(mq);
            RelModifiedMonotonicity monotonicity = fmq.getRelModifiedMonotonicity(input);
            boolean isMonotonic = false;
            if (monotonicity != null && !fieldCollations.isEmpty()) {
                isMonotonic = fieldCollations.stream().allMatch(collation -> {
                    SqlMonotonicity fieldMonotonicity = monotonicity.fieldMonotonicities()[collation.getFieldIndex()];
                    RelFieldCollation.Direction direction = collation.direction;
                    if ((fieldMonotonicity == SqlMonotonicity.DECREASING || fieldMonotonicity == SqlMonotonicity.STRICTLY_DECREASING) && direction == RelFieldCollation.Direction.ASCENDING) {
                        // is decreasing
                        return true;
                    } else if ((fieldMonotonicity == SqlMonotonicity.INCREASING || fieldMonotonicity == SqlMonotonicity.STRICTLY_INCREASING) && direction == RelFieldCollation.Direction.DESCENDING) {
                        // is increasing
                        return true;
                    } else {
                        // it is monotonic
                        return fieldMonotonicity == SqlMonotonicity.CONSTANT;
                    }
                });
            }
            if (isMonotonic) {
                // TODO: choose a set of primary key
                return Arrays.asList(new UpdateFastStrategy(upsertKeys.iterator().next().toArray()), RETRACT_STRATEGY);
            } else {
                return Collections.singletonList(RETRACT_STRATEGY);
            }
        }
    } else {
        return Collections.singletonList(APPEND_FAST_STRATEGY);
    }
}
Also used : ImmutableBitSet(org.apache.calcite.util.ImmutableBitSet) SqlMonotonicity(org.apache.calcite.sql.validate.SqlMonotonicity) Arrays(java.util.Arrays) JsonCreator(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonCreator) FlinkRelMetadataQuery(org.apache.flink.table.planner.plan.metadata.FlinkRelMetadataQuery) RelModifiedMonotonicity(org.apache.flink.table.planner.plan.trait.RelModifiedMonotonicity) JsonSubTypes(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonSubTypes) Set(java.util.Set) RelNode(org.apache.calcite.rel.RelNode) FlinkChangelogModeInferenceProgram(org.apache.flink.table.planner.plan.optimize.program.FlinkChangelogModeInferenceProgram) RelFieldCollation(org.apache.calcite.rel.RelFieldCollation) StringUtils(org.apache.commons.lang3.StringUtils) StreamPhysicalRel(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel) JsonProperty(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonProperty) JsonIgnore(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonIgnore) List(java.util.List) RelCollation(org.apache.calcite.rel.RelCollation) JsonTypeInfo(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonTypeInfo) JsonIgnoreProperties(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonIgnoreProperties) JsonTypeName(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.annotation.JsonTypeName) Collections(java.util.Collections) RelNode(org.apache.calcite.rel.RelNode) ImmutableBitSet(org.apache.calcite.util.ImmutableBitSet) RelFieldCollation(org.apache.calcite.rel.RelFieldCollation) SqlMonotonicity(org.apache.calcite.sql.validate.SqlMonotonicity) FlinkRelMetadataQuery(org.apache.flink.table.planner.plan.metadata.FlinkRelMetadataQuery) RelModifiedMonotonicity(org.apache.flink.table.planner.plan.trait.RelModifiedMonotonicity)

Example 7 with SqlMonotonicity

use of org.apache.calcite.sql.validate.SqlMonotonicity in project calcite by apache.

the class RelMdCollation method project.

/**
 * Helper method to determine a {@link Project}'s collation.
 */
public static List<RelCollation> project(RelMetadataQuery mq, RelNode input, List<? extends RexNode> projects) {
    final SortedSet<RelCollation> collations = new TreeSet<>();
    final List<RelCollation> inputCollations = mq.collations(input);
    if (inputCollations == null || inputCollations.isEmpty()) {
        return ImmutableList.of();
    }
    final Multimap<Integer, Integer> targets = LinkedListMultimap.create();
    final Map<Integer, SqlMonotonicity> targetsWithMonotonicity = new HashMap<>();
    for (Ord<RexNode> project : Ord.<RexNode>zip(projects)) {
        if (project.e instanceof RexInputRef) {
            targets.put(((RexInputRef) project.e).getIndex(), project.i);
        } else if (project.e instanceof RexCall) {
            final RexCall call = (RexCall) project.e;
            final RexCallBinding binding = RexCallBinding.create(input.getCluster().getTypeFactory(), call, inputCollations);
            targetsWithMonotonicity.put(project.i, call.getOperator().getMonotonicity(binding));
        }
    }
    final List<RelFieldCollation> fieldCollations = new ArrayList<>();
    loop: for (RelCollation ic : inputCollations) {
        if (ic.getFieldCollations().isEmpty()) {
            continue;
        }
        fieldCollations.clear();
        for (RelFieldCollation ifc : ic.getFieldCollations()) {
            final Collection<Integer> integers = targets.get(ifc.getFieldIndex());
            if (integers.isEmpty()) {
                // cannot do this collation
                continue loop;
            }
            fieldCollations.add(ifc.copy(integers.iterator().next()));
        }
        assert !fieldCollations.isEmpty();
        collations.add(RelCollations.of(fieldCollations));
    }
    final List<RelFieldCollation> fieldCollationsForRexCalls = new ArrayList<>();
    for (Map.Entry<Integer, SqlMonotonicity> entry : targetsWithMonotonicity.entrySet()) {
        final SqlMonotonicity value = entry.getValue();
        switch(value) {
            case NOT_MONOTONIC:
            case CONSTANT:
                break;
            default:
                fieldCollationsForRexCalls.add(new RelFieldCollation(entry.getKey(), RelFieldCollation.Direction.of(value)));
                break;
        }
    }
    if (!fieldCollationsForRexCalls.isEmpty()) {
        collations.add(RelCollations.of(fieldCollationsForRexCalls));
    }
    return ImmutableList.copyOf(collations);
}
Also used : HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) RexCall(org.apache.calcite.rex.RexCall) RexCallBinding(org.apache.calcite.rex.RexCallBinding) RelCollation(org.apache.calcite.rel.RelCollation) TreeSet(java.util.TreeSet) SqlMonotonicity(org.apache.calcite.sql.validate.SqlMonotonicity) RelFieldCollation(org.apache.calcite.rel.RelFieldCollation) RexInputRef(org.apache.calcite.rex.RexInputRef) Collection(java.util.Collection) HashMap(java.util.HashMap) Map(java.util.Map) RexNode(org.apache.calcite.rex.RexNode)

Example 8 with SqlMonotonicity

use of org.apache.calcite.sql.validate.SqlMonotonicity in project calcite by apache.

the class SqlTesterImpl method checkMonotonic.

public void checkMonotonic(String query, SqlMonotonicity expectedMonotonicity) {
    SqlValidator validator = getValidator();
    SqlNode n = parseAndValidate(validator, query);
    final RelDataType rowType = validator.getValidatedNodeType(n);
    final SqlValidatorNamespace selectNamespace = validator.getNamespace(n);
    final String field0 = rowType.getFieldList().get(0).getName();
    final SqlMonotonicity monotonicity = selectNamespace.getMonotonicity(field0);
    assertThat(monotonicity, equalTo(expectedMonotonicity));
}
Also used : SqlValidator(org.apache.calcite.sql.validate.SqlValidator) SqlMonotonicity(org.apache.calcite.sql.validate.SqlMonotonicity) RelDataType(org.apache.calcite.rel.type.RelDataType) SqlValidatorNamespace(org.apache.calcite.sql.validate.SqlValidatorNamespace) SqlNode(org.apache.calcite.sql.SqlNode)

Aggregations

SqlMonotonicity (org.apache.calcite.sql.validate.SqlMonotonicity)8 RelCollation (org.apache.calcite.rel.RelCollation)4 RelFieldCollation (org.apache.calcite.rel.RelFieldCollation)4 ArrayList (java.util.ArrayList)3 TreeSet (java.util.TreeSet)3 RexNode (org.apache.calcite.rex.RexNode)3 BigDecimal (java.math.BigDecimal)2 Collection (java.util.Collection)2 HashMap (java.util.HashMap)2 Map (java.util.Map)2 RexCall (org.apache.calcite.rex.RexCall)2 RexCallBinding (org.apache.calcite.rex.RexCallBinding)2 RexInputRef (org.apache.calcite.rex.RexInputRef)2 SqlNode (org.apache.calcite.sql.SqlNode)2 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 List (java.util.List)1 Set (java.util.Set)1 RelNode (org.apache.calcite.rel.RelNode)1 RelReferentialConstraint (org.apache.calcite.rel.RelReferentialConstraint)1