Search in sources :

Example 1 with MycatView

use of io.mycat.calcite.logical.MycatView in project Mycat2 by MyCATApache.

the class MycatAggDistinctRule method opt.

private void opt(RelOptRuleCall call, Aggregate topAggregate, MycatView mycatView) {
    RelBuilder builder = call.builder();
    builder.push(mycatView.getRelNode().getInput(0));
    List<AggregateCall> aggCallList = topAggregate.getAggCallList();
    List<RexInputRef> collect = aggCallList.get(0).getArgList().stream().map(i -> builder.field(i)).collect(Collectors.toList());
    RelBuilder.AggCall count = builder.count(true, "mycatCountDistinct", collect);
    builder.aggregate(builder.groupKey(), count);
    MycatView newView = mycatView.changeTo(builder.build());
    builder.push(newView);
    builder.aggregate(builder.groupKey(), builder.sum(builder.field(0)));
    call.transformTo(RelOptUtil.createCastRel(builder.build(), topAggregate.getRowType(), true));
}
Also used : AggregateCall(org.apache.calcite.rel.core.AggregateCall) SqlKind(org.apache.calcite.sql.SqlKind) MycatView(io.mycat.calcite.logical.MycatView) RelRule(org.apache.calcite.plan.RelRule) RelNode(org.apache.calcite.rel.RelNode) RelOptUtil(org.apache.calcite.plan.RelOptUtil) Aggregate(org.apache.calcite.rel.core.Aggregate) Collectors(java.util.stream.Collectors) RelOptRuleCall(org.apache.calcite.plan.RelOptRuleCall) RexInputRef(org.apache.calcite.rex.RexInputRef) Consumer(java.util.function.Consumer) List(java.util.List) LocalAggregate(io.mycat.calcite.localrel.LocalAggregate) RelMetadataQuery(org.apache.calcite.rel.metadata.RelMetadataQuery) LocalRules.normalize(io.mycat.calcite.localrel.LocalRules.normalize) RelBuilder(org.apache.calcite.tools.RelBuilder) RelHint(org.apache.calcite.rel.hint.RelHint) HintTools(io.mycat.HintTools) AggregateCall(org.apache.calcite.rel.core.AggregateCall) MycatView(io.mycat.calcite.logical.MycatView) RelBuilder(org.apache.calcite.tools.RelBuilder) RexInputRef(org.apache.calcite.rex.RexInputRef)

Example 2 with MycatView

use of io.mycat.calcite.logical.MycatView in project Mycat2 by MyCATApache.

the class SQLRBORewriter method tryMergeJoin.

@NotNull
public static Optional<RelNode> tryMergeJoin(RelNode left, RelNode right, Join join) {
    if (!DrdsSqlCompiler.RBO_MERGE_JOIN) {
        return Optional.empty();
    }
    boolean isleftView = left instanceof MycatView;
    boolean isRightView = right instanceof MycatView;
    if (isleftView && isRightView) {
        if (((MycatView) left).banPushdown() || ((MycatView) right).banPushdown()) {
            return Optional.empty();
        }
        MycatSortMergeJoin mycatSortMergeJoin = MycatMergeJoinRule.INSTANCE.tryMycatSortMergeJoin(join.copy(join.getTraitSet(), ImmutableList.of(left, right)), true);
        return Optional.ofNullable(mycatSortMergeJoin);
    } else {
        return Optional.empty();
    }
}
Also used : MycatView(io.mycat.calcite.logical.MycatView) MycatSortMergeJoin(io.mycat.calcite.physical.MycatSortMergeJoin) NotNull(org.jetbrains.annotations.NotNull)

Example 3 with MycatView

use of io.mycat.calcite.logical.MycatView in project Mycat2 by MyCATApache.

the class SQLRBORewriter method view.

public static final Optional<RelNode> view(RelNode input, Calc calc) {
    if (input instanceof MycatView && ((MycatView) input).banPushdown()) {
        return Optional.empty();
    }
    final Pair<ImmutableList<RexNode>, ImmutableList<RexNode>> projectFilter = calc.getProgram().split();
    RelBuilder relBuilder = relbuilder(calc.getCluster(), null);
    RelNode relNode = relBuilder.push(input).filter(projectFilter.right).build();
    if (relNode instanceof Filter) {
        return view(input, (Filter) relNode).flatMap(u -> {
            RelNode node = relBuilder.push(u).project(projectFilter.left).build();
            if (node instanceof Project) {
                return view(u, (Project) node);
            } else {
                return Optional.empty();
            }
        });
    } else {
        return Optional.empty();
    }
}
Also used : MycatProject(io.mycat.calcite.physical.MycatProject) MycatView(io.mycat.calcite.logical.MycatView) RelBuilder(org.apache.calcite.tools.RelBuilder) RelNode(org.apache.calcite.rel.RelNode) ImmutableList(com.google.common.collect.ImmutableList)

Example 4 with MycatView

use of io.mycat.calcite.logical.MycatView in project Mycat2 by MyCATApache.

the class SQLRBORewriter method pushDownERTable.

private static Optional<RelNode> pushDownERTable(Join join, MycatView left, MycatView right) {
    if (left.banPushdown() || right.banPushdown()) {
        return Optional.empty();
    }
    switch(join.getJoinType()) {
        case INNER:
        case LEFT:
        case SEMI:
        case ANTI:
        case RIGHT:
            break;
        case FULL:
            return Optional.empty();
        default:
            throw new IllegalStateException("Unexpected value: " + join.getJoinType());
    }
    JoinInfo joinInfo = join.analyzeCondition();
    if (joinInfo.isEqui()) {
        List<IntPair> pairs = joinInfo.pairs();
        if (pairs.isEmpty())
            return Optional.empty();
        RexNode conditions = left.getCondition().orElse(null);
        RelMetadataQuery metadataQuery = join.getCluster().getMetadataQuery();
        IdentityHashMap<TableHandler, Set<String>> keysMap = new IdentityHashMap<>();
        IdentityHashMap<RelColumnOrigin, RelColumnOrigin> equals = new IdentityHashMap<>();
        for (IntPair pair : pairs) {
            RelColumnOrigin leftColumnOrigin = metadataQuery.getColumnOrigin(left.getRelNode(), pair.source);
            RelColumnOrigin rightColumnOrigin = metadataQuery.getColumnOrigin(right.getRelNode(), pair.target);
            if (leftColumnOrigin == null || rightColumnOrigin == null) {
                continue;
            }
            MycatLogicTable leftLogicTable = leftColumnOrigin.getOriginTable().unwrap(MycatLogicTable.class);
            MycatLogicTable rightLogicTable = rightColumnOrigin.getOriginTable().unwrap(MycatLogicTable.class);
            if (!leftColumnOrigin.isDerived() && !rightColumnOrigin.isDerived()) {
                TableHandler leftTableHandler = leftLogicTable.getTable();
                keysMap.computeIfAbsent(leftTableHandler, mycatLogicTable -> new HashSet<>()).add(leftTableHandler.getColumns().get(leftColumnOrigin.getOriginColumnOrdinal()).getColumnName());
                TableHandler rightTableHandler = rightLogicTable.getTable();
                keysMap.computeIfAbsent(rightTableHandler, mycatLogicTable -> new HashSet<>()).add(rightTableHandler.getColumns().get(rightColumnOrigin.getOriginColumnOrdinal()).getColumnName());
                equals.put(leftColumnOrigin, rightColumnOrigin);
            }
        }
        boolean allHitPartition = keysMap.entrySet().stream().allMatch(shardingTableHandlerSetEntry -> {
            TableHandler tableHandler = shardingTableHandlerSetEntry.getKey();
            switch(tableHandler.getType()) {
                case SHARDING:
                    CustomRuleFunction function = ((ShardingTableHandler) tableHandler).function();
                    Set<String> columns = shardingTableHandlerSetEntry.getValue();
                    return function.requireShardingKeys(shardingTableHandlerSetEntry.getValue()) || // 保证命中分区
                    columns.stream().anyMatch(function::isShardingPartitionKey);
                case GLOBAL:
                case NORMAL:
                    return true;
                case CUSTOM:
                case VISUAL:
                case VIEW:
                default:
                    return false;
            }
        });
        if (allHitPartition) {
            // erjoin
            boolean pushDown = isErJoin(equals);
            if (!pushDown) {
                pushDown = isSameTargetPartitionJoin(left, equals);
            }
            if (!pushDown) {
                pushDown = isSameTargetPartitionGlobalOrNormalJoin(left, right, equals);
            }
            if (pushDown) {
                return left.getDistribution().join(right.getDistribution()).map(distribution -> MycatView.ofCondition(join.copy(join.getTraitSet(), ImmutableList.of(left.getRelNode(), right.getRelNode())), distribution, conditions));
            }
        }
    }
    return Optional.empty();
}
Also used : io.mycat(io.mycat) RelShuttleImpl(org.apache.calcite.rel.RelShuttleImpl) LoggerFactory(org.slf4j.LoggerFactory) MycatSortMergeJoin(io.mycat.calcite.physical.MycatSortMergeJoin) IntPair(org.apache.calcite.util.mapping.IntPair) ShardingTableHandler(io.mycat.router.ShardingTableHandler) BigDecimal(java.math.BigDecimal) RexUtil(org.apache.calcite.rex.RexUtil) RexNode(org.apache.calcite.rex.RexNode) RelBuilder(org.apache.calcite.tools.RelBuilder) RelHint(org.apache.calcite.rel.hint.RelHint) org.apache.calcite.rel.logical(org.apache.calcite.rel.logical) RelOptCluster(org.apache.calcite.plan.RelOptCluster) ImmutableBitSet(org.apache.calcite.util.ImmutableBitSet) ImmutableIntList(org.apache.calcite.util.ImmutableIntList) MycatView(io.mycat.calcite.logical.MycatView) RexLiteral(org.apache.calcite.rex.RexLiteral) MycatConvention(io.mycat.calcite.MycatConvention) MycatMergeJoinRule(io.mycat.calcite.rules.MycatMergeJoinRule) Collectors(java.util.stream.Collectors) ServerConfig(io.mycat.config.ServerConfig) QueryType(io.mycat.querycondition.QueryType) RelMetadataQuery(org.apache.calcite.rel.metadata.RelMetadataQuery) SqlStdOperatorTable(org.apache.calcite.sql.fun.SqlStdOperatorTable) RelCollation(org.apache.calcite.rel.RelCollation) LocalRules(io.mycat.calcite.localrel.LocalRules) NotNull(org.jetbrains.annotations.NotNull) java.util(java.util) CustomRuleFunction(io.mycat.router.CustomRuleFunction) MycatHashAggregate(io.mycat.calcite.physical.MycatHashAggregate) NumberFormat(java.text.NumberFormat) RelOptTable(org.apache.calcite.plan.RelOptTable) ImmutableList(com.google.common.collect.ImmutableList) Pair(org.apache.calcite.util.Pair) RelColumnOrigin(org.apache.calcite.rel.metadata.RelColumnOrigin) io.mycat.calcite.table(io.mycat.calcite.table) org.apache.calcite.rel.core(org.apache.calcite.rel.core) RelOptListener(org.apache.calcite.plan.RelOptListener) Logger(org.slf4j.Logger) RexBuilder(org.apache.calcite.rex.RexBuilder) MycatProject(io.mycat.calcite.physical.MycatProject) MycatCalciteSupport(io.mycat.calcite.MycatCalciteSupport) HepPlanner(org.apache.calcite.plan.hep.HepPlanner) HepProgramBuilder(org.apache.calcite.plan.hep.HepProgramBuilder) RelNode(org.apache.calcite.rel.RelNode) LocalSort(io.mycat.calcite.localrel.LocalSort) NameMap(io.mycat.util.NameMap) RelOptSchema(org.apache.calcite.plan.RelOptSchema) MycatAggregateUnionTransposeRule(io.mycat.calcite.localrel.MycatAggregateUnionTransposeRule) RelMetadataQuery(org.apache.calcite.rel.metadata.RelMetadataQuery) ShardingTableHandler(io.mycat.router.ShardingTableHandler) ImmutableBitSet(org.apache.calcite.util.ImmutableBitSet) CustomRuleFunction(io.mycat.router.CustomRuleFunction) IntPair(org.apache.calcite.util.mapping.IntPair) ShardingTableHandler(io.mycat.router.ShardingTableHandler) RelColumnOrigin(org.apache.calcite.rel.metadata.RelColumnOrigin) RexNode(org.apache.calcite.rex.RexNode)

Example 5 with MycatView

use of io.mycat.calcite.logical.MycatView in project Mycat2 by MyCATApache.

the class MycatExtraSortRule method pushSort.

private RelNode pushSort(RelNode relNode, RelCollation targetRelCollation) {
    RelCollation orgianlCollation = relNode.getTraitSet().getCollation();
    if (orgianlCollation != null && orgianlCollation.equals(targetRelCollation)) {
        return relNode;
    }
    RelBuilder relBuilder = relBuilderFactory.create(relNode.getCluster(), null);
    if (relNode instanceof MycatView) {
        MycatView mycatView = (MycatView) relNode;
        RelNode innerRelNode = mycatView.getRelNode();
        relNode = mycatView.changeTo(relBuilder.push(innerRelNode).sort(targetRelCollation).build());
        return relNode;
    } else {
        return LogicalSort.create(relNode, targetRelCollation, null, null);
    }
}
Also used : MycatView(io.mycat.calcite.logical.MycatView) RelBuilder(org.apache.calcite.tools.RelBuilder)

Aggregations

MycatView (io.mycat.calcite.logical.MycatView)29 RelNode (org.apache.calcite.rel.RelNode)21 RexNode (org.apache.calcite.rex.RexNode)9 RelBuilder (org.apache.calcite.tools.RelBuilder)8 RelOptCluster (org.apache.calcite.plan.RelOptCluster)7 RelHint (org.apache.calcite.rel.hint.RelHint)7 RelMetadataQuery (org.apache.calcite.rel.metadata.RelMetadataQuery)7 NotNull (org.jetbrains.annotations.NotNull)7 MycatSQLTableLookup (io.mycat.calcite.physical.MycatSQLTableLookup)6 Collectors (java.util.stream.Collectors)6 ImmutableList (com.google.common.collect.ImmutableList)5 RelCollation (org.apache.calcite.rel.RelCollation)5 RelShuttleImpl (org.apache.calcite.rel.RelShuttleImpl)5 RexBuilder (org.apache.calcite.rex.RexBuilder)5 MycatProject (io.mycat.calcite.physical.MycatProject)4 Join (org.apache.calcite.rel.core.Join)4 JoinRelType (org.apache.calcite.rel.core.JoinRelType)4 RelDataType (org.apache.calcite.rel.type.RelDataType)4 SqlString (org.apache.calcite.sql.util.SqlString)4 io.mycat (io.mycat)3