Examples with HiveSortLimit org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit used on opensource projects

Example 6 with HiveSortLimit

use of org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit in project hive by apache.

the class HiveSortLimitRemoveRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final HiveSortLimit sortLimit = call.rel(0);
    // We remove the limit operator
    call.transformTo(sortLimit.getInput());
}

Example 7 with HiveSortLimit

use of org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit in project hive by apache.

the class HiveSortRemoveRule method onMatch.

@Override
public void onMatch(RelOptRuleCall call) {
    final HiveSortLimit sortLimit = call.rel(0);
    // We remove the limit operator
    call.transformTo(sortLimit.getInput());
}

Example 8 with HiveSortLimit

use of org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit in project hive by apache.

the class HiveSortRemoveRule method matches.

// ~ Methods ----------------------------------------------------------------
@Override
public boolean matches(RelOptRuleCall call) {
    final HiveSortLimit sortLimit = call.rel(0);
    // If it is not created by HiveSortJoinReduceRule, we cannot remove it
    if (!sortLimit.isRuleCreated()) {
        return false;
    }
    // Finally, if we do not reduce the size input enough, we bail out
    int limit = RexLiteral.intValue(sortLimit.fetch);
    Double rowCount = call.getMetadataQuery().getRowCount(sortLimit.getInput());
    if (rowCount != null && limit <= reductionProportion * rowCount && rowCount - limit >= reductionTuples) {
        return false;
    }
    return true;
}

Example 9 with HiveSortLimit

use of org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit in project hive by apache.

the class HiveSortUnionReduceRule method onMatch.

public void onMatch(RelOptRuleCall call) {
    final HiveSortLimit sort = call.rel(0);
    final HiveUnion union = call.rel(1);
    List<RelNode> inputs = new ArrayList<>();
    // Thus we use 'finishPushSortPastUnion' as a flag to identify if we have finished pushing the
    // sort past a union.
    boolean finishPushSortPastUnion = true;
    final int offset = sort.offset == null ? 0 : RexLiteral.intValue(sort.offset);
    for (RelNode input : union.getInputs()) {
        // If we do not reduce the input size, we bail out
        if (RexLiteral.intValue(sort.fetch) + offset < call.getMetadataQuery().getRowCount(input)) {
            finishPushSortPastUnion = false;
            // Here we do some query rewrite. We first get the new fetchRN, which is
            // a sum of offset and fetch.
            // We then push it through by creating a new branchSort with the new
            // fetchRN but no offset.
            RexNode fetchRN = sort.getCluster().getRexBuilder().makeExactLiteral(BigDecimal.valueOf(RexLiteral.intValue(sort.fetch) + offset));
            HiveSortLimit branchSort = sort.copy(sort.getTraitSet(), input, sort.getCollation(), null, fetchRN);
            branchSort.setRuleCreated(true);
            inputs.add(branchSort);
        } else {
            inputs.add(input);
        }
    }
    // there is nothing to change
    if (finishPushSortPastUnion) {
        return;
    }
    // create new union and sort
    HiveUnion unionCopy = (HiveUnion) union.copy(union.getTraitSet(), inputs, union.all);
    HiveSortLimit result = sort.copy(sort.getTraitSet(), unionCopy, sort.getCollation(), sort.offset, sort.fetch);
    call.transformTo(result);
}

Example 10 with HiveSortLimit

use of org.apache.hadoop.hive.ql.optimizer.calcite.reloperators.HiveSortLimit in project hive by apache.

the class HiveProjectSortTransposeRule method onMatch.

// ~ Methods ----------------------------------------------------------------
// implement RelOptRule
public void onMatch(RelOptRuleCall call) {
    final HiveProject project = call.rel(0);
    final HiveSortLimit sort = call.rel(1);
    final RelOptCluster cluster = project.getCluster();
    List<RelFieldCollation> fieldCollations = getNewRelFieldCollations(project, sort.getCollation(), cluster);
    if (fieldCollations == null) {
        return;
    }
    RelTraitSet traitSet = sort.getCluster().traitSetOf(HiveRelNode.CONVENTION);
    RelCollation newCollation = traitSet.canonize(RelCollationImpl.of(fieldCollations));
    // New operators
    final RelNode newProject = project.copy(sort.getInput().getTraitSet(), ImmutableList.of(sort.getInput()));
    final HiveSortLimit newSort = sort.copy(newProject.getTraitSet(), newProject, newCollation, sort.offset, sort.fetch);
    call.transformTo(newSort);
}