Examples with Expr org.apache.druid.math.expr.Expr used on opensource projects

Example 61 with Expr

use of org.apache.druid.math.expr.Expr in project druid by druid-io.

the class JoinFilterCorrelations method eliminateCorrelationDuplicates.

/**
 * Given a list of JoinFilterColumnCorrelationAnalysis, prune the list so that we only have one
 * JoinFilterColumnCorrelationAnalysis for each unique combination of base columns.
 * <p>
 * Suppose we have a join condition like the following, where A is the base table:
 * A.joinColumn == B.joinColumn && A.joinColumn == B.joinColumn2
 * <p>
 * We only need to consider one correlation to A.joinColumn since B.joinColumn and B.joinColumn2 must
 * have the same value in any row that matches the join condition.
 * <p>
 * In the future this method could consider which column correlation should be preserved based on availability of
 * indices and other heuristics.
 * <p>
 * When push down of filters with LHS expressions in the join condition is supported, this method should also
 * consider expressions.
 *
 * @param originalList Original list of column correlation analyses.
 * @return Pruned list of column correlation analyses.
 */
private static List<JoinFilterColumnCorrelationAnalysis> eliminateCorrelationDuplicates(List<JoinFilterColumnCorrelationAnalysis> originalList) {
    Map<Set<String>, JoinFilterColumnCorrelationAnalysis> uniquesMap = new HashMap<>();
    for (JoinFilterColumnCorrelationAnalysis jca : originalList) {
        Set<String> mapKey = new HashSet<>(jca.getBaseColumns());
        for (Expr expr : jca.getBaseExpressions()) {
            mapKey.add(expr.stringify());
        }
        uniquesMap.put(mapKey, jca);
    }
    return new ArrayList<>(uniquesMap.values());
}

Example 62 with Expr

use of org.apache.druid.math.expr.Expr in project druid by druid-io.

the class JoinFilterCorrelations method findCorrelatedBaseTableColumns.

/**
 * For each rhs column that appears in the equiconditions for a table's JoinableClause,
 * we try to determine what base table columns are related to the rhs column through the total set of equiconditions.
 * We do this by searching backwards through the chain of join equiconditions using the provided equicondition map.
 * <p>
 * For example, suppose we have 3 tables, A,B,C, joined with the following conditions, where A is the base table:
 * A.joinColumn == B.joinColumn
 * B.joinColum == C.joinColumnenableRewriteValueColumnFilters
 * <p>
 * We would determine that C.joinColumn is correlated with A.joinColumn: we first see that
 * C.joinColumn is linked to B.joinColumn which in turn is linked to A.joinColumn
 * <p>
 * Suppose we had the following join conditions instead:
 * f(A.joinColumn) == B.joinColumn
 * B.joinColum == C.joinColumn
 * In this case, the JoinFilterColumnCorrelationAnalysis for C.joinColumn would be linked to f(A.joinColumn).
 * <p>
 * Suppose we had the following join conditions instead:
 * A.joinColumn == B.joinColumn
 * f(B.joinColum) == C.joinColumn
 * <p>
 * Because we cannot reverse the function f() applied to the second table B in all cases,
 * we cannot relate C.joinColumn to A.joinColumn, and we would not generate a correlation for C.joinColumn
 *
 * @param joinableClauses     List of joinable clauses for the query
 * @param tablePrefix         Prefix for a join table
 * @param rhsRewriteCandidate RHS rewrite candidate that we find correlated base table columns for
 * @param equiConditions      Map of equiconditions, keyed by the right hand columns
 * @return A list of correlatation analyses for the equicondition RHS columns that reside in the table associated with
 * the tablePrefix
 */
private static Optional<Map<String, JoinFilterColumnCorrelationAnalysis>> findCorrelatedBaseTableColumns(JoinableClauses joinableClauses, String tablePrefix, RhsRewriteCandidate rhsRewriteCandidate, Equiconditions equiConditions) {
    JoinableClause clauseForTablePrefix = rhsRewriteCandidate.getJoinableClause();
    JoinConditionAnalysis jca = clauseForTablePrefix.getCondition();
    Set<String> rhsColumns = new HashSet<>();
    if (rhsRewriteCandidate.isDirectRewrite()) {
        // If we filter on a RHS join column, we only need to consider that column from the RHS side
        rhsColumns.add(rhsRewriteCandidate.getRhsColumn());
    } else {
        for (Equality eq : jca.getEquiConditions()) {
            rhsColumns.add(tablePrefix + eq.getRightColumn());
        }
    }
    Map<String, JoinFilterColumnCorrelationAnalysis> correlations = new LinkedHashMap<>();
    for (String rhsColumn : rhsColumns) {
        Set<String> correlatedBaseColumns = new HashSet<>();
        Set<Expr> correlatedBaseExpressions = new HashSet<>();
        getCorrelationForRHSColumn(joinableClauses, equiConditions, rhsColumn, correlatedBaseColumns, correlatedBaseExpressions);
        if (correlatedBaseColumns.isEmpty() && correlatedBaseExpressions.isEmpty()) {
            continue;
        }
        correlations.put(rhsColumn, new JoinFilterColumnCorrelationAnalysis(rhsColumn, correlatedBaseColumns, correlatedBaseExpressions));
    }
    if (correlations.size() == 0) {
        return Optional.empty();
    } else {
        return Optional.of(correlations);
    }
}

Example 63 with Expr

use of org.apache.druid.math.expr.Expr in project druid by druid-io.

the class JoinFilterCorrelations method getCorrelationForRHSColumn.

/**
 * Helper method for {@link #findCorrelatedBaseTableColumns} that determines correlated base table columns
 * and/or expressions for a single RHS column and adds them to the provided sets as it traverses the
 * equicondition column relationships.
 *
 * @param equiConditions            Map of equiconditions, keyed by the right hand columns
 * @param rhsColumn                 RHS column to find base table correlations for
 * @param correlatedBaseColumns     Set of correlated base column names for the provided RHS column. Will be modified.
 * @param correlatedBaseExpressions Set of correlated base column expressions for the provided RHS column. Will be
 *                                  modified.
 */
private static void getCorrelationForRHSColumn(JoinableClauses joinableClauses, Equiconditions equiConditions, String rhsColumn, Set<String> correlatedBaseColumns, Set<Expr> correlatedBaseExpressions) {
    String findMappingFor = rhsColumn;
    Set<Expr> lhsExprs = equiConditions.getLhsExprs(findMappingFor);
    if (lhsExprs == null) {
        return;
    }
    for (Expr lhsExpr : lhsExprs) {
        String identifier = lhsExpr.getBindingIfIdentifier();
        if (identifier == null) {
            // We push down if the function only requires base table columns
            Expr.BindingAnalysis bindingAnalysis = lhsExpr.analyzeInputs();
            Set<String> requiredBindings = bindingAnalysis.getRequiredBindings();
            if (joinableClauses.areSomeColumnsFromJoin(requiredBindings)) {
                break;
            }
            correlatedBaseExpressions.add(lhsExpr);
        } else {
            // simple identifier, see if we can correlate it with a column on the base table
            findMappingFor = identifier;
            if (joinableClauses.getColumnFromJoinIfExists(identifier) == null) {
                correlatedBaseColumns.add(findMappingFor);
            } else {
                getCorrelationForRHSColumn(joinableClauses, equiConditions, findMappingFor, correlatedBaseColumns, correlatedBaseExpressions);
            }
        }
    }
}

Example 64 with Expr

use of org.apache.druid.math.expr.Expr in project druid by druid-io.

the class JoinConditionAnalysis method computeRequiredColumns.

private static Set<String> computeRequiredColumns(final String rightPrefix, final List<Equality> equiConditions, final List<Expr> nonEquiConditions) {
    final Set<String> requiredColumns = new HashSet<>();
    for (Equality equality : equiConditions) {
        requiredColumns.add(rightPrefix + equality.getRightColumn());
        requiredColumns.addAll(equality.getLeftExpr().analyzeInputs().getRequiredBindings());
    }
    for (Expr expr : nonEquiConditions) {
        requiredColumns.addAll(expr.analyzeInputs().getRequiredBindings());
    }
    return requiredColumns;
}

Example 65 with Expr

use of org.apache.druid.math.expr.Expr in project druid by druid-io.

the class JoinConditionAnalysis method forExpression.

/**
 * Analyze a join condition.
 *
 * @param condition   the condition expression
 * @param rightPrefix prefix for the right-hand side of the join; will be used to determine which identifiers in
 *                    the condition come from the right-hand side and which come from the left-hand side
 * @param macroTable  macro table for parsing the condition expression
 */
public static JoinConditionAnalysis forExpression(final String condition, final String rightPrefix, final ExprMacroTable macroTable) {
    final Expr conditionExpr = Parser.parse(condition, macroTable);
    final List<Equality> equiConditions = new ArrayList<>();
    final List<Expr> nonEquiConditions = new ArrayList<>();
    final List<Expr> exprs = Exprs.decomposeAnd(conditionExpr);
    for (Expr childExpr : exprs) {
        final Optional<Pair<Expr, Expr>> maybeDecomposed = Exprs.decomposeEquals(childExpr);
        if (!maybeDecomposed.isPresent()) {
            nonEquiConditions.add(childExpr);
        } else {
            final Pair<Expr, Expr> decomposed = maybeDecomposed.get();
            final Expr lhs = Objects.requireNonNull(decomposed.lhs);
            final Expr rhs = Objects.requireNonNull(decomposed.rhs);
            if (isLeftExprAndRightColumn(lhs, rhs, rightPrefix)) {
                // rhs is a right-hand column; lhs is an expression solely of the left-hand side.
                equiConditions.add(new Equality(lhs, Objects.requireNonNull(rhs.getBindingIfIdentifier()).substring(rightPrefix.length())));
            } else if (isLeftExprAndRightColumn(rhs, lhs, rightPrefix)) {
                equiConditions.add(new Equality(rhs, Objects.requireNonNull(lhs.getBindingIfIdentifier()).substring(rightPrefix.length())));
            } else {
                nonEquiConditions.add(childExpr);
            }
        }
    }
    return new JoinConditionAnalysis(condition, rightPrefix, equiConditions, nonEquiConditions);
}