Examples with ParameterValueExpression org.voltdb.expressions.ParameterValueExpression used on opensource projects

Example 6 with ParameterValueExpression

use of org.voltdb.expressions.ParameterValueExpression in project voltdb by VoltDB.

the class SubPlanAssembler method getIndexableExpressionFromFilters.

/**
     * For a given filter expression, return a normalized version of it that is always a comparison operator whose
     * left-hand-side references the table specified and whose right-hand-side does not.
     * Returns null if no such formulation of the filter expression is possible.
     * For example, "WHERE F_ID = 2" would return it input intact if F_ID is in the table passed in.
     * For join expressions like, "WHERE F_ID = Q_ID", it would also return the input expression if F_ID is in the table
     * but Q_ID is not. If only Q_ID were defined for the table, it would return an expression for (Q_ID = F_ID).
     * If both Q_ID and F_ID were defined on the table, null would be returned.
     * Ideally, the left-hand-side expression is intended to be an indexed expression on the table using the current
     * index. To help reduce false positives, the (base) columns and/or indexed expressions of the index are also
     * provided to help further reduce non-null returns in uninteresting cases.
     *
     * @param targetComparator An allowed comparison operator
     *                         -- its reverse is allowed in reversed expressions
     * @param altTargetComparator An alternatively allowed comparison operator
     *                            -- its reverse is allowed in reversed expressions
     * @param coveringExpr The indexed expression on the table's column
     *                     that might match a query filter, possibly null.
     * @param coveringColId When coveringExpr is null,
     *                      the id of the indexed column might match a query filter.
     * @param tableScan The table scan on which the indexed expression is based
     * @param filtersToCover the query conditions that may contain the desired filter
     * @param allowIndexedJoinFilters Whether filters referencing other tables' columns are acceptable
     * @param filterAction the desired disposition of the matched filter,
                           either EXCLUDE_FROM_POST_FILTERS or KEEP_IN_POST_FILTERS
     * @return An IndexableExpression -- really just a pairing of a normalized form of expr with the
     * potentially indexed expression on the left-hand-side and the potential index key expression on
     * the right of a comparison operator, and a list of parameter bindings that are required for the
     * index scan to be applicable.
     * -- or null if there is no filter that matches the indexed expression
     */
private static IndexableExpression getIndexableExpressionFromFilters(ExpressionType targetComparator, ExpressionType altTargetComparator, AbstractExpression coveringExpr, int coveringColId, StmtTableScan tableScan, List<AbstractExpression> filtersToCover, boolean allowIndexedJoinFilters, boolean filterAction) {
    List<AbstractExpression> binding = null;
    AbstractExpression indexableExpr = null;
    AbstractExpression otherExpr = null;
    ComparisonExpression normalizedExpr = null;
    AbstractExpression originalFilter = null;
    for (AbstractExpression filter : filtersToCover) {
        // ENG-8203: Not going to try to use index with sub-query expression
        if (filter.hasSubquerySubexpression()) {
            // SelectSubqueryExpression also can be scalar sub-query
            continue;
        }
        // Expression type must be resolvable by an index scan
        if ((filter.getExpressionType() == targetComparator) || (filter.getExpressionType() == altTargetComparator)) {
            normalizedExpr = (ComparisonExpression) filter;
            indexableExpr = filter.getLeft();
            otherExpr = filter.getRight();
            binding = bindingIfValidIndexedFilterOperand(tableScan, indexableExpr, otherExpr, coveringExpr, coveringColId);
            if (binding != null) {
                if (!allowIndexedJoinFilters) {
                    if (otherExpr.hasTupleValueSubexpression()) {
                        // This filter can not be used with the index, possibly due to interactions
                        // wih IN LIST processing that would require a three-way NLIJ.
                        binding = null;
                        continue;
                    }
                }
                // Additional restrictions apply to LIKE pattern arguments
                if (targetComparator == ExpressionType.COMPARE_LIKE) {
                    if (otherExpr instanceof ParameterValueExpression) {
                        ParameterValueExpression pve = (ParameterValueExpression) otherExpr;
                        // Can't use an index for parameterized LIKE filters,
                        // e.g. "T1.column LIKE ?"
                        // UNLESS the parameter was artificially substituted
                        // for a user-specified constant AND that constant was a prefix pattern.
                        // In that case, the parameter has to be added to the bound list
                        // for this index/statement.
                        ConstantValueExpression cve = pve.getOriginalValue();
                        if (cve == null || !cve.isPrefixPatternString()) {
                            // the filter is not usable, so the binding is invalid
                            binding = null;
                            continue;
                        }
                        // Remember that the binding list returned by
                        // bindingIfValidIndexedFilterOperand above
                        // is often a "shared object" and is intended to be treated as immutable.
                        // To add a parameter to it, first copy the List.
                        List<AbstractExpression> moreBinding = new ArrayList<>(binding);
                        moreBinding.add(pve);
                        binding = moreBinding;
                    } else if (otherExpr instanceof ConstantValueExpression) {
                        // Can't use an index for non-prefix LIKE filters,
                        // e.g. " T1.column LIKE '%ish' "
                        ConstantValueExpression cve = (ConstantValueExpression) otherExpr;
                        if (!cve.isPrefixPatternString()) {
                            // The constant is not an index-friendly prefix pattern.
                            // the filter is not usable, so the binding is invalid
                            binding = null;
                            continue;
                        }
                    } else {
                        // Other cases are not indexable, e.g. " T1.column LIKE T2.column "
                        // the filter is not usable, so the binding is invalid
                        binding = null;
                        continue;
                    }
                }
                if (targetComparator == ExpressionType.COMPARE_IN) {
                    if (otherExpr.hasTupleValueSubexpression()) {
                        // This is a fancy edge case where the expression could only be indexed
                        // if it:
                        // A) does not reference the indexed table and
                        // B) has ee support for a three-way NLIJ where the table referenced in
                        // the list element expression feeds values from its current row to the
                        // Materialized scan which then re-evaluates its expressions to
                        // re-populate the temp table that drives the injected NLIJ with
                        // this index scan.
                        // This is a slightly more twisted variant of the three-way NLIJ that
                        // would be needed to support compound key indexing on a combination
                        // of (fixed) IN LIST elements and join key values from other tables.
                        // Punt for now on indexing this IN LIST filter.
                        // the filter is not usable, so the binding is invalid
                        binding = null;
                        continue;
                    }
                    if (otherExpr instanceof ParameterValueExpression) {
                    // It's OK to use an index for a parameterized IN filter,
                    // e.g. "T1.column IN ?"
                    // EVEN if the parameter was -- someday -- artificially substituted
                    // for an entire user-specified list of constants.
                    // As of now, that is beyond the capabilities of the ad hoc statement
                    // parameterizer, so "T1.column IN (3, 4)" can use the plan for
                    // "T1.column IN (?, ?)" that might have been originally cached for
                    // "T1.column IN (1, 2)" but "T1.column IN (1, 2, 3)" would need its own
                    // "T1.column IN (?, ?, ?)" plan, etc. per list element count.
                    } else //TODO: Some day, there may be an optimization here that allows an entire
                    // IN LIST of constants to be serialized as a single value instead of a
                    // VectorValue composed of ConstantValue arguments.
                    // What's TBD is whether that would get its own AbstractExpression class or
                    // just be a special case of ConstantValueExpression.
                    {
                        assert (otherExpr instanceof VectorValueExpression);
                    }
                }
                originalFilter = filter;
                if (filterAction == EXCLUDE_FROM_POST_FILTERS) {
                    filtersToCover.remove(filter);
                }
                break;
            }
        }
        if ((filter.getExpressionType() == ComparisonExpression.reverses.get(targetComparator)) || (filter.getExpressionType() == ComparisonExpression.reverses.get(altTargetComparator))) {
            normalizedExpr = (ComparisonExpression) filter;
            normalizedExpr = normalizedExpr.reverseOperator();
            indexableExpr = filter.getRight();
            otherExpr = filter.getLeft();
            binding = bindingIfValidIndexedFilterOperand(tableScan, indexableExpr, otherExpr, coveringExpr, coveringColId);
            if (binding != null) {
                if (!allowIndexedJoinFilters) {
                    if (otherExpr.hasTupleValueSubexpression()) {
                        // This filter can not be used with the index, probably due to interactions
                        // with IN LIST processing of another key component that would require a
                        // three-way NLIJ to be injected.
                        binding = null;
                        continue;
                    }
                }
                originalFilter = filter;
                if (filterAction == EXCLUDE_FROM_POST_FILTERS) {
                    filtersToCover.remove(filter);
                }
                break;
            }
        }
    }
    if (binding == null) {
        // ran out of candidate filters.
        return null;
    }
    return new IndexableExpression(originalFilter, normalizedExpr, binding);
}

Example 7 with ParameterValueExpression

use of org.voltdb.expressions.ParameterValueExpression in project voltdb by VoltDB.

the class SubPlanAssembler method injectIndexedJoinWithMaterializedScan.

// Generate a plan for an IN-LIST-driven index scan
private static AbstractPlanNode injectIndexedJoinWithMaterializedScan(AbstractExpression listElements, IndexScanPlanNode scanNode) {
    MaterializedScanPlanNode matScan = new MaterializedScanPlanNode();
    assert (listElements instanceof VectorValueExpression || listElements instanceof ParameterValueExpression);
    matScan.setRowData(listElements);
    matScan.setSortDirection(scanNode.getSortDirection());
    NestLoopIndexPlanNode nlijNode = new NestLoopIndexPlanNode();
    nlijNode.setJoinType(JoinType.INNER);
    nlijNode.addInlinePlanNode(scanNode);
    nlijNode.addAndLinkChild(matScan);
    // resolve the sort direction
    nlijNode.resolveSortDirection();
    return nlijNode;
}

Example 8 with ParameterValueExpression

use of org.voltdb.expressions.ParameterValueExpression in project voltdb by VoltDB.

the class StatementPartitioning method addPartitioningExpression.

/**
     * @param string table.column name of a(nother) equality-filtered partitioning column
     * @param constExpr -- a constant/parameter-based expression that equality-filters the partitioning column
     */
public void addPartitioningExpression(String fullColumnName, AbstractExpression constExpr, VoltType valueType) {
    //* enable to debug */ System.out.println("DEBUG: addPartitioningExpression(" + fullColumnName + ", " + constExpr + ")");
    if (m_fullColumnName == null) {
        m_fullColumnName = fullColumnName;
    }
    m_inferredExpression.add(constExpr);
    if (constExpr instanceof ParameterValueExpression) {
        ParameterValueExpression pve = (ParameterValueExpression) constExpr;
        m_inferredParameterIndex = pve.getParameterIndex();
    } else {
        m_inferredValue = ConstantValueExpression.extractPartitioningValue(valueType, constExpr);
    }
}

Example 9 with ParameterValueExpression

use of org.voltdb.expressions.ParameterValueExpression in project voltdb by VoltDB.

the class ProcedureCompiler method compileSingleStmtProcedure.

static void compileSingleStmtProcedure(VoltCompiler compiler, HSQLInterface hsql, DatabaseEstimates estimates, Database db, ProcedureDescriptor procedureDescriptor) throws VoltCompiler.VoltCompilerException {
    final String className = procedureDescriptor.m_className;
    if (className.indexOf('@') != -1) {
        throw compiler.new VoltCompilerException("User procedure names can't contain \"@\".");
    }
    // get the short name of the class (no package if a user procedure)
    // use the Table.<builtin> name (allowing the period) if builtin.
    String shortName = className;
    if (procedureDescriptor.m_builtInStmt == false) {
        String[] parts = className.split("\\.");
        shortName = parts[parts.length - 1];
    }
    // add an entry to the catalog (using the full className)
    final Procedure procedure = db.getProcedures().add(shortName);
    for (String groupName : procedureDescriptor.m_authGroups) {
        final Group group = db.getGroups().get(groupName);
        if (group == null) {
            throw compiler.new VoltCompilerException("Procedure " + className + " allows access by a role " + groupName + " that does not exist");
        }
        final GroupRef groupRef = procedure.getAuthgroups().add(groupName);
        groupRef.setGroup(group);
    }
    procedure.setClassname(className);
    // sysprocs don't use the procedure compiler
    procedure.setSystemproc(false);
    procedure.setDefaultproc(procedureDescriptor.m_builtInStmt);
    procedure.setHasjava(false);
    procedure.setTransactional(true);
    // get the annotation
    // first try to get one that has been passed from the compiler
    ProcInfoData info = compiler.getProcInfoOverride(shortName);
    // and create a ProcInfo.Data instance for it
    if (info == null) {
        info = new ProcInfoData();
        if (procedureDescriptor.m_partitionString != null) {
            info.partitionInfo = procedureDescriptor.m_partitionString;
            info.singlePartition = true;
        }
    }
    assert (info != null);
    // ADD THE STATEMENT
    // add the statement to the catalog
    Statement catalogStmt = procedure.getStatements().add(VoltDB.ANON_STMT_NAME);
    // compile the statement
    StatementPartitioning partitioning = info.singlePartition ? StatementPartitioning.forceSP() : StatementPartitioning.forceMP();
    // default to FASTER detmode because stmt procs can't feed read output into writes
    StatementCompiler.compileFromSqlTextAndUpdateCatalog(compiler, hsql, db, estimates, catalogStmt, procedureDescriptor.m_singleStmt, procedureDescriptor.m_joinOrder, DeterminismMode.FASTER, partitioning);
    // if the single stmt is not read only, then the proc is not read only
    boolean procHasWriteStmts = (catalogStmt.getReadonly() == false);
    // set the read onlyness of a proc
    procedure.setReadonly(procHasWriteStmts == false);
    int seqs = catalogStmt.getSeqscancount();
    procedure.setHasseqscans(seqs > 0);
    // set procedure parameter types
    CatalogMap<ProcParameter> params = procedure.getParameters();
    CatalogMap<StmtParameter> stmtParams = catalogStmt.getParameters();
    // set the procedure parameter types from the statement parameter types
    int paramCount = 0;
    for (StmtParameter stmtParam : CatalogUtil.getSortedCatalogItems(stmtParams, "index")) {
        // name each parameter "param1", "param2", etc...
        ProcParameter procParam = params.add("param" + String.valueOf(paramCount));
        procParam.setIndex(stmtParam.getIndex());
        procParam.setIsarray(stmtParam.getIsarray());
        procParam.setType(stmtParam.getJavatype());
        paramCount++;
    }
    // parse the procinfo
    procedure.setSinglepartition(info.singlePartition);
    if (info.singlePartition) {
        parsePartitionInfo(compiler, db, procedure, info.partitionInfo);
        if (procedure.getPartitionparameter() >= params.size()) {
            String msg = "PartitionInfo parameter not a valid parameter for procedure: " + procedure.getClassname();
            throw compiler.new VoltCompilerException(msg);
        }
    // TODO: The planner does not currently validate that a single-statement plan declared as single-partition correctly uses
    // the designated parameter as a partitioning filter, maybe some day.
    // In theory, the PartitioningForStatement would confirm the use of (only) a parameter as a partition key --
    // or if the partition key was determined to be some other hard-coded constant (expression?) it might display a warning
    // message that the passed parameter is assumed to be equal to that constant (expression).
    } else {
        if (partitioning.getCountOfIndependentlyPartitionedTables() == 1) {
            AbstractExpression statementPartitionExpression = partitioning.singlePartitioningExpressionForReport();
            if (statementPartitionExpression != null) {
                // The planner has uncovered an overlooked opportunity to run the statement SP.
                String msg = "This procedure " + shortName + " would benefit from being partitioned, by ";
                String tableName = "tableName", partitionColumnName = "partitionColumnName";
                try {
                    assert (partitioning.getFullColumnName() != null);
                    String[] array = partitioning.getFullColumnName().split("\\.");
                    tableName = array[0];
                    partitionColumnName = array[1];
                } catch (Exception ex) {
                }
                if (statementPartitionExpression instanceof ParameterValueExpression) {
                    paramCount = ((ParameterValueExpression) statementPartitionExpression).getParameterIndex();
                } else {
                    String valueDescription = null;
                    Object partitionValue = partitioning.getInferredPartitioningValue();
                    if (partitionValue == null) {
                        // Statement partitioned on a runtime constant. This is likely to be cryptic, but hopefully gets the idea across.
                        valueDescription = "of " + statementPartitionExpression.explain("");
                    } else {
                        // A simple constant value COULD have been a parameter.
                        valueDescription = partitionValue.toString();
                    }
                    msg += "adding a parameter to be passed the value " + valueDescription + " and ";
                }
                msg += "adding a 'PARTITION ON TABLE " + tableName + " COLUMN " + partitionColumnName + " PARAMETER " + paramCount + "' clause to the " + "CREATE PROCEDURE statement. or using a separate PARTITION PROCEDURE statement";
                compiler.addWarn(msg);
            }
        }
    }
}

Example 10 with ParameterValueExpression

use of org.voltdb.expressions.ParameterValueExpression in project voltdb by VoltDB.

the class TupleScanPlanNode method toJSONString.

@Override
public void toJSONString(JSONStringer stringer) throws JSONException {
    super.toJSONString(stringer);
    // Output the correlated parameter ids
    stringer.key(Members.PARAM_IDX.name()).array();
    for (AbstractExpression colExpr : m_columnList) {
        assert (colExpr instanceof ParameterValueExpression);
        ParameterValueExpression pve = (ParameterValueExpression) colExpr;
        stringer.value(pve.getParameterIndex());
    }
    stringer.endArray();
}