Search in sources :

Example 16 with CQLWord

use of com.datastax.oss.dsbulk.mapping.CQLWord in project dsbulk by datastax.

the class SchemaSettings method validateKeyPresent.

private void validateKeyPresent(ImmutableMultimap<MappingField, CQLFragment> fieldsToVariables, List<ColumnMetadata> columns) {
    Collection<CQLFragment> mappingVariables = fieldsToVariables.values();
    Map<CQLWord, CQLFragment> queryVariables = queryInspector.getAssignments();
    for (ColumnMetadata pk : columns) {
        CQLWord pkVariable = CQLWord.fromCqlIdentifier(pk.getName());
        CQLFragment queryVariable = queryVariables.get(pkVariable);
        // the provided query did not contain such column
        if (queryVariable == null) {
            throw new IllegalArgumentException("Missing required primary key column " + pkVariable.render(VARIABLE) + " from schema.mapping or schema.query");
        }
        // if the PK is mapped to a function or a literal in the query (DAT-326)
        if (queryVariable instanceof CQLWord) {
            // the mapping did not contain such column
            if (!mappingVariables.contains(queryVariable)) {
                throw new IllegalArgumentException("Missing required primary key column " + pkVariable.render(VARIABLE) + " from schema.mapping");
            }
        }
    }
}
Also used : ColumnMetadata(com.datastax.oss.driver.api.core.metadata.schema.ColumnMetadata) CQLWord(com.datastax.oss.dsbulk.mapping.CQLWord) CQLFragment(com.datastax.oss.dsbulk.mapping.CQLFragment)

Example 17 with CQLWord

use of com.datastax.oss.dsbulk.mapping.CQLWord in project dsbulk by datastax.

the class SchemaSettings method columnsToVariables.

@NonNull
private Set<CQLWord> columnsToVariables(Collection<ColumnMetadata> columns) {
    Map<CQLWord, CQLFragment> boundVariables = queryInspector.getAssignments();
    Set<CQLWord> variables = new HashSet<>(columns.size());
    for (ColumnMetadata column : columns) {
        CQLFragment variable = boundVariables.get(CQLWord.fromCqlIdentifier(column.getName()));
        if (variable instanceof CQLWord) {
            variables.add((CQLWord) variable);
        }
    }
    return variables;
}
Also used : ColumnMetadata(com.datastax.oss.driver.api.core.metadata.schema.ColumnMetadata) CQLWord(com.datastax.oss.dsbulk.mapping.CQLWord) CQLFragment(com.datastax.oss.dsbulk.mapping.CQLFragment) LinkedHashSet(java.util.LinkedHashSet) HashSet(java.util.HashSet) NonNull(edu.umd.cs.findbugs.annotations.NonNull)

Example 18 with CQLWord

use of com.datastax.oss.dsbulk.mapping.CQLWord in project dsbulk by datastax.

the class SchemaSettings method inferFieldsToVariablesMap.

private ImmutableMultimap<MappingField, CQLFragment> inferFieldsToVariablesMap(Collection<CQLFragment> columns) {
    // use a builder to preserve iteration order
    ImmutableMultimap.Builder<MappingField, CQLFragment> fieldsToVariables = ImmutableMultimap.builder();
    List<CQLFragment> excludedVariables = new ArrayList<>(mapping.getExcludedVariables());
    if (!isCounterTable() && schemaGenerationStrategy.isMapping()) {
        for (CQLWord variable : mapping.getExcludedVariables()) {
            if (preserveTimestamp) {
                excludedVariables.add(new FunctionCall(null, WRITETIME, variable));
            }
            if (preserveTtl) {
                excludedVariables.add(new FunctionCall(null, TTL, variable));
            }
        }
    }
    int i = 0;
    for (CQLFragment colName : columns) {
        if (!excludedVariables.contains(colName)) {
            if (mappingPreference == INDEXED_ONLY) {
                fieldsToVariables.put(new IndexedMappingField(i), colName);
            } else {
                fieldsToVariables.put(new MappedMappingField(colName.render(INTERNAL)), colName);
            }
        }
        i++;
    }
    return fieldsToVariables.build();
}
Also used : MappedMappingField(com.datastax.oss.dsbulk.mapping.MappedMappingField) ArrayList(java.util.ArrayList) IndexedMappingField(com.datastax.oss.dsbulk.mapping.IndexedMappingField) CQLWord(com.datastax.oss.dsbulk.mapping.CQLWord) ImmutableMultimap(com.datastax.oss.driver.shaded.guava.common.collect.ImmutableMultimap) CQLFragment(com.datastax.oss.dsbulk.mapping.CQLFragment) FunctionCall(com.datastax.oss.dsbulk.mapping.FunctionCall) IndexedMappingField(com.datastax.oss.dsbulk.mapping.IndexedMappingField) MappingField(com.datastax.oss.dsbulk.mapping.MappingField) MappedMappingField(com.datastax.oss.dsbulk.mapping.MappedMappingField)

Example 19 with CQLWord

use of com.datastax.oss.dsbulk.mapping.CQLWord in project dsbulk by datastax.

the class QueryInspectorTest method should_detect_function_select.

@Test
void should_detect_function_select() {
    QueryInspector inspector = new QueryInspector("SELECT myFunction(col1) as f1, \"MyFunction\"(\"My Col 2\") FROM ks.table1");
    CQLWord name1 = CQLWord.fromInternal("myfunction");
    CQLWord name2 = CQLWord.fromInternal("MyFunction");
    FunctionCall f1 = new FunctionCall(null, name1, COL_1);
    FunctionCall f2 = new FunctionCall(null, name2, MY_COL_2);
    assertThat(inspector.getResultSetVariables()).hasSize(2).containsKeys(f1, f2).containsValues(CQLWord.fromInternal("f1"), f2);
}
Also used : CQLWord(com.datastax.oss.dsbulk.mapping.CQLWord) FunctionCall(com.datastax.oss.dsbulk.mapping.FunctionCall) Test(org.junit.jupiter.api.Test) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest)

Example 20 with CQLWord

use of com.datastax.oss.dsbulk.mapping.CQLWord in project dsbulk by datastax.

the class QueryInspector method visitNormalInsertStatement.

@Override
public CQLFragment visitNormalInsertStatement(NormalInsertStatementContext ctx) {
    if (ctx.cident().size() != ctx.term().size()) {
        throw new IllegalArgumentException(String.format("Invalid query: the number of columns to insert (%d) does not match the number of terms (%d): %s.", ctx.cident().size(), ctx.term().size(), query));
    }
    for (int i = 0; i < ctx.cident().size(); i++) {
        CQLWord column = visitCident(ctx.cident().get(i));
        CQLFragment variable = visitTerm(ctx.term().get(i));
        assignmentsBuilder.put(column, variable == QUESTION_MARK ? column : variable);
    }
    if (ctx.usingClause() != null) {
        visitUsingClause(ctx.usingClause());
    }
    return null;
}
Also used : CQLWord(com.datastax.oss.dsbulk.mapping.CQLWord) CQLFragment(com.datastax.oss.dsbulk.mapping.CQLFragment)

Aggregations

CQLWord (com.datastax.oss.dsbulk.mapping.CQLWord)23 CQLFragment (com.datastax.oss.dsbulk.mapping.CQLFragment)12 ColumnDefinitions (com.datastax.oss.driver.api.core.cql.ColumnDefinitions)8 RecordMapper (com.datastax.oss.dsbulk.workflow.commons.schema.RecordMapper)8 Config (com.typesafe.config.Config)8 EnumSet (java.util.EnumSet)8 Set (java.util.Set)8 DefaultMapping (com.datastax.oss.dsbulk.mapping.DefaultMapping)7 FunctionCall (com.datastax.oss.dsbulk.mapping.FunctionCall)7 CqlIdentifier (com.datastax.oss.driver.api.core.CqlIdentifier)6 Field (com.datastax.oss.dsbulk.connectors.api.Field)6 IndexedMappingField (com.datastax.oss.dsbulk.mapping.IndexedMappingField)6 MappedMappingField (com.datastax.oss.dsbulk.mapping.MappedMappingField)6 MappingField (com.datastax.oss.dsbulk.mapping.MappingField)6 NonNull (edu.umd.cs.findbugs.annotations.NonNull)6 Test (org.junit.jupiter.api.Test)6 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)6 ColumnMetadata (com.datastax.oss.driver.api.core.metadata.schema.ColumnMetadata)5 DataType (com.datastax.oss.driver.api.core.type.DataType)5 ArrayList (java.util.ArrayList)5