Search in sources :

Example 1 with RequiredColumns

use of io.confluent.ksql.planner.RequiredColumns in project ksql by confluentinc.

the class FinalProjectNode method validateProjection.

/**
 * Called to validate that columns referenced in the projection are valid.
 *
 * <p>This is necessary as some joins can create synthetic key columns that do not come
 * from any data source.  This means the normal column validation done during analysis can not
 * fail on unknown column with generated column names.
 *
 * <p>Once the logical model has been built the synthetic key names are known and generated
 * column names can be validated.
 */
private void validateProjection() {
    // Validate any column in the projection that might be a synthetic
    // Only really need to include any that might be, but we include all:
    final RequiredColumns requiredColumns = RequiredColumns.builder().addAll(projection.singleExpressions()).build();
    final Set<ColumnReferenceExp> unknown = getSource().validateColumns(requiredColumns);
    if (!unknown.isEmpty()) {
        final String errors = unknown.stream().map(columnRef -> NodeLocation.asPrefix(columnRef.getLocation()) + "Column '" + columnRef + "' cannot be resolved.").collect(Collectors.joining(System.lineSeparator()));
        throw new KsqlException(errors);
    }
}
Also used : DataSource(io.confluent.ksql.metastore.model.DataSource) ColumnName(io.confluent.ksql.name.ColumnName) SourceName(io.confluent.ksql.name.SourceName) GrammaticalJoiner(io.confluent.ksql.util.GrammaticalJoiner) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) Name(io.confluent.ksql.name.Name) HashMap(java.util.HashMap) SelectItem(io.confluent.ksql.parser.tree.SelectItem) HashSet(java.util.HashSet) AsValue(io.confluent.ksql.function.udf.AsValue) Pair(io.confluent.ksql.util.Pair) ImmutableList(com.google.common.collect.ImmutableList) Analysis(io.confluent.ksql.analyzer.Analysis) Map(java.util.Map) MetaStore(io.confluent.ksql.metastore.MetaStore) Projection(io.confluent.ksql.planner.Projection) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) RequiredColumns(io.confluent.ksql.planner.RequiredColumns) SystemColumns(io.confluent.ksql.schema.ksql.SystemColumns) Set(java.util.Set) KsqlConfig(io.confluent.ksql.util.KsqlConfig) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Collectors(java.util.stream.Collectors) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) Builder(io.confluent.ksql.schema.ksql.LogicalSchema.Builder) NodeLocation(io.confluent.ksql.parser.NodeLocation) List(java.util.List) Entry(java.util.Map.Entry) KsqlException(io.confluent.ksql.util.KsqlException) Optional(java.util.Optional) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) RequiredColumns(io.confluent.ksql.planner.RequiredColumns) KsqlException(io.confluent.ksql.util.KsqlException)

Example 2 with RequiredColumns

use of io.confluent.ksql.planner.RequiredColumns in project ksql by confluentinc.

the class JoinNode method validateColumns.

@Override
protected Set<ColumnReferenceExp> validateColumns(final RequiredColumns requiredColumns) {
    final boolean noSyntheticKey = !finalJoin || !joinKey.isSynthetic();
    final RequiredColumns updated = noSyntheticKey ? requiredColumns : requiredColumns.asBuilder().remove(new UnqualifiedColumnReferenceExp(getOnlyElement(schema.key()).name())).build();
    final Set<ColumnReferenceExp> leftUnknown = left.validateColumns(updated);
    final Set<ColumnReferenceExp> rightUnknown = right.validateColumns(updated);
    return Sets.intersection(leftUnknown, rightUnknown);
}
Also used : ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) QualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.QualifiedColumnReferenceExp) RequiredColumns(io.confluent.ksql.planner.RequiredColumns) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp)

Example 3 with RequiredColumns

use of io.confluent.ksql.planner.RequiredColumns in project ksql by confluentinc.

the class PreJoinProjectNode method validateColumns.

@Override
protected Set<ColumnReferenceExp> validateColumns(final RequiredColumns requiredColumns) {
    final List<? extends ColumnReferenceExp> aliased = requiredColumns.get().stream().filter(columnRef -> columnRef instanceof UnqualifiedColumnReferenceExp).filter(columnRef -> aliases.inverse().containsKey(columnRef.getColumnName())).collect(Collectors.toList());
    final Builder builder = requiredColumns.asBuilder();
    aliased.forEach(columnRef -> {
        builder.remove(columnRef);
        builder.add(new UnqualifiedColumnReferenceExp(columnRef.getLocation(), aliases.inverse().get(columnRef.getColumnName())));
    });
    return super.validateColumns(builder.build());
}
Also used : ColumnNames(io.confluent.ksql.schema.ksql.ColumnNames) ColumnName(io.confluent.ksql.name.ColumnName) SourceName(io.confluent.ksql.name.SourceName) KeyFormat(io.confluent.ksql.serde.KeyFormat) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) Set(java.util.Set) Builder(io.confluent.ksql.planner.RequiredColumns.Builder) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Collectors(java.util.stream.Collectors) Iterators(com.google.common.collect.Iterators) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) Namespace(io.confluent.ksql.schema.ksql.Column.Namespace) ImmutableBiMap(com.google.common.collect.ImmutableBiMap) List(java.util.List) Stream(java.util.stream.Stream) ImmutableList(com.google.common.collect.ImmutableList) Optional(java.util.Optional) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) RequiredColumns(io.confluent.ksql.planner.RequiredColumns) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) Builder(io.confluent.ksql.planner.RequiredColumns.Builder) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp)

Aggregations

ColumnReferenceExp (io.confluent.ksql.execution.expression.tree.ColumnReferenceExp)3 UnqualifiedColumnReferenceExp (io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp)3 RequiredColumns (io.confluent.ksql.planner.RequiredColumns)3 ImmutableList (com.google.common.collect.ImmutableList)2 SuppressFBWarnings (edu.umd.cs.findbugs.annotations.SuppressFBWarnings)2 SelectExpression (io.confluent.ksql.execution.plan.SelectExpression)2 ColumnName (io.confluent.ksql.name.ColumnName)2 SourceName (io.confluent.ksql.name.SourceName)2 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)2 List (java.util.List)2 Optional (java.util.Optional)2 Set (java.util.Set)2 Collectors (java.util.stream.Collectors)2 ImmutableBiMap (com.google.common.collect.ImmutableBiMap)1 Iterators (com.google.common.collect.Iterators)1 Analysis (io.confluent.ksql.analyzer.Analysis)1 QualifiedColumnReferenceExp (io.confluent.ksql.execution.expression.tree.QualifiedColumnReferenceExp)1 AsValue (io.confluent.ksql.function.udf.AsValue)1 MetaStore (io.confluent.ksql.metastore.MetaStore)1 DataSource (io.confluent.ksql.metastore.model.DataSource)1