Search in sources :

Example 1 with SelectExpression

use of io.confluent.ksql.execution.plan.SelectExpression in project ksql by confluentinc.

the class FinalProjectNode method build.

private Pair<LogicalSchema, List<SelectExpression>> build(final MetaStore metaStore, final KsqlConfig ksqlConfig) {
    final LogicalSchema parentSchema = getSource().getSchema();
    final Optional<LogicalSchema> targetSchema = getTargetSchema(metaStore);
    final List<SelectExpression> selectExpressions = SelectionUtil.buildSelectExpressions(getSource(), projection.selectItems(), targetSchema);
    final LogicalSchema schema = SelectionUtil.buildProjectionSchema(parentSchema, selectExpressions, metaStore);
    if (into.isPresent()) {
        // Persistent queries have key columns as value columns - final projection can exclude them:
        final Map<ColumnName, Set<ColumnName>> seenKeyColumns = new HashMap<>();
        selectExpressions.removeIf(se -> {
            if (se.getExpression() instanceof UnqualifiedColumnReferenceExp) {
                final ColumnName columnName = ((UnqualifiedColumnReferenceExp) se.getExpression()).getColumnName();
                // Window bounds columns are currently removed if not aliased:
                if (SystemColumns.isWindowBound(columnName) && se.getAlias().equals(columnName)) {
                    return true;
                }
                if (parentSchema.isKeyColumn(columnName)) {
                    seenKeyColumns.computeIfAbsent(columnName, k -> new HashSet<>()).add(se.getAlias());
                    return true;
                }
            }
            return false;
        });
        for (final Entry<ColumnName, Set<ColumnName>> seenKey : seenKeyColumns.entrySet()) {
            if (seenKey.getValue().size() > 1) {
                final String keys = GrammaticalJoiner.and().join(seenKey.getValue().stream().map(Name::text).sorted());
                throw new KsqlException("The projection contains a key column (" + seenKey.getKey() + ") more than once, aliased as: " + keys + "." + System.lineSeparator() + "Each key column must only be in the projection once. " + "If you intended to copy the key into the value, then consider using the " + AsValue.NAME + " function to indicate which key reference should be copied.");
            }
        }
    }
    final LogicalSchema nodeSchema;
    if (into.isPresent()) {
        nodeSchema = schema.withoutPseudoAndKeyColsInValue(ksqlConfig);
    } else {
        // Transient queries return key columns in the value, so the projection includes them, and
        // the schema needs to include them too:
        final Builder builder = LogicalSchema.builder();
        builder.keyColumns(parentSchema.key());
        schema.columns().forEach(builder::valueColumn);
        nodeSchema = builder.build();
    }
    return Pair.of(nodeSchema, selectExpressions);
}
Also used : DataSource(io.confluent.ksql.metastore.model.DataSource) ColumnName(io.confluent.ksql.name.ColumnName) SourceName(io.confluent.ksql.name.SourceName) GrammaticalJoiner(io.confluent.ksql.util.GrammaticalJoiner) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) Name(io.confluent.ksql.name.Name) HashMap(java.util.HashMap) SelectItem(io.confluent.ksql.parser.tree.SelectItem) HashSet(java.util.HashSet) AsValue(io.confluent.ksql.function.udf.AsValue) Pair(io.confluent.ksql.util.Pair) ImmutableList(com.google.common.collect.ImmutableList) Analysis(io.confluent.ksql.analyzer.Analysis) Map(java.util.Map) MetaStore(io.confluent.ksql.metastore.MetaStore) Projection(io.confluent.ksql.planner.Projection) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) RequiredColumns(io.confluent.ksql.planner.RequiredColumns) SystemColumns(io.confluent.ksql.schema.ksql.SystemColumns) Set(java.util.Set) KsqlConfig(io.confluent.ksql.util.KsqlConfig) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Collectors(java.util.stream.Collectors) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) Builder(io.confluent.ksql.schema.ksql.LogicalSchema.Builder) NodeLocation(io.confluent.ksql.parser.NodeLocation) List(java.util.List) Entry(java.util.Map.Entry) KsqlException(io.confluent.ksql.util.KsqlException) Optional(java.util.Optional) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings) HashSet(java.util.HashSet) Set(java.util.Set) HashMap(java.util.HashMap) Builder(io.confluent.ksql.schema.ksql.LogicalSchema.Builder) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) KsqlException(io.confluent.ksql.util.KsqlException) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) ColumnName(io.confluent.ksql.name.ColumnName) SourceName(io.confluent.ksql.name.SourceName) Name(io.confluent.ksql.name.Name) ColumnName(io.confluent.ksql.name.ColumnName) HashSet(java.util.HashSet)

Example 2 with SelectExpression

use of io.confluent.ksql.execution.plan.SelectExpression in project ksql by confluentinc.

the class SelectionUtil method resolveSelectItem.

private static Stream<SelectExpression> resolveSelectItem(final int idx, final List<? extends SelectItem> selectItems, final PlanNode parentNode, final Optional<LogicalSchema> targetSchema) {
    final SelectItem selectItem = selectItems.get(idx);
    if (selectItem instanceof SingleColumn) {
        final SingleColumn column = (SingleColumn) selectItem;
        // if the column we are trying to coerce into a target schema is beyond
        // the target schema's max columns ignore it. this will generate a failure
        // down the line when we check that the result schema is identical to
        // the schema of the source we are attempting to fit
        final Optional<Column> targetColumn = targetSchema.filter(schema -> schema.columns().size() > idx).map(schema -> schema.columns().get(idx));
        return resolveSingleColumn(idx, parentNode, column, targetColumn);
    }
    if (selectItem instanceof AllColumns) {
        return resolveAllColumns(parentNode, (AllColumns) selectItem);
    }
    throw new IllegalArgumentException("Unsupported SelectItem type: " + selectItem.getClass().getName());
}
Also used : IntStream(java.util.stream.IntStream) Expression(io.confluent.ksql.execution.expression.tree.Expression) ColumnName(io.confluent.ksql.name.ColumnName) FunctionRegistry(io.confluent.ksql.function.FunctionRegistry) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) Set(java.util.Set) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Collectors(java.util.stream.Collectors) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) SelectItem(io.confluent.ksql.parser.tree.SelectItem) Namespace(io.confluent.ksql.schema.ksql.Column.Namespace) Builder(io.confluent.ksql.schema.ksql.LogicalSchema.Builder) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) List(java.util.List) SingleColumn(io.confluent.ksql.parser.tree.SingleColumn) Stream(java.util.stream.Stream) ExpressionTypeManager(io.confluent.ksql.execution.util.ExpressionTypeManager) Optional(java.util.Optional) AllColumns(io.confluent.ksql.parser.tree.AllColumns) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) Column(io.confluent.ksql.schema.ksql.Column) SqlType(io.confluent.ksql.schema.ksql.types.SqlType) SingleColumn(io.confluent.ksql.parser.tree.SingleColumn) Column(io.confluent.ksql.schema.ksql.Column) SelectItem(io.confluent.ksql.parser.tree.SelectItem) SingleColumn(io.confluent.ksql.parser.tree.SingleColumn) AllColumns(io.confluent.ksql.parser.tree.AllColumns)

Example 3 with SelectExpression

use of io.confluent.ksql.execution.plan.SelectExpression in project ksql by confluentinc.

the class SelectionUtil method resolveSingleColumn.

private static Stream<SelectExpression> resolveSingleColumn(final int idx, final PlanNode parentNode, final SingleColumn column, final Optional<Column> targetColumn) {
    final Expression expression = parentNode.resolveSelect(idx, column.getExpression());
    final ColumnName alias = column.getAlias().orElseThrow(() -> new IllegalStateException("Alias should be present by this point"));
    return Stream.of(SelectExpression.of(alias, targetColumn.map(col -> ImplicitlyCastResolver.resolve(expression, col.type())).orElse(expression)));
}
Also used : IntStream(java.util.stream.IntStream) Expression(io.confluent.ksql.execution.expression.tree.Expression) ColumnName(io.confluent.ksql.name.ColumnName) FunctionRegistry(io.confluent.ksql.function.FunctionRegistry) UnqualifiedColumnReferenceExp(io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp) Set(java.util.Set) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Collectors(java.util.stream.Collectors) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) SelectItem(io.confluent.ksql.parser.tree.SelectItem) Namespace(io.confluent.ksql.schema.ksql.Column.Namespace) Builder(io.confluent.ksql.schema.ksql.LogicalSchema.Builder) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) List(java.util.List) SingleColumn(io.confluent.ksql.parser.tree.SingleColumn) Stream(java.util.stream.Stream) ExpressionTypeManager(io.confluent.ksql.execution.util.ExpressionTypeManager) Optional(java.util.Optional) AllColumns(io.confluent.ksql.parser.tree.AllColumns) ColumnReferenceExp(io.confluent.ksql.execution.expression.tree.ColumnReferenceExp) Column(io.confluent.ksql.schema.ksql.Column) SqlType(io.confluent.ksql.schema.ksql.types.SqlType) ColumnName(io.confluent.ksql.name.ColumnName) Expression(io.confluent.ksql.execution.expression.tree.Expression) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression)

Example 4 with SelectExpression

use of io.confluent.ksql.execution.plan.SelectExpression in project ksql by confluentinc.

the class ProjectOperator method createRowForSelectStar.

// Optimization for select star, to avoid having to do code generation
private List<?> createRowForSelectStar(final GenericRow intermediate) {
    final List<Object> rowList = new ArrayList<>();
    for (SelectExpression selectExpression : logicalNode.getSelectExpressions()) {
        final Optional<Column> column = logicalNode.getIntermediateSchema().findValueColumn(selectExpression.getAlias());
        if (!column.isPresent()) {
            throw new IllegalStateException("Couldn't find alias in intermediate schema " + selectExpression.getAlias());
        }
        final int i = column.get().index();
        rowList.add(intermediate.get(i));
    }
    return rowList;
}
Also used : Column(io.confluent.ksql.schema.ksql.Column) ArrayList(java.util.ArrayList) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression)

Example 5 with SelectExpression

use of io.confluent.ksql.execution.plan.SelectExpression in project ksql by confluentinc.

the class ExpressionParserTest method shouldParseSelectExpression.

@Test
public void shouldParseSelectExpression() {
    // When:
    final SelectExpression parsed = ExpressionParser.parseSelectExpression("1 + 2 AS `three`");
    // Then:
    assertThat(parsed, equalTo(SelectExpression.of(ColumnName.of("three"), new ArithmeticBinaryExpression(parsed.getExpression().getLocation(), Operator.ADD, ONE, TWO))));
}
Also used : ArithmeticBinaryExpression(io.confluent.ksql.execution.expression.tree.ArithmeticBinaryExpression) SelectExpression(io.confluent.ksql.execution.plan.SelectExpression) ExpressionParser.parseSelectExpression(io.confluent.ksql.parser.ExpressionParser.parseSelectExpression) Test(org.junit.Test)

Aggregations

SelectExpression (io.confluent.ksql.execution.plan.SelectExpression)12 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)9 Builder (io.confluent.ksql.schema.ksql.LogicalSchema.Builder)7 ExpressionTypeManager (io.confluent.ksql.execution.util.ExpressionTypeManager)6 SqlType (io.confluent.ksql.schema.ksql.types.SqlType)6 List (java.util.List)6 Optional (java.util.Optional)6 Collectors (java.util.stream.Collectors)6 ColumnReferenceExp (io.confluent.ksql.execution.expression.tree.ColumnReferenceExp)5 Expression (io.confluent.ksql.execution.expression.tree.Expression)5 UnqualifiedColumnReferenceExp (io.confluent.ksql.execution.expression.tree.UnqualifiedColumnReferenceExp)5 ColumnName (io.confluent.ksql.name.ColumnName)5 Column (io.confluent.ksql.schema.ksql.Column)5 Set (java.util.Set)5 SelectItem (io.confluent.ksql.parser.tree.SelectItem)4 AllColumns (io.confluent.ksql.parser.tree.AllColumns)3 SingleColumn (io.confluent.ksql.parser.tree.SingleColumn)3 ArrayList (java.util.ArrayList)3 HashSet (java.util.HashSet)3 ImmutableList (com.google.common.collect.ImmutableList)2