Search in sources :

Example 11 with Expression

use of io.confluent.ksql.parser.tree.Expression in project ksql by confluentinc.

the class AstBuilder method visitSelectSingle.

@Override
public Node visitSelectSingle(SqlBaseParser.SelectSingleContext context) {
    Expression selectItemExpression = (Expression) visit(context.expression());
    Optional<String> alias = Optional.ofNullable(context.identifier()).map(AstBuilder::getIdentifierText);
    if (!alias.isPresent()) {
        if (selectItemExpression instanceof QualifiedNameReference) {
            QualifiedNameReference qualifiedNameReference = (QualifiedNameReference) selectItemExpression;
            alias = Optional.of(qualifiedNameReference.getName().getSuffix());
        } else if (selectItemExpression instanceof DereferenceExpression) {
            DereferenceExpression dereferenceExpression = (DereferenceExpression) selectItemExpression;
            if ((dataSourceExtractor.getJoinLeftSchema() != null) && (dataSourceExtractor.getCommonFieldNames().contains(dereferenceExpression.getFieldName()))) {
                alias = Optional.of(dereferenceExpression.getBase().toString() + "_" + dereferenceExpression.getFieldName());
            } else {
                alias = Optional.of(dereferenceExpression.getFieldName());
            }
        } else {
            alias = Optional.of("KSQL_COL_" + selectItemIndex);
        }
    } else {
        alias = Optional.of(alias.get());
    }
    selectItemIndex++;
    return new SingleColumn(getLocation(context), selectItemExpression, alias);
}
Also used : DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) InListExpression(io.confluent.ksql.parser.tree.InListExpression) NullIfExpression(io.confluent.ksql.parser.tree.NullIfExpression) SimpleCaseExpression(io.confluent.ksql.parser.tree.SimpleCaseExpression) ComparisonExpression(io.confluent.ksql.parser.tree.ComparisonExpression) DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) Expression(io.confluent.ksql.parser.tree.Expression) LogicalBinaryExpression(io.confluent.ksql.parser.tree.LogicalBinaryExpression) TumblingWindowExpression(io.confluent.ksql.parser.tree.TumblingWindowExpression) ArithmeticBinaryExpression(io.confluent.ksql.parser.tree.ArithmeticBinaryExpression) NotExpression(io.confluent.ksql.parser.tree.NotExpression) HoppingWindowExpression(io.confluent.ksql.parser.tree.HoppingWindowExpression) SubscriptExpression(io.confluent.ksql.parser.tree.SubscriptExpression) SessionWindowExpression(io.confluent.ksql.parser.tree.SessionWindowExpression) SearchedCaseExpression(io.confluent.ksql.parser.tree.SearchedCaseExpression) LambdaExpression(io.confluent.ksql.parser.tree.LambdaExpression) SubqueryExpression(io.confluent.ksql.parser.tree.SubqueryExpression) WindowExpression(io.confluent.ksql.parser.tree.WindowExpression) ArithmeticUnaryExpression(io.confluent.ksql.parser.tree.ArithmeticUnaryExpression) SingleColumn(io.confluent.ksql.parser.tree.SingleColumn) QualifiedNameReference(io.confluent.ksql.parser.tree.QualifiedNameReference)

Example 12 with Expression

use of io.confluent.ksql.parser.tree.Expression in project ksql by confluentinc.

the class AstBuilder method visitFunctionCall.

@Override
public Node visitFunctionCall(SqlBaseParser.FunctionCallContext context) {
    Optional<Window> window = visitIfPresent(context.over(), Window.class);
    QualifiedName name = getQualifiedName(context.qualifiedName());
    boolean distinct = false;
    if (name.toString().equals("NULLIF")) {
        check(context.expression().size() == 2, "Invalid number of arguments for 'nullif' function", context);
        check(!window.isPresent(), "OVER clause not valid for 'nullif' function", context);
        check(!distinct, "DISTINCT not valid for 'nullif' function", context);
        return new NullIfExpression(getLocation(context), (Expression) visit(context.expression(0)), (Expression) visit(context.expression(1)));
    }
    return new FunctionCall(getLocation(context), getQualifiedName(context.qualifiedName()), window, distinct, visit(context.expression(), Expression.class));
}
Also used : Window(io.confluent.ksql.parser.tree.Window) InListExpression(io.confluent.ksql.parser.tree.InListExpression) NullIfExpression(io.confluent.ksql.parser.tree.NullIfExpression) SimpleCaseExpression(io.confluent.ksql.parser.tree.SimpleCaseExpression) ComparisonExpression(io.confluent.ksql.parser.tree.ComparisonExpression) DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) Expression(io.confluent.ksql.parser.tree.Expression) LogicalBinaryExpression(io.confluent.ksql.parser.tree.LogicalBinaryExpression) TumblingWindowExpression(io.confluent.ksql.parser.tree.TumblingWindowExpression) ArithmeticBinaryExpression(io.confluent.ksql.parser.tree.ArithmeticBinaryExpression) NotExpression(io.confluent.ksql.parser.tree.NotExpression) HoppingWindowExpression(io.confluent.ksql.parser.tree.HoppingWindowExpression) SubscriptExpression(io.confluent.ksql.parser.tree.SubscriptExpression) SessionWindowExpression(io.confluent.ksql.parser.tree.SessionWindowExpression) SearchedCaseExpression(io.confluent.ksql.parser.tree.SearchedCaseExpression) LambdaExpression(io.confluent.ksql.parser.tree.LambdaExpression) SubqueryExpression(io.confluent.ksql.parser.tree.SubqueryExpression) WindowExpression(io.confluent.ksql.parser.tree.WindowExpression) ArithmeticUnaryExpression(io.confluent.ksql.parser.tree.ArithmeticUnaryExpression) QualifiedName(io.confluent.ksql.parser.tree.QualifiedName) NullIfExpression(io.confluent.ksql.parser.tree.NullIfExpression) FunctionCall(io.confluent.ksql.parser.tree.FunctionCall)

Example 13 with Expression

use of io.confluent.ksql.parser.tree.Expression in project ksql by confluentinc.

the class KsqlRestApplication method buildApplication.

public static KsqlRestApplication buildApplication(KsqlRestConfig restConfig, boolean isUiEnabled, VersionCheckerAgent versionCheckerAgent) throws Exception {
    Map<String, Object> ksqlConfProperties = new HashMap<>();
    ksqlConfProperties.putAll(restConfig.getKsqlConfigProperties());
    KsqlConfig ksqlConfig = new KsqlConfig(ksqlConfProperties);
    adminClient = AdminClient.create(ksqlConfig.getKsqlAdminClientConfigProps());
    KsqlEngine ksqlEngine = new KsqlEngine(ksqlConfig, new KafkaTopicClientImpl(adminClient));
    KafkaTopicClient topicClient = ksqlEngine.getTopicClient();
    final String kafkaClusterId;
    try {
        kafkaClusterId = adminClient.describeCluster().clusterId().get();
    } catch (final UnsupportedVersionException e) {
        throw new KsqlException("The kafka brokers are incompatible with. " + "KSQL requires broker versions >= 0.10.1.x");
    }
    String commandTopic = restConfig.getCommandTopic(ksqlConfig.getString(KsqlConfig.KSQL_SERVICE_ID_CONFIG));
    ensureCommandTopic(restConfig, topicClient, commandTopic);
    Map<String, Expression> commandTopicProperties = new HashMap<>();
    commandTopicProperties.put(DdlConfig.VALUE_FORMAT_PROPERTY, new StringLiteral("json"));
    commandTopicProperties.put(DdlConfig.KAFKA_TOPIC_NAME_PROPERTY, new StringLiteral(commandTopic));
    ksqlEngine.getDdlCommandExec().execute(new RegisterTopicCommand(new RegisterTopic(QualifiedName.of(COMMANDS_KSQL_TOPIC_NAME), false, commandTopicProperties)));
    ksqlEngine.getDdlCommandExec().execute(new CreateStreamCommand("statementText", new CreateStream(QualifiedName.of(COMMANDS_STREAM_NAME), Collections.singletonList(new TableElement("STATEMENT", "STRING")), false, Collections.singletonMap(DdlConfig.TOPIC_NAME_PROPERTY, new StringLiteral(COMMANDS_KSQL_TOPIC_NAME))), Collections.emptyMap(), ksqlEngine.getTopicClient(), true));
    Map<String, Object> commandConsumerProperties = restConfig.getCommandConsumerProperties();
    KafkaConsumer<CommandId, Command> commandConsumer = new KafkaConsumer<>(commandConsumerProperties, getJsonDeserializer(CommandId.class, true), getJsonDeserializer(Command.class, false));
    KafkaProducer<CommandId, Command> commandProducer = new KafkaProducer<>(restConfig.getCommandProducerProperties(), getJsonSerializer(true), getJsonSerializer(false));
    CommandStore commandStore = new CommandStore(commandTopic, commandConsumer, commandProducer, new CommandIdAssigner(ksqlEngine.getMetaStore()));
    StatementParser statementParser = new StatementParser(ksqlEngine);
    StatementExecutor statementExecutor = new StatementExecutor(ksqlEngine, statementParser);
    CommandRunner commandRunner = new CommandRunner(statementExecutor, commandStore);
    RootDocument rootDocument = new RootDocument(isUiEnabled, restConfig.getList(RestConfig.LISTENERS_CONFIG).get(0));
    StatusResource statusResource = new StatusResource(statementExecutor);
    StreamedQueryResource streamedQueryResource = new StreamedQueryResource(ksqlEngine, statementParser, restConfig.getLong(KsqlRestConfig.STREAMED_QUERY_DISCONNECT_CHECK_MS_CONFIG));
    KsqlResource ksqlResource = new KsqlResource(ksqlEngine, commandStore, statementExecutor, restConfig.getLong(KsqlRestConfig.DISTRIBUTED_COMMAND_RESPONSE_TIMEOUT_MS_CONFIG));
    commandRunner.processPriorCommands();
    return new KsqlRestApplication(ksqlEngine, restConfig, commandRunner, rootDocument, statusResource, streamedQueryResource, ksqlResource, isUiEnabled, versionCheckerAgent, new ServerInfo(Version.getVersion(), kafkaClusterId));
}
Also used : KafkaProducer(org.apache.kafka.clients.producer.KafkaProducer) KsqlEngine(io.confluent.ksql.KsqlEngine) HashMap(java.util.HashMap) RegisterTopic(io.confluent.ksql.parser.tree.RegisterTopic) KsqlResource(io.confluent.ksql.rest.server.resources.KsqlResource) ServerInfo(io.confluent.ksql.rest.entity.ServerInfo) CommandStore(io.confluent.ksql.rest.server.computation.CommandStore) KsqlException(io.confluent.ksql.util.KsqlException) StatementExecutor(io.confluent.ksql.rest.server.computation.StatementExecutor) StatusResource(io.confluent.ksql.rest.server.resources.StatusResource) TableElement(io.confluent.ksql.parser.tree.TableElement) StreamedQueryResource(io.confluent.ksql.rest.server.resources.streaming.StreamedQueryResource) CreateStreamCommand(io.confluent.ksql.ddl.commands.CreateStreamCommand) CommandRunner(io.confluent.ksql.rest.server.computation.CommandRunner) KafkaTopicClientImpl(io.confluent.ksql.util.KafkaTopicClientImpl) RootDocument(io.confluent.ksql.rest.server.resources.RootDocument) KsqlConfig(io.confluent.ksql.util.KsqlConfig) CreateStream(io.confluent.ksql.parser.tree.CreateStream) KafkaConsumer(org.apache.kafka.clients.consumer.KafkaConsumer) RegisterTopicCommand(io.confluent.ksql.ddl.commands.RegisterTopicCommand) KafkaTopicClient(io.confluent.ksql.util.KafkaTopicClient) StringLiteral(io.confluent.ksql.parser.tree.StringLiteral) Expression(io.confluent.ksql.parser.tree.Expression) Command(io.confluent.ksql.rest.server.computation.Command) RegisterTopicCommand(io.confluent.ksql.ddl.commands.RegisterTopicCommand) CreateStreamCommand(io.confluent.ksql.ddl.commands.CreateStreamCommand) CommandIdAssigner(io.confluent.ksql.rest.server.computation.CommandIdAssigner) CommandId(io.confluent.ksql.rest.server.computation.CommandId) UnsupportedVersionException(org.apache.kafka.common.errors.UnsupportedVersionException)

Example 14 with Expression

use of io.confluent.ksql.parser.tree.Expression in project ksql by confluentinc.

the class AggregateAnalyzer method visitFunctionCall.

@Override
protected Node visitFunctionCall(final FunctionCall node, final AnalysisContext context) {
    String functionName = node.getName().getSuffix();
    if (functionRegistry.isAnAggregateFunction(functionName)) {
        if (node.getArguments().isEmpty()) {
            Expression argExpression;
            if (analysis.getJoin() != null) {
                Expression baseExpression = new QualifiedNameReference(QualifiedName.of(analysis.getJoin().getLeftAlias()));
                argExpression = new DereferenceExpression(baseExpression, SchemaUtil.ROWTIME_NAME);
            } else {
                Expression baseExpression = new QualifiedNameReference(QualifiedName.of(analysis.getFromDataSources().get(0).getRight()));
                argExpression = new DereferenceExpression(baseExpression, SchemaUtil.ROWTIME_NAME);
            }
            aggregateAnalysis.addAggregateFunctionArgument(argExpression);
            node.getArguments().add(argExpression);
        } else {
            aggregateAnalysis.addAggregateFunctionArgument(node.getArguments().get(0));
        }
        aggregateAnalysis.addFunction(node);
        hasAggregateFunction = true;
    }
    for (Expression argExp : node.getArguments()) {
        process(argExp, context);
    }
    return null;
}
Also used : DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) Expression(io.confluent.ksql.parser.tree.Expression) QualifiedNameReference(io.confluent.ksql.parser.tree.QualifiedNameReference)

Example 15 with Expression

use of io.confluent.ksql.parser.tree.Expression in project ksql by confluentinc.

the class Analyzer method analyzeGroupBy.

private void analyzeGroupBy(final GroupBy groupBy) {
    for (GroupingElement groupingElement : groupBy.getGroupingElements()) {
        Set<Expression> groupingSet = groupingElement.enumerateGroupingSets().get(0);
        analysis.getGroupByExpressions().addAll(groupingSet);
    }
}
Also used : GroupingElement(io.confluent.ksql.parser.tree.GroupingElement) ComparisonExpression(io.confluent.ksql.parser.tree.ComparisonExpression) DereferenceExpression(io.confluent.ksql.parser.tree.DereferenceExpression) WindowExpression(io.confluent.ksql.parser.tree.WindowExpression) Expression(io.confluent.ksql.parser.tree.Expression)

Aggregations

Expression (io.confluent.ksql.parser.tree.Expression)36 DereferenceExpression (io.confluent.ksql.parser.tree.DereferenceExpression)17 ComparisonExpression (io.confluent.ksql.parser.tree.ComparisonExpression)13 WindowExpression (io.confluent.ksql.parser.tree.WindowExpression)10 HashMap (java.util.HashMap)10 ArithmeticBinaryExpression (io.confluent.ksql.parser.tree.ArithmeticBinaryExpression)8 ArithmeticUnaryExpression (io.confluent.ksql.parser.tree.ArithmeticUnaryExpression)8 InListExpression (io.confluent.ksql.parser.tree.InListExpression)8 LogicalBinaryExpression (io.confluent.ksql.parser.tree.LogicalBinaryExpression)8 NotExpression (io.confluent.ksql.parser.tree.NotExpression)8 NullIfExpression (io.confluent.ksql.parser.tree.NullIfExpression)8 QualifiedNameReference (io.confluent.ksql.parser.tree.QualifiedNameReference)8 SearchedCaseExpression (io.confluent.ksql.parser.tree.SearchedCaseExpression)8 SimpleCaseExpression (io.confluent.ksql.parser.tree.SimpleCaseExpression)8 StringLiteral (io.confluent.ksql.parser.tree.StringLiteral)8 SubqueryExpression (io.confluent.ksql.parser.tree.SubqueryExpression)8 SubscriptExpression (io.confluent.ksql.parser.tree.SubscriptExpression)8 ArrayList (java.util.ArrayList)8 Test (org.junit.Test)8 HoppingWindowExpression (io.confluent.ksql.parser.tree.HoppingWindowExpression)7