Search in sources :

Example 6 with Stacker

use of io.confluent.ksql.execution.context.QueryContext.Stacker in project ksql by confluentinc.

the class AggregateNode method aggregate.

private SchemaKTable<?> aggregate(final SchemaKGroupedStream grouped, final InternalSchema internalSchema, final Stacker contextStacker) {
    final List<FunctionCall> functions = internalSchema.updateFunctionList(functionList);
    final Stacker aggregationContext = contextStacker.push(AGGREGATION_OP_NAME);
    final List<ColumnName> requiredColumnNames = requiredColumns.stream().map(e -> (UnqualifiedColumnReferenceExp) internalSchema.resolveToInternal(e)).map(UnqualifiedColumnReferenceExp::getColumnName).collect(Collectors.toList());
    return grouped.aggregate(requiredColumnNames, functions, windowExpression, valueFormat.getFormatInfo(), aggregationContext);
}
Also used : ColumnName(io.confluent.ksql.name.ColumnName) Stacker(io.confluent.ksql.execution.context.QueryContext.Stacker) FunctionCall(io.confluent.ksql.execution.expression.tree.FunctionCall)

Example 7 with Stacker

use of io.confluent.ksql.execution.context.QueryContext.Stacker in project ksql by confluentinc.

the class SchemaKSourceFactory method buildTable.

private static SchemaKTable<?> buildTable(final PlanBuildContext buildContext, final DataSource dataSource, final Stacker contextStacker) {
    final KeyFormat keyFormat = dataSource.getKsqlTopic().getKeyFormat();
    if (keyFormat.isWindowed()) {
        throw new IllegalArgumentException("windowed");
    }
    final SourceStep<KTableHolder<GenericKey>> step;
    final int pseudoColumnVersionToUse = determinePseudoColumnVersionToUse(buildContext);
    // If the old query has a v1 table step, continue to use it.
    // See https://github.com/confluentinc/ksql/pull/7990
    boolean useOldExecutionStepVersion = false;
    if (buildContext.getPlanInfo().isPresent()) {
        final Set<ExecutionStep<?>> sourceSteps = buildContext.getPlanInfo().get().getSources();
        useOldExecutionStepVersion = sourceSteps.stream().anyMatch(executionStep -> executionStep instanceof TableSourceV1);
    }
    if (useOldExecutionStepVersion && pseudoColumnVersionToUse != SystemColumns.LEGACY_PSEUDOCOLUMN_VERSION_NUMBER) {
        throw new IllegalStateException("TableSourceV2 was released in conjunction with pseudocolumn" + "version 1. Something has gone very wrong");
    }
    if (buildContext.getKsqlConfig().getBoolean(KsqlConfig.KSQL_ROWPARTITION_ROWOFFSET_ENABLED) && !useOldExecutionStepVersion) {
        step = ExecutionStepFactory.tableSource(contextStacker, dataSource.getSchema(), dataSource.getKafkaTopicName(), Formats.from(dataSource.getKsqlTopic()), dataSource.getTimestampColumn(), InternalFormats.of(keyFormat, Formats.from(dataSource.getKsqlTopic()).getValueFormat()), pseudoColumnVersionToUse);
    } else {
        step = ExecutionStepFactory.tableSourceV1(contextStacker, dataSource.getSchema(), dataSource.getKafkaTopicName(), Formats.from(dataSource.getKsqlTopic()), dataSource.getTimestampColumn(), pseudoColumnVersionToUse);
    }
    return schemaKTable(buildContext, resolveSchema(buildContext, step, dataSource), dataSource.getKsqlTopic().getKeyFormat(), step);
}
Also used : ExecutionStep(io.confluent.ksql.execution.plan.ExecutionStep) DataSource(io.confluent.ksql.metastore.model.DataSource) ExecutionStep(io.confluent.ksql.execution.plan.ExecutionStep) SystemColumns(io.confluent.ksql.schema.ksql.SystemColumns) PlanBuildContext(io.confluent.ksql.planner.plan.PlanBuildContext) KeyFormat(io.confluent.ksql.serde.KeyFormat) QueryContext(io.confluent.ksql.execution.context.QueryContext) Set(java.util.Set) KsqlConfig(io.confluent.ksql.util.KsqlConfig) Formats(io.confluent.ksql.execution.plan.Formats) InternalFormats(io.confluent.ksql.serde.InternalFormats) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) StreamSource(io.confluent.ksql.execution.plan.StreamSource) StepSchemaResolver(io.confluent.ksql.execution.streams.StepSchemaResolver) WindowInfo(io.confluent.ksql.serde.WindowInfo) KStreamHolder(io.confluent.ksql.execution.plan.KStreamHolder) KTableHolder(io.confluent.ksql.execution.plan.KTableHolder) SourceStep(io.confluent.ksql.execution.plan.SourceStep) WindowedStreamSource(io.confluent.ksql.execution.plan.WindowedStreamSource) TableSourceV1(io.confluent.ksql.execution.plan.TableSourceV1) Windowed(org.apache.kafka.streams.kstream.Windowed) Stacker(io.confluent.ksql.execution.context.QueryContext.Stacker) GenericKey(io.confluent.ksql.GenericKey) ExecutionStepFactory(io.confluent.ksql.execution.streams.ExecutionStepFactory) TableSourceV1(io.confluent.ksql.execution.plan.TableSourceV1) KTableHolder(io.confluent.ksql.execution.plan.KTableHolder) KeyFormat(io.confluent.ksql.serde.KeyFormat)

Example 8 with Stacker

use of io.confluent.ksql.execution.context.QueryContext.Stacker in project ksql by confluentinc.

the class PlanBuildContextTest method shouldBuildNodeContext.

@Test
public void shouldBuildNodeContext() {
    // When:
    final Stacker result = runtimeBuildContext.buildNodeContext("some-id");
    // Then:
    assertThat(result, is(new Stacker().push("some-id")));
}
Also used : Stacker(io.confluent.ksql.execution.context.QueryContext.Stacker) Test(org.junit.Test)

Example 9 with Stacker

use of io.confluent.ksql.execution.context.QueryContext.Stacker in project ksql by confluentinc.

the class KsqlMaterializationFactoryTest method shouldUseCorrectLoggerForPredicate.

@Test
public void shouldUseCorrectLoggerForPredicate() {
    // When:
    factory.create(materialization, info, queryId, new Stacker().push("filter"));
    // Then:
    verify(predicateInfo).getPredicate(loggerCaptor.capture());
    assertThat(loggerCaptor.getValue().apply(new Stacker().getQueryContext()), is(filterProcessingLogger));
}
Also used : Stacker(io.confluent.ksql.execution.context.QueryContext.Stacker) Test(org.junit.Test)

Aggregations

Stacker (io.confluent.ksql.execution.context.QueryContext.Stacker)9 KTableHolder (io.confluent.ksql.execution.plan.KTableHolder)3 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)3 Optional (java.util.Optional)3 Test (org.junit.Test)3 GenericKey (io.confluent.ksql.GenericKey)2 GenericRow (io.confluent.ksql.GenericRow)2 QueryContext (io.confluent.ksql.execution.context.QueryContext)2 FunctionCall (io.confluent.ksql.execution.expression.tree.FunctionCall)2 ExecutionStep (io.confluent.ksql.execution.plan.ExecutionStep)2 Formats (io.confluent.ksql.execution.plan.Formats)2 RuntimeBuildContext (io.confluent.ksql.execution.runtime.RuntimeBuildContext)2 ExecutionStepFactory (io.confluent.ksql.execution.streams.ExecutionStepFactory)2 KsTransformer (io.confluent.ksql.execution.streams.transform.KsTransformer)2 ProcessingLogger (io.confluent.ksql.logging.processing.ProcessingLogger)2 ColumnName (io.confluent.ksql.name.ColumnName)2 InternalFormats (io.confluent.ksql.serde.InternalFormats)2 KeyFormat (io.confluent.ksql.serde.KeyFormat)2 KsqlConfig (io.confluent.ksql.util.KsqlConfig)2 KTable (org.apache.kafka.streams.kstream.KTable)2