Search in sources :

Example 1 with StreamSelectKeyV1

use of io.confluent.ksql.execution.plan.StreamSelectKeyV1 in project ksql by confluentinc.

the class StreamSelectKeyBuilderV1 method build.

public static KStreamHolder<GenericKey> build(final KStreamHolder<?> stream, final StreamSelectKeyV1 selectKey, final RuntimeBuildContext buildContext) {
    final LogicalSchema sourceSchema = stream.getSchema();
    final CompiledExpression expression = buildExpressionEvaluator(selectKey, buildContext, sourceSchema);
    final ProcessingLogger processingLogger = buildContext.getProcessingLogger(selectKey.getProperties().getQueryContext());
    final String errorMsg = "Error extracting new key using expression " + selectKey.getKeyExpression();
    final Function<GenericRow, Object> evaluator = val -> expression.evaluate(val, null, processingLogger, () -> errorMsg);
    final LogicalSchema resultSchema = new StepSchemaResolver(buildContext.getKsqlConfig(), buildContext.getFunctionRegistry()).resolve(selectKey, sourceSchema);
    final KStream<?, GenericRow> kstream = stream.getStream();
    final KStream<GenericKey, GenericRow> rekeyed = kstream.filter((key, val) -> val != null && evaluator.apply(val) != null).selectKey((key, val) -> GenericKey.genericKey(evaluator.apply(val)));
    return new KStreamHolder<>(rekeyed, resultSchema, ExecutionKeyFactory.unwindowed(buildContext));
}
Also used : KStreamHolder(io.confluent.ksql.execution.plan.KStreamHolder) StreamSelectKeyV1(io.confluent.ksql.execution.plan.StreamSelectKeyV1) RuntimeBuildContext(io.confluent.ksql.execution.runtime.RuntimeBuildContext) GenericRow(io.confluent.ksql.GenericRow) ProcessingLogger(io.confluent.ksql.logging.processing.ProcessingLogger) CompiledExpression(io.confluent.ksql.execution.codegen.CompiledExpression) GenericKey(io.confluent.ksql.GenericKey) KStream(org.apache.kafka.streams.kstream.KStream) ExecutionKeyFactory(io.confluent.ksql.execution.plan.ExecutionKeyFactory) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) Function(java.util.function.Function) CodeGenRunner(io.confluent.ksql.execution.codegen.CodeGenRunner) ProcessingLogger(io.confluent.ksql.logging.processing.ProcessingLogger) KStreamHolder(io.confluent.ksql.execution.plan.KStreamHolder) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) CompiledExpression(io.confluent.ksql.execution.codegen.CompiledExpression) GenericRow(io.confluent.ksql.GenericRow) GenericKey(io.confluent.ksql.GenericKey)

Example 2 with StreamSelectKeyV1

use of io.confluent.ksql.execution.plan.StreamSelectKeyV1 in project ksql by confluentinc.

the class StreamSelectKeyBuilderV1Test method init.

@Before
@SuppressWarnings({ "unchecked", "rawtypes" })
public void init() {
    when(buildContext.getFunctionRegistry()).thenReturn(functionRegistry);
    when(buildContext.getKsqlConfig()).thenReturn(new KsqlConfig(ImmutableMap.of()));
    when(kstream.filter(any())).thenReturn(filteredKStream);
    when(filteredKStream.selectKey(any(KeyValueMapper.class))).thenReturn(rekeyedKstream);
    when(sourceStep.build(any(), eq(planInfo))).thenReturn(new KStreamHolder<>(kstream, SOURCE_SCHEMA, mock(ExecutionKeyFactory.class)));
    planBuilder = new KSPlanBuilder(buildContext, mock(SqlPredicateFactory.class), mock(AggregateParamsFactory.class), mock(StreamsFactories.class));
    selectKey = new StreamSelectKeyV1(new ExecutionStepPropertiesV1(queryContext), sourceStep, KEY);
}
Also used : StreamSelectKeyV1(io.confluent.ksql.execution.plan.StreamSelectKeyV1) KsqlConfig(io.confluent.ksql.util.KsqlConfig) ExecutionStepPropertiesV1(io.confluent.ksql.execution.plan.ExecutionStepPropertiesV1) KeyValueMapper(org.apache.kafka.streams.kstream.KeyValueMapper) Before(org.junit.Before)

Aggregations

StreamSelectKeyV1 (io.confluent.ksql.execution.plan.StreamSelectKeyV1)2 GenericKey (io.confluent.ksql.GenericKey)1 GenericRow (io.confluent.ksql.GenericRow)1 CodeGenRunner (io.confluent.ksql.execution.codegen.CodeGenRunner)1 CompiledExpression (io.confluent.ksql.execution.codegen.CompiledExpression)1 ExecutionKeyFactory (io.confluent.ksql.execution.plan.ExecutionKeyFactory)1 ExecutionStepPropertiesV1 (io.confluent.ksql.execution.plan.ExecutionStepPropertiesV1)1 KStreamHolder (io.confluent.ksql.execution.plan.KStreamHolder)1 RuntimeBuildContext (io.confluent.ksql.execution.runtime.RuntimeBuildContext)1 ProcessingLogger (io.confluent.ksql.logging.processing.ProcessingLogger)1 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)1 KsqlConfig (io.confluent.ksql.util.KsqlConfig)1 Function (java.util.function.Function)1 KStream (org.apache.kafka.streams.kstream.KStream)1 KeyValueMapper (org.apache.kafka.streams.kstream.KeyValueMapper)1 Before (org.junit.Before)1