Search in sources :

Example 6 with SerdeFeatures

use of io.confluent.ksql.serde.SerdeFeatures in project ksql by confluentinc.

the class DefaultSchemaInjector method getCreateAsKeySchema.

private Optional<SchemaAndId> getCreateAsKeySchema(final ConfiguredStatement<CreateAsSelect> statement, final CreateSourceCommand createSourceCommand) {
    final CreateAsSelect csStmt = statement.getStatement();
    final CreateSourceAsProperties props = csStmt.getProperties();
    final FormatInfo keyFormat = createSourceCommand.getFormats().getKeyFormat();
    if (!shouldInferSchema(props.getKeySchemaId(), statement, keyFormat, true)) {
        return Optional.empty();
    }
    // until we support user-configuration of single key wrapping/unwrapping, we choose
    // to have key schema inference always result in an unwrapped key
    final SerdeFeatures serdeFeatures = SerdeFeaturesFactory.buildKeyFeatures(FormatFactory.of(keyFormat), true);
    if (!shouldInferSchema(props.getKeySchemaId(), statement, keyFormat, true)) {
        return Optional.empty();
    }
    final SchemaAndId schemaAndId = getSchema(props.getKafkaTopic(), props.getKeySchemaId(), keyFormat, serdeFeatures, statement.getStatementText(), true);
    final List<Column> tableColumns = createSourceCommand.getSchema().key();
    checkColumnsCompatibility(props.getKeySchemaId(), tableColumns, schemaAndId.columns, true);
    return Optional.of(schemaAndId);
}
Also used : SchemaAndId(io.confluent.ksql.schema.ksql.inference.TopicSchemaSupplier.SchemaAndId) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn) Column(io.confluent.ksql.schema.ksql.Column) CreateSourceAsProperties(io.confluent.ksql.parser.properties.with.CreateSourceAsProperties) CreateAsSelect(io.confluent.ksql.parser.tree.CreateAsSelect) FormatInfo(io.confluent.ksql.serde.FormatInfo) SerdeFeatures(io.confluent.ksql.serde.SerdeFeatures)

Example 7 with SerdeFeatures

use of io.confluent.ksql.serde.SerdeFeatures in project ksql by confluentinc.

the class SchemaRegisterInjector method registerForCreateSource.

private void registerForCreateSource(final ConfiguredStatement<? extends CreateSource> cs) {
    // since this injector is chained after the TopicCreateInjector,
    // we can assume that the kafka topic is always present in the
    // statement properties
    final CreateSource statement = cs.getStatement();
    final LogicalSchema schema = statement.getElements().toLogicalSchema();
    final FormatInfo keyFormatInfo = SourcePropertiesUtil.getKeyFormat(statement.getProperties(), statement.getName());
    final Format keyFormat = tryGetFormat(keyFormatInfo, true, cs.getStatementText());
    final SerdeFeatures keyFeatures = SerdeFeaturesFactory.buildKeyFeatures(schema, keyFormat);
    final FormatInfo valueFormatInfo = SourcePropertiesUtil.getValueFormat(statement.getProperties());
    final Format valueFormat = tryGetFormat(valueFormatInfo, false, cs.getStatementText());
    final SerdeFeatures valFeatures = SerdeFeaturesFactory.buildValueFeatures(schema, valueFormat, statement.getProperties().getValueSerdeFeatures(), cs.getSessionConfig().getConfig(false));
    final SchemaAndId rawKeySchema = (SchemaAndId) cs.getSessionConfig().getOverrides().get(CommonCreateConfigs.KEY_SCHEMA_ID);
    final SchemaAndId rawValueSchema = (SchemaAndId) cs.getSessionConfig().getOverrides().get(CommonCreateConfigs.VALUE_SCHEMA_ID);
    registerSchemas(schema, Pair.of(rawKeySchema, rawValueSchema), statement.getProperties().getKafkaTopic(), keyFormatInfo, keyFeatures, valueFormatInfo, valFeatures, cs.getSessionConfig().getConfig(false), cs.getStatementText(), false);
}
Also used : Format(io.confluent.ksql.serde.Format) SchemaAndId(io.confluent.ksql.schema.ksql.inference.TopicSchemaSupplier.SchemaAndId) CreateSource(io.confluent.ksql.parser.tree.CreateSource) LogicalSchema(io.confluent.ksql.schema.ksql.LogicalSchema) FormatInfo(io.confluent.ksql.serde.FormatInfo) SerdeFeatures(io.confluent.ksql.serde.SerdeFeatures)

Aggregations

SerdeFeatures (io.confluent.ksql.serde.SerdeFeatures)7 FormatInfo (io.confluent.ksql.serde.FormatInfo)6 LogicalSchema (io.confluent.ksql.schema.ksql.LogicalSchema)3 KsqlTopic (io.confluent.ksql.execution.ddl.commands.KsqlTopic)2 Formats (io.confluent.ksql.execution.plan.Formats)2 CreateSource (io.confluent.ksql.parser.tree.CreateSource)2 SchemaAndId (io.confluent.ksql.schema.ksql.inference.TopicSchemaSupplier.SchemaAndId)2 Format (io.confluent.ksql.serde.Format)2 Optional (java.util.Optional)2 NewTopic (io.confluent.ksql.analyzer.Analysis.Into.NewTopic)1 StatementRewriteForMagicPseudoTimestamp (io.confluent.ksql.engine.rewrite.StatementRewriteForMagicPseudoTimestamp)1 QueryContext (io.confluent.ksql.execution.context.QueryContext)1 Stacker (io.confluent.ksql.execution.context.QueryContext.Stacker)1 Expression (io.confluent.ksql.execution.expression.tree.Expression)1 FunctionCall (io.confluent.ksql.execution.expression.tree.FunctionCall)1 ExecutionStep (io.confluent.ksql.execution.plan.ExecutionStep)1 JoinType (io.confluent.ksql.execution.plan.JoinType)1 KStreamHolder (io.confluent.ksql.execution.plan.KStreamHolder)1 SelectExpression (io.confluent.ksql.execution.plan.SelectExpression)1 StreamFilter (io.confluent.ksql.execution.plan.StreamFilter)1