Search in sources :

Example 6 with SimpleColumn

use of io.confluent.ksql.schema.ksql.SimpleColumn in project ksql by confluentinc.

the class KafkaSerdeFactory method createSerde.

static Serde<List<?>> createSerde(final PersistenceSchema schema) {
    final List<SimpleColumn> columns = schema.columns();
    if (columns.isEmpty()) {
        // No columns:
        return new KsqlVoidSerde<>();
    }
    if (columns.size() != 1) {
        throw new KsqlException("The '" + FormatFactory.KAFKA.name() + "' format only supports a single field. Got: " + columns);
    }
    final SimpleColumn singleColumn = columns.get(0);
    final Class<?> javaType = SchemaConverters.sqlToJavaConverter().toJavaType(singleColumn.type());
    return createSerde(singleColumn, javaType);
}
Also used : KsqlVoidSerde(io.confluent.ksql.serde.voids.KsqlVoidSerde) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn) KsqlException(io.confluent.ksql.util.KsqlException)

Example 7 with SimpleColumn

use of io.confluent.ksql.schema.ksql.SimpleColumn in project ksql by confluentinc.

the class ConnectSchemas method columnsToConnectSchema.

/**
 * Convert a list of columns into a Connect Struct schema with fields to match.
 *
 * @param columns the list of columns.
 * @return the Struct schema.
 */
public static ConnectSchema columnsToConnectSchema(final List<? extends SimpleColumn> columns) {
    final SqlToConnectTypeConverter converter = SchemaConverters.sqlToConnectConverter();
    final SchemaBuilder builder = SchemaBuilder.struct();
    for (final SimpleColumn column : columns) {
        final Schema colSchema = converter.toConnectSchema(column.type());
        builder.field(column.name().text(), colSchema);
    }
    return (ConnectSchema) builder.build();
}
Also used : ConnectSchema(org.apache.kafka.connect.data.ConnectSchema) Schema(org.apache.kafka.connect.data.Schema) SchemaBuilder(org.apache.kafka.connect.data.SchemaBuilder) ConnectSchema(org.apache.kafka.connect.data.ConnectSchema) SqlToConnectTypeConverter(io.confluent.ksql.schema.ksql.SchemaConverters.SqlToConnectTypeConverter) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn)

Example 8 with SimpleColumn

use of io.confluent.ksql.schema.ksql.SimpleColumn in project ksql by confluentinc.

the class KsqlDelimitedDeserializerTest method column.

private static SimpleColumn column(final String name, final SqlType type) {
    final SimpleColumn column = mock(SimpleColumn.class);
    when(column.name()).thenReturn(ColumnName.of(name));
    when(column.type()).thenReturn(type);
    return column;
}
Also used : SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn)

Example 9 with SimpleColumn

use of io.confluent.ksql.schema.ksql.SimpleColumn in project ksql by confluentinc.

the class SchemaRegistryTopicSchemaSupplier method fromParsedSchema.

private SchemaResult fromParsedSchema(final Optional<String> topic, final int id, final Optional<Integer> schemaId, final ParsedSchema parsedSchema, final FormatInfo expectedFormat, final SerdeFeatures serdeFeatures, final boolean isKey) {
    final Format format = formatFactory.apply(expectedFormat);
    // Topic existence means schema id is not provided original so that default
    // policy should be used
    final SchemaTranslator translator = schemaId.isPresent() ? format.getSchemaTranslator(expectedFormat.getProperties(), SchemaTranslationPolicies.of(SchemaTranslationPolicy.ORIGINAL_FIELD_NAME)) : format.getSchemaTranslator(expectedFormat.getProperties());
    if (!parsedSchema.schemaType().equals(translator.name())) {
        return incorrectFormat(topic, schemaId, translator.name(), parsedSchema.schemaType(), isKey);
    }
    final List<SimpleColumn> columns;
    try {
        columns = translator.toColumns(parsedSchema, serdeFeatures, isKey);
    } catch (final Exception e) {
        return notCompatible(topic, schemaId, parsedSchema.canonicalString(), e, isKey);
    }
    if (isKey && columns.size() > 1) {
        return multiColumnKeysNotSupported(topic, schemaId, parsedSchema.canonicalString());
    }
    return SchemaResult.success(SchemaAndId.schemaAndId(columns, parsedSchema, id));
}
Also used : SchemaTranslator(io.confluent.ksql.serde.SchemaTranslator) Format(io.confluent.ksql.serde.Format) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn) KsqlException(io.confluent.ksql.util.KsqlException) RestClientException(io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException)

Example 10 with SimpleColumn

use of io.confluent.ksql.schema.ksql.SimpleColumn in project ksql by confluentinc.

the class DefaultSchemaInjectorTest method shouldEscapeAvroSchemaThatHasReservedColumnName.

@Test
public void shouldEscapeAvroSchemaThatHasReservedColumnName() {
    // Given:
    givenKeyAndValueInferenceSupported();
    reset(schemaSupplier);
    when(schemaSupplier.getKeySchema(any(), any(), any(), any())).thenReturn(SchemaResult.success(schemaAndId(SR_KEY_SCHEMA, KEY_AVRO_SCHEMA, KEY_SCHEMA_ID)));
    final SimpleColumn col0 = mock(SimpleColumn.class);
    when(col0.name()).thenReturn(ColumnName.of("CREATE"));
    when(col0.type()).thenReturn(SqlTypes.BIGINT);
    when(schemaSupplier.getValueSchema(any(), any(), any(), any())).thenReturn(SchemaResult.success(schemaAndId(ImmutableList.of(col0), VALUE_AVRO_SCHEMA, VALUE_SCHEMA_ID)));
    // When:
    final ConfiguredStatement<CreateTable> inject = injector.inject(ctStatement);
    // Then:
    assertThat(inject.getStatementText(), containsString("`CREATE`"));
}
Also used : CreateTable(io.confluent.ksql.parser.tree.CreateTable) SimpleColumn(io.confluent.ksql.schema.ksql.SimpleColumn) Test(org.junit.Test)

Aggregations

SimpleColumn (io.confluent.ksql.schema.ksql.SimpleColumn)13 Test (org.junit.Test)4 KsqlException (io.confluent.ksql.util.KsqlException)3 ConnectSchema (org.apache.kafka.connect.data.ConnectSchema)3 ImmutableList (com.google.common.collect.ImmutableList)2 List (java.util.List)2 RestClientException (io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException)1 CreateTable (io.confluent.ksql.parser.tree.CreateTable)1 SqlToConnectTypeConverter (io.confluent.ksql.schema.ksql.SchemaConverters.SqlToConnectTypeConverter)1 SqlBaseType (io.confluent.ksql.schema.ksql.types.SqlBaseType)1 Format (io.confluent.ksql.serde.Format)1 SchemaTranslator (io.confluent.ksql.serde.SchemaTranslator)1 UnwrappedDeserializer (io.confluent.ksql.serde.unwrapped.UnwrappedDeserializer)1 UnwrappedSerializer (io.confluent.ksql.serde.unwrapped.UnwrappedSerializer)1 KsqlVoidSerde (io.confluent.ksql.serde.voids.KsqlVoidSerde)1 ArrayList (java.util.ArrayList)1 CSVParser (org.apache.commons.csv.CSVParser)1 SerializationException (org.apache.kafka.common.errors.SerializationException)1 Schema (org.apache.kafka.connect.data.Schema)1 SchemaBuilder (org.apache.kafka.connect.data.SchemaBuilder)1