Search in sources :

Example 16 with SourceRecord

use of org.apache.kafka.connect.source.SourceRecord in project apache-kafka-on-k8s by banzaicloud.

the class CastTest method castWholeRecordValueSchemalessBooleanFalse.

@Test
public void castWholeRecordValueSchemalessBooleanFalse() {
    xformValue.configure(Collections.singletonMap(Cast.SPEC_CONFIG, "boolean"));
    SourceRecord transformed = xformValue.apply(new SourceRecord(null, null, "topic", 0, null, 0));
    assertNull(transformed.valueSchema());
    assertEquals(false, transformed.value());
}
Also used : SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.Test)

Example 17 with SourceRecord

use of org.apache.kafka.connect.source.SourceRecord in project apache-kafka-on-k8s by banzaicloud.

the class CastTest method castWholeRecordValueWithSchemaInt8.

@Test
public void castWholeRecordValueWithSchemaInt8() {
    xformValue.configure(Collections.singletonMap(Cast.SPEC_CONFIG, "int8"));
    SourceRecord transformed = xformValue.apply(new SourceRecord(null, null, "topic", 0, Schema.INT32_SCHEMA, 42));
    assertEquals(Schema.Type.INT8, transformed.valueSchema().type());
    assertEquals((byte) 42, transformed.value());
}
Also used : SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.Test)

Example 18 with SourceRecord

use of org.apache.kafka.connect.source.SourceRecord in project apache-kafka-on-k8s by banzaicloud.

the class CastTest method castWholeRecordValueSchemalessInt32.

@Test
public void castWholeRecordValueSchemalessInt32() {
    xformValue.configure(Collections.singletonMap(Cast.SPEC_CONFIG, "int32"));
    SourceRecord transformed = xformValue.apply(new SourceRecord(null, null, "topic", 0, null, 42));
    assertNull(transformed.valueSchema());
    assertEquals(42, transformed.value());
}
Also used : SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.Test)

Example 19 with SourceRecord

use of org.apache.kafka.connect.source.SourceRecord in project apache-kafka-on-k8s by banzaicloud.

the class CastTest method castWholeRecordValueWithSchemaInt32.

@Test
public void castWholeRecordValueWithSchemaInt32() {
    xformValue.configure(Collections.singletonMap(Cast.SPEC_CONFIG, "int32"));
    SourceRecord transformed = xformValue.apply(new SourceRecord(null, null, "topic", 0, Schema.INT32_SCHEMA, 42));
    assertEquals(Schema.Type.INT32, transformed.valueSchema().type());
    assertEquals(42, transformed.value());
}
Also used : SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.Test)

Example 20 with SourceRecord

use of org.apache.kafka.connect.source.SourceRecord in project apache-kafka-on-k8s by banzaicloud.

the class FlattenTest method testOptionalAndDefaultValuesNested.

@Test
public void testOptionalAndDefaultValuesNested() {
    // If we have a nested structure where an entire sub-Struct is optional, all flattened fields generated from its
    // children should also be optional. Similarly, if the parent Struct has a default value, the default value for
    // the flattened field
    xformValue.configure(Collections.<String, String>emptyMap());
    SchemaBuilder builder = SchemaBuilder.struct().optional();
    builder.field("req_field", Schema.STRING_SCHEMA);
    builder.field("opt_field", SchemaBuilder.string().optional().defaultValue("child_default").build());
    Struct childDefaultValue = new Struct(builder);
    childDefaultValue.put("req_field", "req_default");
    builder.defaultValue(childDefaultValue);
    Schema schema = builder.build();
    // Intentionally leave this entire value empty since it is optional
    Struct value = new Struct(schema);
    SourceRecord transformed = xformValue.apply(new SourceRecord(null, null, "topic", 0, schema, value));
    assertNotNull(transformed);
    Schema transformedSchema = transformed.valueSchema();
    assertEquals(Schema.Type.STRUCT, transformedSchema.type());
    assertEquals(2, transformedSchema.fields().size());
    // Required field should pick up both being optional and the default value from the parent
    Schema transformedReqFieldSchema = SchemaBuilder.string().optional().defaultValue("req_default").build();
    assertEquals(transformedReqFieldSchema, transformedSchema.field("req_field").schema());
    // The optional field should still be optional but should have picked up the default value. However, since
    // the parent didn't specify the default explicitly, we should still be using the field's normal default
    Schema transformedOptFieldSchema = SchemaBuilder.string().optional().defaultValue("child_default").build();
    assertEquals(transformedOptFieldSchema, transformedSchema.field("opt_field").schema());
}
Also used : Schema(org.apache.kafka.connect.data.Schema) SchemaBuilder(org.apache.kafka.connect.data.SchemaBuilder) SourceRecord(org.apache.kafka.connect.source.SourceRecord) Struct(org.apache.kafka.connect.data.Struct) Test(org.junit.Test)

Aggregations

SourceRecord (org.apache.kafka.connect.source.SourceRecord)308 Test (org.junit.Test)148 Test (org.junit.jupiter.api.Test)98 Struct (org.apache.kafka.connect.data.Struct)68 HashMap (java.util.HashMap)60 Schema (org.apache.kafka.connect.data.Schema)45 ThreadedTest (org.apache.kafka.connect.util.ThreadedTest)27 ParameterizedTest (org.apache.kafka.connect.util.ParameterizedTest)23 ArrayList (java.util.ArrayList)22 RetryWithToleranceOperatorTest (org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperatorTest)21 Map (java.util.Map)15 SchemaBuilder (org.apache.kafka.connect.data.SchemaBuilder)13 ConnectException (org.apache.kafka.connect.errors.ConnectException)13 Document (org.bson.Document)13 FixFor (io.debezium.doc.FixFor)12 List (java.util.List)12 RecordsForCollection (io.debezium.connector.mongodb.RecordMakers.RecordsForCollection)11 ProducerRecord (org.apache.kafka.clients.producer.ProducerRecord)11 ConnectHeaders (org.apache.kafka.connect.header.ConnectHeaders)11 BsonTimestamp (org.bson.BsonTimestamp)11