Search in sources :

Example 26 with SinkRecord

use of org.apache.kafka.connect.sink.SinkRecord in project kafka by apache.

the class ValueToKeyTest method withSchema.

@Test
public void withSchema() {
    final ValueToKey<SinkRecord> xform = new ValueToKey<>();
    xform.configure(Collections.singletonMap("fields", "a,b"));
    final Schema valueSchema = SchemaBuilder.struct().field("a", Schema.INT32_SCHEMA).field("b", Schema.INT32_SCHEMA).field("c", Schema.INT32_SCHEMA).build();
    final Struct value = new Struct(valueSchema);
    value.put("a", 1);
    value.put("b", 2);
    value.put("c", 3);
    final SinkRecord record = new SinkRecord("", 0, null, null, valueSchema, value, 0);
    final SinkRecord transformedRecord = xform.apply(record);
    final Schema expectedKeySchema = SchemaBuilder.struct().field("a", Schema.INT32_SCHEMA).field("b", Schema.INT32_SCHEMA).build();
    final Struct expectedKey = new Struct(expectedKeySchema).put("a", 1).put("b", 2);
    assertEquals(expectedKeySchema, transformedRecord.keySchema());
    assertEquals(expectedKey, transformedRecord.key());
}
Also used : Schema(org.apache.kafka.connect.data.Schema) SinkRecord(org.apache.kafka.connect.sink.SinkRecord) Struct(org.apache.kafka.connect.data.Struct) Test(org.junit.Test)

Example 27 with SinkRecord

use of org.apache.kafka.connect.sink.SinkRecord in project kafka by apache.

the class ValueToKeyTest method schemaless.

@Test
public void schemaless() {
    final ValueToKey<SinkRecord> xform = new ValueToKey<>();
    xform.configure(Collections.singletonMap("fields", "a,b"));
    final HashMap<String, Integer> value = new HashMap<>();
    value.put("a", 1);
    value.put("b", 2);
    value.put("c", 3);
    final SinkRecord record = new SinkRecord("", 0, null, null, null, value, 0);
    final SinkRecord transformedRecord = xform.apply(record);
    final HashMap<String, Integer> expectedKey = new HashMap<>();
    expectedKey.put("a", 1);
    expectedKey.put("b", 2);
    assertNull(transformedRecord.keySchema());
    assertEquals(expectedKey, transformedRecord.key());
}
Also used : HashMap(java.util.HashMap) SinkRecord(org.apache.kafka.connect.sink.SinkRecord) Test(org.junit.Test)

Aggregations

SinkRecord (org.apache.kafka.connect.sink.SinkRecord)27 Test (org.junit.Test)19 HashMap (java.util.HashMap)11 TopicPartition (org.apache.kafka.common.TopicPartition)7 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)7 OffsetAndMetadata (org.apache.kafka.clients.consumer.OffsetAndMetadata)6 Collection (java.util.Collection)4 Schema (org.apache.kafka.connect.data.Schema)3 Struct (org.apache.kafka.connect.data.Struct)3 ConnectException (org.apache.kafka.connect.errors.ConnectException)3 Map (java.util.Map)2 ConsumerRecords (org.apache.kafka.clients.consumer.ConsumerRecords)2 OffsetCommitCallback (org.apache.kafka.clients.consumer.OffsetCommitCallback)2 SchemaAndValue (org.apache.kafka.connect.data.SchemaAndValue)2 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)1 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)1 KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer)1 WakeupException (org.apache.kafka.common.errors.WakeupException)1 TimestampType (org.apache.kafka.common.record.TimestampType)1 RetriableException (org.apache.kafka.connect.errors.RetriableException)1