Search in sources :

Example 6 with ConnectHeaders

use of org.apache.kafka.connect.header.ConnectHeaders in project kafka by apache.

the class HeaderFromTest method withSchema.

@ParameterizedTest
@MethodSource("data")
public void withSchema(String description, boolean keyTransform, RecordBuilder originalBuilder, List<String> transformFields, List<String> headers1, HeaderFrom.Operation operation, RecordBuilder expectedBuilder) {
    HeaderFrom<SourceRecord> xform = keyTransform ? new HeaderFrom.Key<>() : new HeaderFrom.Value<>();
    xform.configure(config(headers1, transformFields, operation));
    ConnectHeaders headers = new ConnectHeaders();
    headers.addString("existing", "existing-value");
    Headers expect = headers.duplicate();
    for (int i = 0; i < headers1.size(); i++) {
        expect.add(headers1.get(i), originalBuilder.fieldValues.get(i), originalBuilder.fieldSchemas.get(i));
    }
    SourceRecord originalRecord = originalBuilder.withSchema(keyTransform);
    SourceRecord expectedRecord = expectedBuilder.withSchema(keyTransform);
    SourceRecord xformed = xform.apply(originalRecord);
    assertSameRecord(expectedRecord, xformed);
}
Also used : ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) Headers(org.apache.kafka.connect.header.Headers) SourceRecord(org.apache.kafka.connect.source.SourceRecord) ParameterizedTest(org.junit.jupiter.params.ParameterizedTest) MethodSource(org.junit.jupiter.params.provider.MethodSource)

Example 7 with ConnectHeaders

use of org.apache.kafka.connect.header.ConnectHeaders in project kafka by apache.

the class InsertHeaderTest method insertionWithExistingSameHeader.

@Test
public void insertionWithExistingSameHeader() {
    xform.configure(config("existing", "inserted-value"));
    ConnectHeaders headers = new ConnectHeaders();
    headers.addString("existing", "preexisting-value");
    Headers expect = headers.duplicate().addString("existing", "inserted-value");
    SourceRecord original = sourceRecord(headers);
    SourceRecord xformed = xform.apply(original);
    assertNonHeaders(original, xformed);
    assertEquals(expect, xformed.headers());
}
Also used : ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) Headers(org.apache.kafka.connect.header.Headers) ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.jupiter.api.Test)

Example 8 with ConnectHeaders

use of org.apache.kafka.connect.header.ConnectHeaders in project kafka by apache.

the class InsertHeaderTest method insertionWithExistingOtherHeader.

@Test
public void insertionWithExistingOtherHeader() {
    xform.configure(config("inserted", "inserted-value"));
    ConnectHeaders headers = new ConnectHeaders();
    headers.addString("existing", "existing-value");
    Headers expect = headers.duplicate().addString("inserted", "inserted-value");
    SourceRecord original = sourceRecord(headers);
    SourceRecord xformed = xform.apply(original);
    assertNonHeaders(original, xformed);
    assertEquals(expect, xformed.headers());
}
Also used : ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) Headers(org.apache.kafka.connect.header.Headers) ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) SourceRecord(org.apache.kafka.connect.source.SourceRecord) Test(org.junit.jupiter.api.Test)

Example 9 with ConnectHeaders

use of org.apache.kafka.connect.header.ConnectHeaders in project apache-kafka-on-k8s by banzaicloud.

the class SinkRecordTest method shouldCreateSinkRecordWithHeaders.

@Test
public void shouldCreateSinkRecordWithHeaders() {
    Headers headers = new ConnectHeaders().addString("h1", "hv1").addBoolean("h2", true);
    record = new SinkRecord(TOPIC_NAME, PARTITION_NUMBER, Schema.STRING_SCHEMA, "key", Schema.BOOLEAN_SCHEMA, false, KAFKA_OFFSET, KAFKA_TIMESTAMP, TS_TYPE, headers);
    assertNotNull(record.headers());
    assertSame(headers, record.headers());
    assertFalse(record.headers().isEmpty());
}
Also used : ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) Headers(org.apache.kafka.connect.header.Headers) Test(org.junit.Test)

Example 10 with ConnectHeaders

use of org.apache.kafka.connect.header.ConnectHeaders in project apache-kafka-on-k8s by banzaicloud.

the class SinkRecordTest method shouldDuplicateRecordUsingNewHeaders.

@Test
public void shouldDuplicateRecordUsingNewHeaders() {
    Headers newHeaders = new ConnectHeaders().addString("h3", "hv3");
    SinkRecord duplicate = record.newRecord(TOPIC_NAME, PARTITION_NUMBER, Schema.STRING_SCHEMA, "key", Schema.BOOLEAN_SCHEMA, false, KAFKA_TIMESTAMP, newHeaders);
    assertEquals(TOPIC_NAME, duplicate.topic());
    assertEquals(PARTITION_NUMBER, duplicate.kafkaPartition());
    assertEquals(Schema.STRING_SCHEMA, duplicate.keySchema());
    assertEquals("key", duplicate.key());
    assertEquals(Schema.BOOLEAN_SCHEMA, duplicate.valueSchema());
    assertEquals(false, duplicate.value());
    assertEquals(KAFKA_OFFSET, duplicate.kafkaOffset());
    assertEquals(KAFKA_TIMESTAMP, duplicate.timestamp());
    assertEquals(TS_TYPE, duplicate.timestampType());
    assertNotNull(duplicate.headers());
    assertEquals(newHeaders, duplicate.headers());
    assertSame(newHeaders, duplicate.headers());
    assertNotSame(record.headers(), duplicate.headers());
    assertNotEquals(record.headers(), duplicate.headers());
}
Also used : ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) ConnectHeaders(org.apache.kafka.connect.header.ConnectHeaders) Headers(org.apache.kafka.connect.header.Headers) Test(org.junit.Test)

Aggregations

ConnectHeaders (org.apache.kafka.connect.header.ConnectHeaders)20 Headers (org.apache.kafka.connect.header.Headers)14 SourceRecord (org.apache.kafka.connect.source.SourceRecord)10 Test (org.junit.jupiter.api.Test)10 Test (org.junit.Test)6 SchemaAndValue (org.apache.kafka.connect.data.SchemaAndValue)3 ArrayList (java.util.ArrayList)2 ProducerRecord (org.apache.kafka.clients.producer.ProducerRecord)2 RetryWithToleranceOperatorTest (org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperatorTest)2 ParameterizedTest (org.apache.kafka.connect.util.ParameterizedTest)2 ThreadedTest (org.apache.kafka.connect.util.ThreadedTest)2 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)2 MethodSource (org.junit.jupiter.params.provider.MethodSource)2 Headers (org.apache.kafka.common.header.Headers)1 RecordHeaders (org.apache.kafka.common.header.internals.RecordHeaders)1 StringConverter (org.apache.kafka.connect.storage.StringConverter)1