Search in sources :

Example 71 with SchemaAndValue

use of org.apache.kafka.connect.data.SchemaAndValue in project kafka by apache.

the class KafkaStatusBackingStoreTest method deleteConnectorState.

@Test
public void deleteConnectorState() {
    final byte[] value = new byte[0];
    Map<String, Object> statusMap = new HashMap<>();
    statusMap.put("worker_id", WORKER_ID);
    statusMap.put("state", "RUNNING");
    statusMap.put("generation", 0L);
    when(converter.fromConnectData(eq(STATUS_TOPIC), any(Schema.class), any(Struct.class))).thenReturn(value);
    when(converter.fromConnectData(eq(STATUS_TOPIC), any(Schema.class), any(Struct.class))).thenReturn(value);
    when(converter.toConnectData(STATUS_TOPIC, value)).thenReturn(new SchemaAndValue(null, statusMap));
    ConnectorStatus connectorStatus = new ConnectorStatus(CONNECTOR, ConnectorStatus.State.RUNNING, WORKER_ID, 0);
    store.put(connectorStatus);
    TaskStatus taskStatus = new TaskStatus(TASK, TaskStatus.State.RUNNING, WORKER_ID, 0);
    store.put(taskStatus);
    store.read(consumerRecord(0, "status-task-conn-0", value));
    verify(kafkaBasedLog).send(eq("status-connector-" + CONNECTOR), eq(value), any(Callback.class));
    verify(kafkaBasedLog).send(eq("status-task-conn-0"), eq(value), any(Callback.class));
    assertEquals(new HashSet<>(Collections.singletonList(CONNECTOR)), store.connectors());
    assertEquals(new HashSet<>(Collections.singletonList(taskStatus)), new HashSet<>(store.getAll(CONNECTOR)));
    store.read(consumerRecord(0, "status-connector-conn", null));
    assertTrue(store.connectors().isEmpty());
    assertTrue(store.getAll(CONNECTOR).isEmpty());
}
Also used : Callback(org.apache.kafka.clients.producer.Callback) HashMap(java.util.HashMap) Schema(org.apache.kafka.connect.data.Schema) ConnectorStatus(org.apache.kafka.connect.runtime.ConnectorStatus) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) TaskStatus(org.apache.kafka.connect.runtime.TaskStatus) Struct(org.apache.kafka.connect.data.Struct) SchemaAndValue(org.apache.kafka.connect.data.SchemaAndValue) Test(org.junit.Test)

Example 72 with SchemaAndValue

use of org.apache.kafka.connect.data.SchemaAndValue in project kafka by apache.

the class KafkaStatusBackingStoreTest method readConnectorState.

@Test
public void readConnectorState() {
    byte[] value = new byte[0];
    Map<String, Object> statusMap = new HashMap<>();
    statusMap.put("worker_id", WORKER_ID);
    statusMap.put("state", "RUNNING");
    statusMap.put("generation", 0L);
    when(converter.toConnectData(STATUS_TOPIC, value)).thenReturn(new SchemaAndValue(null, statusMap));
    store.read(consumerRecord(0, "status-connector-conn", value));
    ConnectorStatus status = new ConnectorStatus(CONNECTOR, ConnectorStatus.State.RUNNING, WORKER_ID, 0);
    assertEquals(status, store.get(CONNECTOR));
}
Also used : HashMap(java.util.HashMap) ConnectorStatus(org.apache.kafka.connect.runtime.ConnectorStatus) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) SchemaAndValue(org.apache.kafka.connect.data.SchemaAndValue) Test(org.junit.Test)

Example 73 with SchemaAndValue

use of org.apache.kafka.connect.data.SchemaAndValue in project kafka by apache.

the class KafkaStatusBackingStoreTest method readTaskState.

@Test
public void readTaskState() {
    byte[] value = new byte[0];
    Map<String, Object> statusMap = new HashMap<>();
    statusMap.put("worker_id", WORKER_ID);
    statusMap.put("state", "RUNNING");
    statusMap.put("generation", 0L);
    when(converter.toConnectData(STATUS_TOPIC, value)).thenReturn(new SchemaAndValue(null, statusMap));
    store.read(consumerRecord(0, "status-task-conn-0", value));
    TaskStatus status = new TaskStatus(TASK, TaskStatus.State.RUNNING, WORKER_ID, 0);
    assertEquals(status, store.get(TASK));
}
Also used : HashMap(java.util.HashMap) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) TaskStatus(org.apache.kafka.connect.runtime.TaskStatus) SchemaAndValue(org.apache.kafka.connect.data.SchemaAndValue) Test(org.junit.Test)

Example 74 with SchemaAndValue

use of org.apache.kafka.connect.data.SchemaAndValue in project kafka by apache.

the class KafkaConfigBackingStoreTest method expectRead.

private void expectRead(LinkedHashMap<String, byte[]> serializedValues, Map<String, Struct> deserializedValues) {
    expectReadToEnd(serializedValues);
    for (Map.Entry<String, Struct> deserializedValueEntry : deserializedValues.entrySet()) {
        byte[] serializedValue = serializedValues.get(deserializedValueEntry.getKey());
        EasyMock.expect(converter.toConnectData(EasyMock.eq(TOPIC), EasyMock.aryEq(serializedValue))).andReturn(new SchemaAndValue(null, structToMap(deserializedValueEntry.getValue())));
    }
}
Also used : Map(java.util.Map) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) Struct(org.apache.kafka.connect.data.Struct) SchemaAndValue(org.apache.kafka.connect.data.SchemaAndValue)

Example 75 with SchemaAndValue

use of org.apache.kafka.connect.data.SchemaAndValue in project kafka by apache.

the class KafkaConfigBackingStoreTest method testRecordToRestartRequestOnlyFailedInconsistent.

@Test
public void testRecordToRestartRequestOnlyFailedInconsistent() {
    ConsumerRecord<String, byte[]> record = new ConsumerRecord<>(TOPIC, 0, 0, 0L, TimestampType.CREATE_TIME, 0, 0, RESTART_CONNECTOR_KEYS.get(0), CONFIGS_SERIALIZED.get(0), new RecordHeaders(), Optional.empty());
    Struct struct = ONLY_FAILED_MISSING_STRUCT;
    SchemaAndValue schemaAndValue = new SchemaAndValue(struct.schema(), structToMap(struct));
    RestartRequest restartRequest = configStorage.recordToRestartRequest(record, schemaAndValue);
    assertEquals(CONNECTOR_1_NAME, restartRequest.connectorName());
    assertEquals(struct.getBoolean(INCLUDE_TASKS_FIELD_NAME), restartRequest.includeTasks());
    assertFalse(restartRequest.onlyFailed());
}
Also used : RecordHeaders(org.apache.kafka.common.header.internals.RecordHeaders) RestartRequest(org.apache.kafka.connect.runtime.RestartRequest) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) Struct(org.apache.kafka.connect.data.Struct) SchemaAndValue(org.apache.kafka.connect.data.SchemaAndValue) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Aggregations

SchemaAndValue (org.apache.kafka.connect.data.SchemaAndValue)140 Test (org.junit.Test)57 Schema (org.apache.kafka.connect.data.Schema)49 Test (org.junit.jupiter.api.Test)46 HashMap (java.util.HashMap)32 Struct (org.apache.kafka.connect.data.Struct)21 Date (org.apache.kafka.connect.data.Date)18 BigInteger (java.math.BigInteger)12 Map (java.util.Map)12 ConnectorStatus (org.apache.kafka.connect.runtime.ConnectorStatus)11 BigDecimal (java.math.BigDecimal)10 TopicPartition (org.apache.kafka.common.TopicPartition)9 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)8 TaskStatus (org.apache.kafka.connect.runtime.TaskStatus)8 Callback (org.apache.kafka.clients.producer.Callback)7 SinkRecord (org.apache.kafka.connect.sink.SinkRecord)7 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)7 Collection (java.util.Collection)6 GregorianCalendar (java.util.GregorianCalendar)6 LinkedHashMap (java.util.LinkedHashMap)6