Search in sources :

Example 11 with Callback

use of org.apache.kafka.connect.util.Callback in project kafka by apache.

the class KafkaStatusBackingStore method configure.

@Override
public void configure(WorkerConfig config) {
    this.topic = config.getString(DistributedConfig.STATUS_STORAGE_TOPIC_CONFIG);
    if (topic.equals(""))
        throw new ConfigException("Must specify topic for connector status.");
    Map<String, Object> producerProps = new HashMap<>();
    producerProps.putAll(config.originals());
    producerProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    producerProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class.getName());
    // we handle retries in this class
    producerProps.put(ProducerConfig.RETRIES_CONFIG, 0);
    Map<String, Object> consumerProps = new HashMap<>();
    consumerProps.putAll(config.originals());
    consumerProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
    consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class.getName());
    Callback<ConsumerRecord<String, byte[]>> readCallback = new Callback<ConsumerRecord<String, byte[]>>() {

        @Override
        public void onCompletion(Throwable error, ConsumerRecord<String, byte[]> record) {
            read(record);
        }
    };
    this.kafkaLog = new KafkaBasedLog<>(topic, producerProps, consumerProps, readCallback, time);
}
Also used : HashMap(java.util.HashMap) StringDeserializer(org.apache.kafka.common.serialization.StringDeserializer) ConfigException(org.apache.kafka.common.config.ConfigException) ByteArraySerializer(org.apache.kafka.common.serialization.ByteArraySerializer) ConsumerRecord(org.apache.kafka.clients.consumer.ConsumerRecord) Callback(org.apache.kafka.connect.util.Callback) ByteArrayDeserializer(org.apache.kafka.common.serialization.ByteArrayDeserializer) StringSerializer(org.apache.kafka.common.serialization.StringSerializer)

Example 12 with Callback

use of org.apache.kafka.connect.util.Callback in project kafka by apache.

the class ConnectorsResourceTest method testCreateConnectorNotLeader.

@Test
public void testCreateConnectorNotLeader() throws Throwable {
    CreateConnectorRequest body = new CreateConnectorRequest(CONNECTOR_NAME, Collections.singletonMap(ConnectorConfig.NAME_CONFIG, CONNECTOR_NAME));
    final Capture<Callback<Herder.Created<ConnectorInfo>>> cb = Capture.newInstance();
    herder.putConnectorConfig(EasyMock.eq(CONNECTOR_NAME), EasyMock.eq(body.config()), EasyMock.eq(false), EasyMock.capture(cb));
    expectAndCallbackNotLeaderException(cb);
    // Should forward request
    EasyMock.expect(RestServer.httpRequest(EasyMock.eq("http://leader:8083/connectors?forward=false"), EasyMock.eq("POST"), EasyMock.eq(body), EasyMock.<TypeReference>anyObject())).andReturn(new RestServer.HttpResponse<>(201, new HashMap<String, List<String>>(), new ConnectorInfo(CONNECTOR_NAME, CONNECTOR_CONFIG, CONNECTOR_TASK_NAMES)));
    PowerMock.replayAll();
    connectorsResource.createConnector(FORWARD, body);
    PowerMock.verifyAll();
}
Also used : RestServer(org.apache.kafka.connect.runtime.rest.RestServer) Callback(org.apache.kafka.connect.util.Callback) ConnectorInfo(org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo) HashMap(java.util.HashMap) CreateConnectorRequest(org.apache.kafka.connect.runtime.rest.entities.CreateConnectorRequest) Herder(org.apache.kafka.connect.runtime.Herder) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 13 with Callback

use of org.apache.kafka.connect.util.Callback in project kafka by apache.

the class ConnectorsResourceTest method testCreateConnector.

@Test
public void testCreateConnector() throws Throwable {
    CreateConnectorRequest body = new CreateConnectorRequest(CONNECTOR_NAME, Collections.singletonMap(ConnectorConfig.NAME_CONFIG, CONNECTOR_NAME));
    final Capture<Callback<Herder.Created<ConnectorInfo>>> cb = Capture.newInstance();
    herder.putConnectorConfig(EasyMock.eq(CONNECTOR_NAME), EasyMock.eq(body.config()), EasyMock.eq(false), EasyMock.capture(cb));
    expectAndCallbackResult(cb, new Herder.Created<>(true, new ConnectorInfo(CONNECTOR_NAME, CONNECTOR_CONFIG, CONNECTOR_TASK_NAMES)));
    PowerMock.replayAll();
    connectorsResource.createConnector(FORWARD, body);
    PowerMock.verifyAll();
}
Also used : Callback(org.apache.kafka.connect.util.Callback) ConnectorInfo(org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo) CreateConnectorRequest(org.apache.kafka.connect.runtime.rest.entities.CreateConnectorRequest) Herder(org.apache.kafka.connect.runtime.Herder) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 14 with Callback

use of org.apache.kafka.connect.util.Callback in project kafka by apache.

the class ConnectorsResourceTest method testRestartTaskLeaderRedirect.

@Test
public void testRestartTaskLeaderRedirect() throws Throwable {
    ConnectorTaskId taskId = new ConnectorTaskId(CONNECTOR_NAME, 0);
    final Capture<Callback<Void>> cb = Capture.newInstance();
    herder.restartTask(EasyMock.eq(taskId), EasyMock.capture(cb));
    expectAndCallbackNotLeaderException(cb);
    EasyMock.expect(RestServer.httpRequest(EasyMock.eq("http://leader:8083/connectors/" + CONNECTOR_NAME + "/tasks/0/restart?forward=true"), EasyMock.eq("POST"), EasyMock.isNull(), EasyMock.<TypeReference>anyObject())).andReturn(new RestServer.HttpResponse<>(202, new HashMap<String, List<String>>(), null));
    PowerMock.replayAll();
    connectorsResource.restartTask(CONNECTOR_NAME, 0, null);
    PowerMock.verifyAll();
}
Also used : RestServer(org.apache.kafka.connect.runtime.rest.RestServer) Callback(org.apache.kafka.connect.util.Callback) ConnectorTaskId(org.apache.kafka.connect.util.ConnectorTaskId) HashMap(java.util.HashMap) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 15 with Callback

use of org.apache.kafka.connect.util.Callback in project kafka by apache.

the class ConnectorsResourceTest method testRestartConnectorNotFound.

@Test(expected = NotFoundException.class)
public void testRestartConnectorNotFound() throws Throwable {
    final Capture<Callback<Void>> cb = Capture.newInstance();
    herder.restartConnector(EasyMock.eq(CONNECTOR_NAME), EasyMock.capture(cb));
    expectAndCallbackException(cb, new NotFoundException("not found"));
    PowerMock.replayAll();
    connectorsResource.restartConnector(CONNECTOR_NAME, FORWARD);
    PowerMock.verifyAll();
}
Also used : Callback(org.apache.kafka.connect.util.Callback) NotFoundException(org.apache.kafka.connect.errors.NotFoundException) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Aggregations

Callback (org.apache.kafka.connect.util.Callback)20 Test (org.junit.Test)18 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)18 HashMap (java.util.HashMap)8 NotFoundException (org.apache.kafka.connect.errors.NotFoundException)6 ConnectorInfo (org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo)6 Herder (org.apache.kafka.connect.runtime.Herder)5 RestServer (org.apache.kafka.connect.runtime.rest.RestServer)4 CreateConnectorRequest (org.apache.kafka.connect.runtime.rest.entities.CreateConnectorRequest)4 ByteBuffer (java.nio.ByteBuffer)3 AtomicBoolean (java.util.concurrent.atomic.AtomicBoolean)3 ConsumerRecord (org.apache.kafka.clients.consumer.ConsumerRecord)3 ConnectorTaskId (org.apache.kafka.connect.util.ConnectorTaskId)3 Map (java.util.Map)2 NotAssignedException (org.apache.kafka.connect.runtime.distributed.NotAssignedException)2 ExecutionException (java.util.concurrent.ExecutionException)1 Response (javax.ws.rs.core.Response)1 KafkaException (org.apache.kafka.common.KafkaException)1 ConfigException (org.apache.kafka.common.config.ConfigException)1 ByteArrayDeserializer (org.apache.kafka.common.serialization.ByteArrayDeserializer)1