Search in sources :

Example 1 with BadRequestException

use of org.apache.kafka.connect.runtime.rest.errors.BadRequestException in project kafka by apache.

the class DistributedHerderTest method testCreateConnectorFailedCustomValidation.

@Test
public void testCreateConnectorFailedCustomValidation() throws Exception {
    EasyMock.expect(member.memberId()).andStubReturn("leader");
    expectRebalance(1, Collections.<String>emptyList(), Collections.<ConnectorTaskId>emptyList());
    expectPostRebalanceCatchup(SNAPSHOT);
    member.wakeup();
    PowerMock.expectLastCall();
    // config validation
    ConnectorFactory connectorFactoryMock = PowerMock.createMock(ConnectorFactory.class);
    EasyMock.expect(worker.getConnectorFactory()).andStubReturn(connectorFactoryMock);
    Connector connectorMock = PowerMock.createMock(Connector.class);
    EasyMock.expect(connectorFactoryMock.newConnector(EasyMock.anyString())).andReturn(connectorMock);
    ConfigDef configDef = new ConfigDef();
    configDef.define("foo.bar", ConfigDef.Type.STRING, ConfigDef.Importance.HIGH, "foo.bar doc");
    EasyMock.expect(connectorMock.config()).andReturn(configDef);
    ConfigValue validatedValue = new ConfigValue("foo.bar");
    validatedValue.addErrorMessage("Failed foo.bar validation");
    EasyMock.expect(connectorMock.validate(CONN2_CONFIG)).andReturn(new Config(singletonList(validatedValue)));
    // CONN2 creation should fail
    Capture<Throwable> error = EasyMock.newCapture();
    putConnectorCallback.onCompletion(EasyMock.capture(error), EasyMock.<Herder.Created<ConnectorInfo>>isNull());
    PowerMock.expectLastCall();
    member.poll(EasyMock.anyInt());
    PowerMock.expectLastCall();
    // No immediate action besides this -- change will be picked up via the config log
    PowerMock.replayAll();
    herder.putConnectorConfig(CONN2, CONN2_CONFIG, false, putConnectorCallback);
    herder.tick();
    assertTrue(error.hasCaptured());
    assertTrue(error.getValue() instanceof BadRequestException);
    PowerMock.verifyAll();
}
Also used : SourceConnector(org.apache.kafka.connect.source.SourceConnector) Connector(org.apache.kafka.connect.connector.Connector) SinkConnector(org.apache.kafka.connect.sink.SinkConnector) ConfigValue(org.apache.kafka.common.config.ConfigValue) ConnectorFactory(org.apache.kafka.connect.runtime.ConnectorFactory) ConnectorInfo(org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo) WorkerConfig(org.apache.kafka.connect.runtime.WorkerConfig) ConnectorConfig(org.apache.kafka.connect.runtime.ConnectorConfig) Config(org.apache.kafka.common.config.Config) SinkConnectorConfig(org.apache.kafka.connect.runtime.SinkConnectorConfig) TaskConfig(org.apache.kafka.connect.runtime.TaskConfig) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) ConfigDef(org.apache.kafka.common.config.ConfigDef) Herder(org.apache.kafka.connect.runtime.Herder) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 2 with BadRequestException

use of org.apache.kafka.connect.runtime.rest.errors.BadRequestException in project kafka by apache.

the class DistributedHerderTest method testCreateConnectorFailedBasicValidation.

@Test
public void testCreateConnectorFailedBasicValidation() throws Exception {
    EasyMock.expect(member.memberId()).andStubReturn("leader");
    expectRebalance(1, Collections.<String>emptyList(), Collections.<ConnectorTaskId>emptyList());
    expectPostRebalanceCatchup(SNAPSHOT);
    HashMap<String, String> config = new HashMap<>(CONN2_CONFIG);
    config.remove(ConnectorConfig.NAME_CONFIG);
    member.wakeup();
    PowerMock.expectLastCall();
    // config validation
    ConnectorFactory connectorFactoryMock = PowerMock.createMock(ConnectorFactory.class);
    EasyMock.expect(worker.getConnectorFactory()).andStubReturn(connectorFactoryMock);
    Connector connectorMock = PowerMock.createMock(Connector.class);
    EasyMock.expect(connectorFactoryMock.newConnector(EasyMock.anyString())).andReturn(connectorMock);
    EasyMock.expect(connectorMock.config()).andReturn(new ConfigDef());
    EasyMock.expect(connectorMock.validate(config)).andReturn(new Config(Collections.<ConfigValue>emptyList()));
    // CONN2 creation should fail
    Capture<Throwable> error = EasyMock.newCapture();
    putConnectorCallback.onCompletion(EasyMock.capture(error), EasyMock.<Herder.Created<ConnectorInfo>>isNull());
    PowerMock.expectLastCall();
    member.poll(EasyMock.anyInt());
    PowerMock.expectLastCall();
    // No immediate action besides this -- change will be picked up via the config log
    PowerMock.replayAll();
    herder.putConnectorConfig(CONN2, config, false, putConnectorCallback);
    herder.tick();
    assertTrue(error.hasCaptured());
    assertTrue(error.getValue() instanceof BadRequestException);
    PowerMock.verifyAll();
}
Also used : SourceConnector(org.apache.kafka.connect.source.SourceConnector) Connector(org.apache.kafka.connect.connector.Connector) SinkConnector(org.apache.kafka.connect.sink.SinkConnector) ConfigValue(org.apache.kafka.common.config.ConfigValue) ConnectorInfo(org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo) HashMap(java.util.HashMap) WorkerConfig(org.apache.kafka.connect.runtime.WorkerConfig) ConnectorConfig(org.apache.kafka.connect.runtime.ConnectorConfig) Config(org.apache.kafka.common.config.Config) SinkConnectorConfig(org.apache.kafka.connect.runtime.SinkConnectorConfig) TaskConfig(org.apache.kafka.connect.runtime.TaskConfig) ConnectorFactory(org.apache.kafka.connect.runtime.ConnectorFactory) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) ConfigDef(org.apache.kafka.common.config.ConfigDef) Herder(org.apache.kafka.connect.runtime.Herder) PrepareForTest(org.powermock.core.classloader.annotations.PrepareForTest) Test(org.junit.Test)

Example 3 with BadRequestException

use of org.apache.kafka.connect.runtime.rest.errors.BadRequestException in project kafka by apache.

the class DistributedHerder method putConnectorConfig.

@Override
public void putConnectorConfig(final String connName, final Map<String, String> config, final boolean allowReplace, final Callback<Created<ConnectorInfo>> callback) {
    log.trace("Submitting connector config write request {}", connName);
    addRequest(new Callable<Void>() {

        @Override
        public Void call() throws Exception {
            ConfigInfos validatedConfig = validateConnectorConfig(config);
            if (validatedConfig.errorCount() > 0) {
                callback.onCompletion(new BadRequestException("Connector configuration is invalid " + "(use the endpoint `/{connectorType}/config/validate` to get a full list of errors)"), null);
                return null;
            }
            log.trace("Handling connector config request {}", connName);
            if (!isLeader()) {
                callback.onCompletion(new NotLeaderException("Only the leader can set connector configs.", leaderUrl()), null);
                return null;
            }
            boolean exists = configState.contains(connName);
            if (!allowReplace && exists) {
                callback.onCompletion(new AlreadyExistsException("Connector " + connName + " already exists"), null);
                return null;
            }
            log.trace("Submitting connector config {} {} {}", connName, allowReplace, configState.connectors());
            configBackingStore.putConnectorConfig(connName, config);
            // Note that we use the updated connector config despite the fact that we don't have an updated
            // snapshot yet. The existing task info should still be accurate.
            ConnectorInfo info = new ConnectorInfo(connName, config, configState.tasks(connName));
            callback.onCompletion(null, new Created<>(!exists, info));
            return null;
        }
    }, forwardErrorCallback(callback));
}
Also used : AlreadyExistsException(org.apache.kafka.connect.errors.AlreadyExistsException) ConnectorInfo(org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) TimeoutException(java.util.concurrent.TimeoutException) AlreadyExistsException(org.apache.kafka.connect.errors.AlreadyExistsException) WakeupException(org.apache.kafka.common.errors.WakeupException) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) NotFoundException(org.apache.kafka.connect.errors.NotFoundException) NoSuchElementException(java.util.NoSuchElementException) ConnectException(org.apache.kafka.connect.errors.ConnectException) ConfigInfos(org.apache.kafka.connect.runtime.rest.entities.ConfigInfos)

Example 4 with BadRequestException

use of org.apache.kafka.connect.runtime.rest.errors.BadRequestException in project kafka by apache.

the class StandaloneHerder method putConnectorConfig.

@Override
public synchronized void putConnectorConfig(String connName, final Map<String, String> config, boolean allowReplace, final Callback<Created<ConnectorInfo>> callback) {
    try {
        ConfigInfos validatedConfig = validateConnectorConfig(config);
        if (validatedConfig.errorCount() > 0) {
            callback.onCompletion(new BadRequestException("Connector configuration is invalid " + "(use the endpoint `/{connectorType}/config/validate` to get a full list of errors)"), null);
            return;
        }
        boolean created = false;
        if (configState.contains(connName)) {
            if (!allowReplace) {
                callback.onCompletion(new AlreadyExistsException("Connector " + connName + " already exists"), null);
                return;
            }
            worker.stopConnector(connName);
        } else {
            created = true;
        }
        if (!startConnector(config)) {
            callback.onCompletion(new ConnectException("Failed to start connector: " + connName), null);
            return;
        }
        updateConnectorTasks(connName);
        callback.onCompletion(null, new Created<>(created, createConnectorInfo(connName)));
    } catch (ConnectException e) {
        callback.onCompletion(e, null);
    }
}
Also used : AlreadyExistsException(org.apache.kafka.connect.errors.AlreadyExistsException) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) ConfigInfos(org.apache.kafka.connect.runtime.rest.entities.ConfigInfos) ConnectException(org.apache.kafka.connect.errors.ConnectException)

Example 5 with BadRequestException

use of org.apache.kafka.connect.runtime.rest.errors.BadRequestException in project kafka by apache.

the class AbstractHerder method validateConnectorConfig.

@Override
public ConfigInfos validateConnectorConfig(Map<String, String> connectorConfig) {
    String connType = connectorConfig.get(ConnectorConfig.CONNECTOR_CLASS_CONFIG);
    if (connType == null)
        throw new BadRequestException("Connector config " + connectorConfig + " contains no connector type");
    Connector connector = getConnector(connType);
    final ConfigDef connectorConfigDef = ConnectorConfig.enrich((connector instanceof SourceConnector) ? SourceConnectorConfig.configDef() : SinkConnectorConfig.configDef(), connectorConfig, false);
    List<ConfigValue> configValues = new ArrayList<>();
    Map<String, ConfigKey> configKeys = new HashMap<>();
    List<String> allGroups = new ArrayList<>();
    // do basic connector validation (name, connector type, etc.)
    Map<String, ConfigValue> validatedConnectorConfig = validateBasicConnectorConfig(connector, connectorConfigDef, connectorConfig);
    configValues.addAll(validatedConnectorConfig.values());
    configKeys.putAll(connectorConfigDef.configKeys());
    allGroups.addAll(connectorConfigDef.groups());
    // do custom connector-specific validation
    Config config = connector.validate(connectorConfig);
    ConfigDef configDef = connector.config();
    configKeys.putAll(configDef.configKeys());
    allGroups.addAll(configDef.groups());
    configValues.addAll(config.configValues());
    return generateResult(connType, configKeys, configValues, allGroups);
}
Also used : Connector(org.apache.kafka.connect.connector.Connector) SourceConnector(org.apache.kafka.connect.source.SourceConnector) ConfigValue(org.apache.kafka.common.config.ConfigValue) ConfigKey(org.apache.kafka.common.config.ConfigDef.ConfigKey) HashMap(java.util.HashMap) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) Config(org.apache.kafka.common.config.Config) ArrayList(java.util.ArrayList) SourceConnector(org.apache.kafka.connect.source.SourceConnector) BadRequestException(org.apache.kafka.connect.runtime.rest.errors.BadRequestException) ConfigDef(org.apache.kafka.common.config.ConfigDef)

Aggregations

BadRequestException (org.apache.kafka.connect.runtime.rest.errors.BadRequestException)6 Config (org.apache.kafka.common.config.Config)4 ConfigDef (org.apache.kafka.common.config.ConfigDef)4 ConfigValue (org.apache.kafka.common.config.ConfigValue)4 Connector (org.apache.kafka.connect.connector.Connector)4 SourceConnector (org.apache.kafka.connect.source.SourceConnector)4 HashMap (java.util.HashMap)3 ConnectorConfig (org.apache.kafka.connect.runtime.ConnectorConfig)3 ConnectorFactory (org.apache.kafka.connect.runtime.ConnectorFactory)3 SinkConnectorConfig (org.apache.kafka.connect.runtime.SinkConnectorConfig)3 TaskConfig (org.apache.kafka.connect.runtime.TaskConfig)3 WorkerConfig (org.apache.kafka.connect.runtime.WorkerConfig)3 ConnectorInfo (org.apache.kafka.connect.runtime.rest.entities.ConnectorInfo)3 SinkConnector (org.apache.kafka.connect.sink.SinkConnector)3 Test (org.junit.Test)3 PrepareForTest (org.powermock.core.classloader.annotations.PrepareForTest)3 AlreadyExistsException (org.apache.kafka.connect.errors.AlreadyExistsException)2 ConnectException (org.apache.kafka.connect.errors.ConnectException)2 Herder (org.apache.kafka.connect.runtime.Herder)2 ConfigInfos (org.apache.kafka.connect.runtime.rest.entities.ConfigInfos)2