Search in sources :

Example 6 with Config

use of org.apache.kafka.clients.admin.Config in project strimzi by strimzi.

the class BaseKafkaImpl method topicMetadata.

/**
 * Get a topic config via the Kafka AdminClient API, calling the given handler
 * (in a different thread) with the result.
 */
@Override
public void topicMetadata(TopicName topicName, Handler<AsyncResult<TopicMetadata>> handler) {
    LOGGER.debug("Getting metadata for topic {}", topicName);
    ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topicName.toString());
    KafkaFuture<TopicDescription> descriptionFuture = adminClient.describeTopics(Collections.singleton(topicName.toString())).values().get(topicName.toString());
    KafkaFuture<Config> configFuture = adminClient.describeConfigs(Collections.singleton(resource)).values().get(resource);
    queueWork(new MetadataWork(descriptionFuture, configFuture, result -> handler.handle(result)));
}
Also used : Config(org.apache.kafka.clients.admin.Config) Logger(org.slf4j.Logger) Vertx(io.vertx.core.Vertx) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) KafkaFuture(org.apache.kafka.common.KafkaFuture) Future(io.vertx.core.Future) AdminClient(org.apache.kafka.clients.admin.AdminClient) ExecutionException(java.util.concurrent.ExecutionException) ConfigResource(org.apache.kafka.common.config.ConfigResource) Map(java.util.Map) ListTopicsResult(org.apache.kafka.clients.admin.ListTopicsResult) UnknownTopicOrPartitionException(org.apache.kafka.common.errors.UnknownTopicOrPartitionException) AsyncResult(io.vertx.core.AsyncResult) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) Handler(io.vertx.core.Handler) Collections(java.util.Collections) Config(org.apache.kafka.clients.admin.Config) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) ConfigResource(org.apache.kafka.common.config.ConfigResource)

Example 7 with Config

use of org.apache.kafka.clients.admin.Config in project ksql by confluentinc.

the class KafkaTopicClientImplTest method topicConfigResponse.

private static DescribeConfigsResult topicConfigResponse(final String topicName, final ConfigEntry... entries) {
    final Map<ConfigResource, Config> config = ImmutableMap.of(new ConfigResource(ConfigResource.Type.TOPIC, topicName), new Config(Arrays.asList(entries)));
    final DescribeConfigsResult response = mock(DescribeConfigsResult.class);
    expect(response.all()).andReturn(KafkaFuture.completedFuture(config));
    replay(response);
    return response;
}
Also used : TopicConfig(org.apache.kafka.common.config.TopicConfig) Config(org.apache.kafka.clients.admin.Config) DescribeConfigsResult(org.apache.kafka.clients.admin.DescribeConfigsResult) ConfigResource(org.apache.kafka.common.config.ConfigResource)

Example 8 with Config

use of org.apache.kafka.clients.admin.Config in project ksql by confluentinc.

the class KafkaTopicClientImplTest method withResourceConfig.

/*
   * Config has broken hashCode & equals method:
   * https://issues.apache.org/jira/browse/KAFKA-6727
   */
private static Map<ConfigResource, Config> withResourceConfig(final ConfigResource resource, final ConfigEntry... entries) {
    final Set<ConfigEntry> expected = Arrays.stream(entries).collect(Collectors.toSet());
    class ConfigMatcher implements IArgumentMatcher {

        @SuppressWarnings("unchecked")
        @Override
        public boolean matches(final Object argument) {
            final Map<ConfigResource, Config> request = (Map<ConfigResource, Config>) argument;
            if (request.size() != 1) {
                return false;
            }
            final Config config = request.get(resource);
            if (config == null) {
                return false;
            }
            final Set<ConfigEntry> actual = new HashSet<>(config.entries());
            return actual.equals(expected);
        }

        @Override
        public void appendTo(final StringBuffer buffer) {
            buffer.append(resource).append("->").append("Config{").append(expected).append("}");
        }
    }
    EasyMock.reportMatcher(new ConfigMatcher());
    return null;
}
Also used : ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) TopicConfig(org.apache.kafka.common.config.TopicConfig) Config(org.apache.kafka.clients.admin.Config) IArgumentMatcher(org.easymock.IArgumentMatcher) EasyMock.anyObject(org.easymock.EasyMock.anyObject) Map(java.util.Map) ImmutableMap(com.google.common.collect.ImmutableMap) HashMap(java.util.HashMap) ConfigResource(org.apache.kafka.common.config.ConfigResource) HashSet(java.util.HashSet)

Example 9 with Config

use of org.apache.kafka.clients.admin.Config in project ksql by confluentinc.

the class KafkaTopicClientImplTest method describeBrokerResult.

private DescribeConfigsResult describeBrokerResult() {
    DescribeConfigsResult describeConfigsResult = mock(DescribeConfigsResult.class);
    ConfigEntry configEntryDeleteEnable = new ConfigEntry("delete.topic.enable", "true");
    List<ConfigEntry> configEntries = new ArrayList<>();
    configEntries.add(configEntryDeleteEnable);
    Map<ConfigResource, Config> config = ImmutableMap.of(new ConfigResource(ConfigResource.Type.BROKER, node.idString()), new Config(configEntries));
    expect(describeConfigsResult.all()).andReturn(KafkaFuture.completedFuture(config));
    replay(describeConfigsResult);
    return describeConfigsResult;
}
Also used : ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) TopicConfig(org.apache.kafka.common.config.TopicConfig) Config(org.apache.kafka.clients.admin.Config) ArrayList(java.util.ArrayList) DescribeConfigsResult(org.apache.kafka.clients.admin.DescribeConfigsResult) ConfigResource(org.apache.kafka.common.config.ConfigResource)

Example 10 with Config

use of org.apache.kafka.clients.admin.Config in project ksql by confluentinc.

the class KafkaTopicClientImpl method addTopicConfig.

@Override
public boolean addTopicConfig(final String topicName, final Map<String, ?> overrides) {
    final ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topicName);
    try {
        final Map<String, String> existingConfig = topicConfig(topicName, false);
        final boolean changed = overrides.entrySet().stream().anyMatch(e -> !Objects.equals(existingConfig.get(e.getKey()), e.getValue()));
        if (!changed) {
            return false;
        }
        existingConfig.putAll(toStringConfigs(overrides));
        final Set<ConfigEntry> entries = existingConfig.entrySet().stream().map(e -> new ConfigEntry(e.getKey(), e.getValue())).collect(Collectors.toSet());
        final Map<ConfigResource, Config> request = Collections.singletonMap(resource, new Config(entries));
        executeWithRetries(() -> adminClient.alterConfigs(request).all());
        return true;
    } catch (final Exception e) {
        throw new KafkaResponseGetFailedException("Failed to set config for Kafka Topic " + topicName, e);
    }
}
Also used : Config(org.apache.kafka.clients.admin.Config) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) LoggerFactory(org.slf4j.LoggerFactory) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) RetriableException(org.apache.kafka.common.errors.RetriableException) Supplier(java.util.function.Supplier) AdminClient(org.apache.kafka.clients.admin.AdminClient) ConfigResource(org.apache.kafka.common.config.ConfigResource) Lists(com.google.common.collect.Lists) Map(java.util.Map) DeleteTopicsResult(org.apache.kafka.clients.admin.DeleteTopicsResult) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) TopicConfig(org.apache.kafka.common.config.TopicConfig) Logger(org.slf4j.Logger) Collection(java.util.Collection) NewTopic(org.apache.kafka.clients.admin.NewTopic) Set(java.util.Set) KafkaFuture(org.apache.kafka.common.KafkaFuture) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException) List(java.util.List) Node(org.apache.kafka.common.Node) DescribeClusterResult(org.apache.kafka.clients.admin.DescribeClusterResult) Collections(java.util.Collections) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) Config(org.apache.kafka.clients.admin.Config) TopicConfig(org.apache.kafka.common.config.TopicConfig) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ConfigResource(org.apache.kafka.common.config.ConfigResource) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) RetriableException(org.apache.kafka.common.errors.RetriableException) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ExecutionException(java.util.concurrent.ExecutionException) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException)

Aggregations

Config (org.apache.kafka.clients.admin.Config)13 ConfigResource (org.apache.kafka.common.config.ConfigResource)10 Map (java.util.Map)7 ConfigEntry (org.apache.kafka.clients.admin.ConfigEntry)7 TopicConfig (org.apache.kafka.common.config.TopicConfig)6 ExecutionException (java.util.concurrent.ExecutionException)5 NewTopic (org.apache.kafka.clients.admin.NewTopic)5 Node (org.apache.kafka.common.Node)5 AdminClient (org.apache.kafka.clients.admin.AdminClient)4 TopicDescription (org.apache.kafka.clients.admin.TopicDescription)4 KafkaResponseGetFailedException (io.confluent.ksql.exception.KafkaResponseGetFailedException)3 KafkaTopicException (io.confluent.ksql.exception.KafkaTopicException)3 Collections (java.util.Collections)3 HashMap (java.util.HashMap)3 Set (java.util.Set)3 DescribeClusterResult (org.apache.kafka.clients.admin.DescribeClusterResult)3 KafkaFuture (org.apache.kafka.common.KafkaFuture)3 RetriableException (org.apache.kafka.common.errors.RetriableException)3 TopicExistsException (org.apache.kafka.common.errors.TopicExistsException)3 Logger (org.slf4j.Logger)3