Search in sources :

Example 1 with KafkaResponseGetFailedException

use of io.confluent.ksql.exception.KafkaResponseGetFailedException in project ksql by confluentinc.

the class KafkaTopicClientImpl method createTopic.

@Override
public void createTopic(final String topic, final int numPartitions, final short replicationFactor, final Map<String, ?> configs) {
    if (isTopicExists(topic)) {
        validateTopicProperties(topic, numPartitions, replicationFactor);
        return;
    }
    final NewTopic newTopic = new NewTopic(topic, numPartitions, replicationFactor);
    newTopic.configs(toStringConfigs(configs));
    try {
        log.info("Creating topic '{}'", topic);
        executeWithRetries(() -> adminClient.createTopics(Collections.singleton(newTopic)).all());
    } catch (final InterruptedException e) {
        throw new KafkaResponseGetFailedException("Failed to guarantee existence of topic " + topic, e);
    } catch (final TopicExistsException e) {
        // if the topic already exists, it is most likely because another node just created it.
        // ensure that it matches the partition count and replication factor before returning
        // success
        validateTopicProperties(topic, numPartitions, replicationFactor);
    } catch (final Exception e) {
        throw new KafkaResponseGetFailedException("Failed to guarantee existence of topic " + topic, e);
    }
}
Also used : KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) NewTopic(org.apache.kafka.clients.admin.NewTopic) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) RetriableException(org.apache.kafka.common.errors.RetriableException) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ExecutionException(java.util.concurrent.ExecutionException) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException)

Example 2 with KafkaResponseGetFailedException

use of io.confluent.ksql.exception.KafkaResponseGetFailedException in project ksql by confluentinc.

the class KafkaTopicClientImpl method addTopicConfig.

@Override
public boolean addTopicConfig(final String topicName, final Map<String, ?> overrides) {
    final ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topicName);
    try {
        final Map<String, String> existingConfig = topicConfig(topicName, false);
        final boolean changed = overrides.entrySet().stream().anyMatch(e -> !Objects.equals(existingConfig.get(e.getKey()), e.getValue()));
        if (!changed) {
            return false;
        }
        existingConfig.putAll(toStringConfigs(overrides));
        final Set<ConfigEntry> entries = existingConfig.entrySet().stream().map(e -> new ConfigEntry(e.getKey(), e.getValue())).collect(Collectors.toSet());
        final Map<ConfigResource, Config> request = Collections.singletonMap(resource, new Config(entries));
        executeWithRetries(() -> adminClient.alterConfigs(request).all());
        return true;
    } catch (final Exception e) {
        throw new KafkaResponseGetFailedException("Failed to set config for Kafka Topic " + topicName, e);
    }
}
Also used : Config(org.apache.kafka.clients.admin.Config) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) LoggerFactory(org.slf4j.LoggerFactory) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) RetriableException(org.apache.kafka.common.errors.RetriableException) Supplier(java.util.function.Supplier) AdminClient(org.apache.kafka.clients.admin.AdminClient) ConfigResource(org.apache.kafka.common.config.ConfigResource) Lists(com.google.common.collect.Lists) Map(java.util.Map) DeleteTopicsResult(org.apache.kafka.clients.admin.DeleteTopicsResult) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) TopicConfig(org.apache.kafka.common.config.TopicConfig) Logger(org.slf4j.Logger) Collection(java.util.Collection) NewTopic(org.apache.kafka.clients.admin.NewTopic) Set(java.util.Set) KafkaFuture(org.apache.kafka.common.KafkaFuture) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException) List(java.util.List) Node(org.apache.kafka.common.Node) DescribeClusterResult(org.apache.kafka.clients.admin.DescribeClusterResult) Collections(java.util.Collections) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) Config(org.apache.kafka.clients.admin.Config) TopicConfig(org.apache.kafka.common.config.TopicConfig) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ConfigResource(org.apache.kafka.common.config.ConfigResource) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) RetriableException(org.apache.kafka.common.errors.RetriableException) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ExecutionException(java.util.concurrent.ExecutionException) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException)

Example 3 with KafkaResponseGetFailedException

use of io.confluent.ksql.exception.KafkaResponseGetFailedException in project ksql by confluentinc.

the class KafkaTopicClientImpl method topicConfig.

private Map<String, String> topicConfig(final String topicName, final boolean includeDefaults) {
    final ConfigResource resource = new ConfigResource(ConfigResource.Type.TOPIC, topicName);
    final List<ConfigResource> request = Collections.singletonList(resource);
    try {
        final Config config = executeWithRetries(() -> adminClient.describeConfigs(request).all()).get(resource);
        return config.entries().stream().filter(e -> includeDefaults || e.source().equals(ConfigEntry.ConfigSource.DYNAMIC_TOPIC_CONFIG)).collect(Collectors.toMap(ConfigEntry::name, ConfigEntry::value));
    } catch (final Exception e) {
        throw new KafkaResponseGetFailedException("Failed to get config for Kafka Topic " + topicName, e);
    }
}
Also used : Config(org.apache.kafka.clients.admin.Config) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) LoggerFactory(org.slf4j.LoggerFactory) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) RetriableException(org.apache.kafka.common.errors.RetriableException) Supplier(java.util.function.Supplier) AdminClient(org.apache.kafka.clients.admin.AdminClient) ConfigResource(org.apache.kafka.common.config.ConfigResource) Lists(com.google.common.collect.Lists) Map(java.util.Map) DeleteTopicsResult(org.apache.kafka.clients.admin.DeleteTopicsResult) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) TopicConfig(org.apache.kafka.common.config.TopicConfig) Logger(org.slf4j.Logger) Collection(java.util.Collection) NewTopic(org.apache.kafka.clients.admin.NewTopic) Set(java.util.Set) KafkaFuture(org.apache.kafka.common.KafkaFuture) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) ExecutionException(java.util.concurrent.ExecutionException) TimeUnit(java.util.concurrent.TimeUnit) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException) List(java.util.List) Node(org.apache.kafka.common.Node) DescribeClusterResult(org.apache.kafka.clients.admin.DescribeClusterResult) Collections(java.util.Collections) Config(org.apache.kafka.clients.admin.Config) TopicConfig(org.apache.kafka.common.config.TopicConfig) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ConfigResource(org.apache.kafka.common.config.ConfigResource) KafkaTopicException(io.confluent.ksql.exception.KafkaTopicException) RetriableException(org.apache.kafka.common.errors.RetriableException) KafkaResponseGetFailedException(io.confluent.ksql.exception.KafkaResponseGetFailedException) ExecutionException(java.util.concurrent.ExecutionException) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException)

Aggregations

KafkaResponseGetFailedException (io.confluent.ksql.exception.KafkaResponseGetFailedException)3 KafkaTopicException (io.confluent.ksql.exception.KafkaTopicException)3 ExecutionException (java.util.concurrent.ExecutionException)3 NewTopic (org.apache.kafka.clients.admin.NewTopic)3 RetriableException (org.apache.kafka.common.errors.RetriableException)3 TopicExistsException (org.apache.kafka.common.errors.TopicExistsException)3 Lists (com.google.common.collect.Lists)2 Collection (java.util.Collection)2 Collections (java.util.Collections)2 List (java.util.List)2 Map (java.util.Map)2 Objects (java.util.Objects)2 Set (java.util.Set)2 TimeUnit (java.util.concurrent.TimeUnit)2 Supplier (java.util.function.Supplier)2 Collectors (java.util.stream.Collectors)2 AdminClient (org.apache.kafka.clients.admin.AdminClient)2 Config (org.apache.kafka.clients.admin.Config)2 ConfigEntry (org.apache.kafka.clients.admin.ConfigEntry)2 DeleteTopicsResult (org.apache.kafka.clients.admin.DeleteTopicsResult)2