Examples with ZkUtils kafka.utils.ZkUtils used on opensource projects

Example 56 with ZkUtils

use of kafka.utils.ZkUtils in project kafka-streams-examples by confluentinc.

the class KafkaEmbedded method createTopic.

/**
 * Create a Kafka topic with the given parameters.
 *
 * @param topic       The name of the topic.
 * @param partitions  The number of partitions for this topic.
 * @param replication The replication factor for (partitions of) this topic.
 * @param topicConfig Additional topic-level configuration settings.
 */
public void createTopic(String topic, int partitions, int replication, Properties topicConfig) {
    log.debug("Creating topic { name: {}, partitions: {}, replication: {}, config: {} }", topic, partitions, replication, topicConfig);
    // Note: You must initialize the ZkClient with ZKStringSerializer.  If you don't, then
    // createTopic() will only seem to work (it will return without error).  The topic will exist in
    // only ZooKeeper and will be returned when listing topics, but Kafka itself does not create the
    // topic.
    ZkClient zkClient = new ZkClient(zookeeperConnect(), DEFAULT_ZK_SESSION_TIMEOUT_MS, DEFAULT_ZK_CONNECTION_TIMEOUT_MS, ZKStringSerializer$.MODULE$);
    boolean isSecure = false;
    ZkUtils zkUtils = new ZkUtils(zkClient, new ZkConnection(zookeeperConnect()), isSecure);
    AdminUtils.createTopic(zkUtils, topic, partitions, replication, topicConfig, RackAwareMode.Enforced$.MODULE$);
    zkClient.close();
}

Example 57 with ZkUtils

use of kafka.utils.ZkUtils in project kafka-streams-examples by confluentinc.

the class KafkaEmbedded method deleteTopic.

/**
 * Delete a Kafka topic.
 *
 * @param topic The name of the topic.
 */
public void deleteTopic(String topic) {
    log.debug("Deleting topic {}", topic);
    ZkClient zkClient = new ZkClient(zookeeperConnect(), DEFAULT_ZK_SESSION_TIMEOUT_MS, DEFAULT_ZK_CONNECTION_TIMEOUT_MS, ZKStringSerializer$.MODULE$);
    boolean isSecure = false;
    ZkUtils zkUtils = new ZkUtils(zkClient, new ZkConnection(zookeeperConnect()), isSecure);
    AdminUtils.deleteTopic(zkUtils, topic);
    zkClient.close();
}

Example 58 with ZkUtils

use of kafka.utils.ZkUtils in project nakadi by zalando.

the class KafkaTestHelper method getTopicRetentionTime.

public static Long getTopicRetentionTime(final String topic, final String zkPath) {
    final ZkUtils zkUtils = ZkUtils.apply(zkPath, 30000, 10000, false);
    final Properties topicConfig = AdminUtils.fetchEntityConfig(zkUtils, ConfigType.Topic(), topic);
    return Long.valueOf(topicConfig.getProperty("retention.ms"));
}

Example 59 with ZkUtils

use of kafka.utils.ZkUtils in project incubator-gobblin by apache.

the class Kafka09DataWriter method provisionTopic.

private void provisionTopic(String topicName, Config config) {
    String zooKeeperPropKey = KafkaWriterConfigurationKeys.CLUSTER_ZOOKEEPER;
    if (!config.hasPath(zooKeeperPropKey)) {
        log.debug("Topic " + topicName + " is configured without the partition and replication");
        return;
    }
    String zookeeperConnect = config.getString(zooKeeperPropKey);
    int sessionTimeoutMs = ConfigUtils.getInt(config, KafkaWriterConfigurationKeys.ZOOKEEPER_SESSION_TIMEOUT, KafkaWriterConfigurationKeys.ZOOKEEPER_SESSION_TIMEOUT_DEFAULT);
    int connectionTimeoutMs = ConfigUtils.getInt(config, KafkaWriterConfigurationKeys.ZOOKEEPER_CONNECTION_TIMEOUT, KafkaWriterConfigurationKeys.ZOOKEEPER_CONNECTION_TIMEOUT_DEFAULT);
    // Note: You must initialize the ZkClient with ZKStringSerializer.  If you don't, then
    // createTopic() will only seem to work (it will return without error).  The topic will exist in
    // only ZooKeeper and will be returned when listing topics, but Kafka itself does not create the
    // topic.
    ZkClient zkClient = new ZkClient(zookeeperConnect, sessionTimeoutMs, connectionTimeoutMs, ZKStringSerializer$.MODULE$);
    // Security for Kafka was added in Kafka 0.9.0.0
    ZkUtils zkUtils = new ZkUtils(zkClient, new ZkConnection(zookeeperConnect), false);
    int partitions = ConfigUtils.getInt(config, KafkaWriterConfigurationKeys.PARTITION_COUNT, KafkaWriterConfigurationKeys.PARTITION_COUNT_DEFAULT);
    int replication = ConfigUtils.getInt(config, KafkaWriterConfigurationKeys.REPLICATION_COUNT, KafkaWriterConfigurationKeys.PARTITION_COUNT_DEFAULT);
    Properties topicConfig = new Properties();
    if (AdminUtils.topicExists(zkUtils, topicName)) {
        log.debug("Topic" + topicName + " already Exists with replication: " + replication + " and partitions :" + partitions);
        return;
    }
    try {
        AdminUtils.createTopic(zkUtils, topicName, partitions, replication, topicConfig);
    } catch (RuntimeException e) {
        throw new RuntimeException(e);
    }
    log.info("Created Topic " + topicName + " with replication: " + replication + " and partitions :" + partitions);
}

Example 60 with ZkUtils

use of kafka.utils.ZkUtils in project JavaForFun by gumartinm.

the class OffsetManagement method retrieveAllTopics.

private List<String> retrieveAllTopics() {
    List<String> allTopics = new LinkedList<>();
    ZkUtils zkUtils = null;
    try {
        zkUtils = ZkUtils.apply(options.valueOf(zookeeperOption), 30000, 30000, JaasUtils.isZkSecurityEnabled());
        allTopics.addAll(scala.collection.JavaConversions.seqAsJavaList(zkUtils.getAllTopics()));
    } finally {
        if (zkUtils != null) {
            zkUtils.close();
        }
    }
    return allTopics;
}