Search in sources :

Example 1 with ListTopicsOptions

use of org.apache.kafka.clients.admin.ListTopicsOptions in project kafka by apache.

the class WorkerUtils method getMatchingTopicPartitions.

/**
 * Returns list of existing, not internal, topics/partitions that match given pattern and
 * where partitions are in range [startPartition, endPartition]
 * @param adminClient     AdminClient
 * @param topicRegex      Topic regular expression to match
 * @return                List of topic names
 * @throws Throwable      If failed to get list of existing topics
 */
static Collection<TopicPartition> getMatchingTopicPartitions(Admin adminClient, String topicRegex, int startPartition, int endPartition) throws Throwable {
    final Pattern topicNamePattern = Pattern.compile(topicRegex);
    // first get list of matching topics
    List<String> matchedTopics = new ArrayList<>();
    ListTopicsResult res = adminClient.listTopics(new ListTopicsOptions().timeoutMs(ADMIN_REQUEST_TIMEOUT));
    Map<String, TopicListing> topicListingMap = res.namesToListings().get();
    for (Map.Entry<String, TopicListing> topicListingEntry : topicListingMap.entrySet()) {
        if (!topicListingEntry.getValue().isInternal() && topicNamePattern.matcher(topicListingEntry.getKey()).matches()) {
            matchedTopics.add(topicListingEntry.getKey());
        }
    }
    // create a list of topic/partitions
    List<TopicPartition> out = new ArrayList<>();
    DescribeTopicsResult topicsResult = adminClient.describeTopics(matchedTopics, new DescribeTopicsOptions().timeoutMs(ADMIN_REQUEST_TIMEOUT));
    Map<String, TopicDescription> topicDescriptionMap = topicsResult.allTopicNames().get();
    for (TopicDescription desc : topicDescriptionMap.values()) {
        List<TopicPartitionInfo> partitions = desc.partitions();
        for (TopicPartitionInfo info : partitions) {
            if ((info.partition() >= startPartition) && (info.partition() <= endPartition)) {
                out.add(new TopicPartition(desc.name(), info.partition()));
            }
        }
    }
    return out;
}
Also used : Pattern(java.util.regex.Pattern) ArrayList(java.util.ArrayList) TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TopicPartition(org.apache.kafka.common.TopicPartition) ListTopicsResult(org.apache.kafka.clients.admin.ListTopicsResult) TopicListing(org.apache.kafka.clients.admin.TopicListing) DescribeTopicsResult(org.apache.kafka.clients.admin.DescribeTopicsResult) DescribeTopicsOptions(org.apache.kafka.clients.admin.DescribeTopicsOptions) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) ListTopicsOptions(org.apache.kafka.clients.admin.ListTopicsOptions) HashMap(java.util.HashMap) Map(java.util.Map)

Example 2 with ListTopicsOptions

use of org.apache.kafka.clients.admin.ListTopicsOptions in project kafka by apache.

the class TransactionsCommandTest method expectListTopics.

private void expectListTopics(Set<String> topics) {
    ListTopicsResult result = Mockito.mock(ListTopicsResult.class);
    Mockito.when(result.names()).thenReturn(completedFuture(topics));
    ListTopicsOptions listOptions = new ListTopicsOptions().listInternal(true);
    Mockito.when(admin.listTopics(listOptions)).thenReturn(result);
}
Also used : ListTopicsResult(org.apache.kafka.clients.admin.ListTopicsResult) ListTopicsOptions(org.apache.kafka.clients.admin.ListTopicsOptions)

Aggregations

ListTopicsOptions (org.apache.kafka.clients.admin.ListTopicsOptions)2 ListTopicsResult (org.apache.kafka.clients.admin.ListTopicsResult)2 ArrayList (java.util.ArrayList)1 HashMap (java.util.HashMap)1 Map (java.util.Map)1 Pattern (java.util.regex.Pattern)1 DescribeTopicsOptions (org.apache.kafka.clients.admin.DescribeTopicsOptions)1 DescribeTopicsResult (org.apache.kafka.clients.admin.DescribeTopicsResult)1 TopicDescription (org.apache.kafka.clients.admin.TopicDescription)1 TopicListing (org.apache.kafka.clients.admin.TopicListing)1 TopicPartition (org.apache.kafka.common.TopicPartition)1 TopicPartitionInfo (org.apache.kafka.common.TopicPartitionInfo)1