Search in sources :

Example 56 with ConfigResource

use of org.apache.kafka.common.config.ConfigResource in project kafka by apache.

the class InternalTopicManagerTest method shouldThrowWhenTopicDescriptionsDoNotContainTopicDuringValidation.

@Test
public void shouldThrowWhenTopicDescriptionsDoNotContainTopicDuringValidation() {
    final AdminClient admin = EasyMock.createNiceMock(AdminClient.class);
    final InternalTopicManager topicManager = new InternalTopicManager(time, admin, new StreamsConfig(config));
    final KafkaFutureImpl<TopicDescription> topicDescriptionSuccessfulFuture = new KafkaFutureImpl<>();
    topicDescriptionSuccessfulFuture.complete(new TopicDescription(topic1, false, Collections.singletonList(new TopicPartitionInfo(0, broker1, cluster, Collections.emptyList()))));
    EasyMock.expect(admin.describeTopics(Collections.singleton(topic1))).andStubAnswer(() -> new MockDescribeTopicsResult(mkMap(mkEntry(topic2, topicDescriptionSuccessfulFuture))));
    final KafkaFutureImpl<Config> topicConfigSuccessfulFuture = new KafkaFutureImpl<>();
    topicConfigSuccessfulFuture.complete(new Config(Collections.emptySet()));
    final ConfigResource topicResource = new ConfigResource(Type.TOPIC, topic1);
    EasyMock.expect(admin.describeConfigs(Collections.singleton(topicResource))).andStubAnswer(() -> new MockDescribeConfigsResult(mkMap(mkEntry(topicResource, topicConfigSuccessfulFuture))));
    EasyMock.replay(admin);
    final InternalTopicConfig internalTopicConfig = setupRepartitionTopicConfig(topic1, 1);
    assertThrows(IllegalStateException.class, () -> topicManager.validate(Collections.singletonMap(topic1, internalTopicConfig)));
}
Also used : TopicConfig(org.apache.kafka.common.config.TopicConfig) ConsumerConfig(org.apache.kafka.clients.consumer.ConsumerConfig) Config(org.apache.kafka.clients.admin.Config) StreamsConfig(org.apache.kafka.streams.StreamsConfig) ProducerConfig(org.apache.kafka.clients.producer.ProducerConfig) TopicMetadataAndConfig(org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) ConfigResource(org.apache.kafka.common.config.ConfigResource) TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) AdminClient(org.apache.kafka.clients.admin.AdminClient) MockAdminClient(org.apache.kafka.clients.admin.MockAdminClient) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Example 57 with ConfigResource

use of org.apache.kafka.common.config.ConfigResource in project kafka by apache.

the class InternalTopicManagerTest method shouldThrowWhenConfigDescriptionsDoNotContainTopicDuringValidation.

@Test
public void shouldThrowWhenConfigDescriptionsDoNotContainTopicDuringValidation() {
    final AdminClient admin = EasyMock.createNiceMock(AdminClient.class);
    final InternalTopicManager topicManager = new InternalTopicManager(time, admin, new StreamsConfig(config));
    final KafkaFutureImpl<TopicDescription> topicDescriptionSuccessfulFuture = new KafkaFutureImpl<>();
    topicDescriptionSuccessfulFuture.complete(new TopicDescription(topic1, false, Collections.singletonList(new TopicPartitionInfo(0, broker1, cluster, Collections.emptyList()))));
    EasyMock.expect(admin.describeTopics(Collections.singleton(topic1))).andStubAnswer(() -> new MockDescribeTopicsResult(mkMap(mkEntry(topic1, topicDescriptionSuccessfulFuture))));
    final KafkaFutureImpl<Config> topicConfigSuccessfulFuture = new KafkaFutureImpl<>();
    topicConfigSuccessfulFuture.complete(new Config(Collections.emptySet()));
    final ConfigResource topicResource1 = new ConfigResource(Type.TOPIC, topic1);
    final ConfigResource topicResource2 = new ConfigResource(Type.TOPIC, topic2);
    EasyMock.expect(admin.describeConfigs(Collections.singleton(topicResource1))).andStubAnswer(() -> new MockDescribeConfigsResult(mkMap(mkEntry(topicResource2, topicConfigSuccessfulFuture))));
    EasyMock.replay(admin);
    final InternalTopicConfig internalTopicConfig = setupRepartitionTopicConfig(topic1, 1);
    assertThrows(IllegalStateException.class, () -> topicManager.validate(Collections.singletonMap(topic1, internalTopicConfig)));
}
Also used : TopicConfig(org.apache.kafka.common.config.TopicConfig) ConsumerConfig(org.apache.kafka.clients.consumer.ConsumerConfig) Config(org.apache.kafka.clients.admin.Config) StreamsConfig(org.apache.kafka.streams.StreamsConfig) ProducerConfig(org.apache.kafka.clients.producer.ProducerConfig) TopicMetadataAndConfig(org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) ConfigResource(org.apache.kafka.common.config.ConfigResource) TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) AdminClient(org.apache.kafka.clients.admin.AdminClient) MockAdminClient(org.apache.kafka.clients.admin.MockAdminClient) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Example 58 with ConfigResource

use of org.apache.kafka.common.config.ConfigResource in project kafka by apache.

the class InternalTopicManagerTest method shouldThrowWhenConfigDescriptionsDoNotContainConfigDuringValidation.

private void shouldThrowWhenConfigDescriptionsDoNotContainConfigDuringValidation(final InternalTopicConfig streamsSideTopicConfig, final Config brokerSideTopicConfig) {
    final AdminClient admin = EasyMock.createNiceMock(AdminClient.class);
    final InternalTopicManager topicManager = new InternalTopicManager(time, admin, new StreamsConfig(config));
    final KafkaFutureImpl<TopicDescription> topicDescriptionSuccessfulFuture = new KafkaFutureImpl<>();
    topicDescriptionSuccessfulFuture.complete(new TopicDescription(topic1, false, Collections.singletonList(new TopicPartitionInfo(0, broker1, cluster, Collections.emptyList()))));
    EasyMock.expect(admin.describeTopics(Collections.singleton(topic1))).andStubAnswer(() -> new MockDescribeTopicsResult(mkMap(mkEntry(topic1, topicDescriptionSuccessfulFuture))));
    final KafkaFutureImpl<Config> topicConfigSuccessfulFuture = new KafkaFutureImpl<>();
    topicConfigSuccessfulFuture.complete(brokerSideTopicConfig);
    final ConfigResource topicResource1 = new ConfigResource(Type.TOPIC, topic1);
    EasyMock.expect(admin.describeConfigs(Collections.singleton(topicResource1))).andStubAnswer(() -> new MockDescribeConfigsResult(mkMap(mkEntry(topicResource1, topicConfigSuccessfulFuture))));
    EasyMock.replay(admin);
    assertThrows(IllegalStateException.class, () -> topicManager.validate(Collections.singletonMap(topic1, streamsSideTopicConfig)));
}
Also used : TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TopicConfig(org.apache.kafka.common.config.TopicConfig) ConsumerConfig(org.apache.kafka.clients.consumer.ConsumerConfig) Config(org.apache.kafka.clients.admin.Config) StreamsConfig(org.apache.kafka.streams.StreamsConfig) ProducerConfig(org.apache.kafka.clients.producer.ProducerConfig) TopicMetadataAndConfig(org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) ConfigResource(org.apache.kafka.common.config.ConfigResource) AdminClient(org.apache.kafka.clients.admin.AdminClient) MockAdminClient(org.apache.kafka.clients.admin.MockAdminClient) StreamsConfig(org.apache.kafka.streams.StreamsConfig)

Example 59 with ConfigResource

use of org.apache.kafka.common.config.ConfigResource in project kafka by apache.

the class InternalTopicManagerTest method shouldOnlyRetryNotSuccessfulFuturesDuringValidation.

@Test
public void shouldOnlyRetryNotSuccessfulFuturesDuringValidation() {
    final AdminClient admin = EasyMock.createNiceMock(AdminClient.class);
    final InternalTopicManager topicManager = new InternalTopicManager(time, admin, new StreamsConfig(config));
    final KafkaFutureImpl<TopicDescription> topicDescriptionFailFuture = new KafkaFutureImpl<>();
    topicDescriptionFailFuture.completeExceptionally(new LeaderNotAvailableException("Leader Not Available!"));
    final KafkaFutureImpl<TopicDescription> topicDescriptionSuccessfulFuture1 = new KafkaFutureImpl<>();
    topicDescriptionSuccessfulFuture1.complete(new TopicDescription(topic1, false, Collections.singletonList(new TopicPartitionInfo(0, broker1, cluster, Collections.emptyList()))));
    final KafkaFutureImpl<TopicDescription> topicDescriptionSuccessfulFuture2 = new KafkaFutureImpl<>();
    topicDescriptionSuccessfulFuture2.complete(new TopicDescription(topic2, false, Collections.singletonList(new TopicPartitionInfo(0, broker1, cluster, Collections.emptyList()))));
    EasyMock.expect(admin.describeTopics(mkSet(topic1, topic2))).andAnswer(() -> new MockDescribeTopicsResult(mkMap(mkEntry(topic1, topicDescriptionSuccessfulFuture1), mkEntry(topic2, topicDescriptionFailFuture))));
    EasyMock.expect(admin.describeTopics(mkSet(topic2))).andAnswer(() -> new MockDescribeTopicsResult(mkMap(mkEntry(topic2, topicDescriptionSuccessfulFuture2))));
    final KafkaFutureImpl<Config> topicConfigSuccessfulFuture = new KafkaFutureImpl<>();
    topicConfigSuccessfulFuture.complete(new Config(repartitionTopicConfig().entrySet().stream().map(entry -> new ConfigEntry(entry.getKey(), entry.getValue())).collect(Collectors.toSet())));
    final ConfigResource topicResource1 = new ConfigResource(Type.TOPIC, topic1);
    final ConfigResource topicResource2 = new ConfigResource(Type.TOPIC, topic2);
    EasyMock.expect(admin.describeConfigs(mkSet(topicResource1, topicResource2))).andAnswer(() -> new MockDescribeConfigsResult(mkMap(mkEntry(topicResource1, topicConfigSuccessfulFuture), mkEntry(topicResource2, topicConfigSuccessfulFuture))));
    EasyMock.replay(admin);
    final InternalTopicConfig internalTopicConfig1 = setupRepartitionTopicConfig(topic1, 1);
    final InternalTopicConfig internalTopicConfig2 = setupRepartitionTopicConfig(topic2, 1);
    final ValidationResult validationResult = topicManager.validate(mkMap(mkEntry(topic1, internalTopicConfig1), mkEntry(topic2, internalTopicConfig2)));
    assertThat(validationResult.missingTopics(), empty());
    assertThat(validationResult.misconfigurationsForTopics(), anEmptyMap());
    EasyMock.verify(admin);
}
Also used : MockTime(org.apache.kafka.common.utils.MockTime) DescribeConfigsResult(org.apache.kafka.clients.admin.DescribeConfigsResult) Matchers.not(org.hamcrest.Matchers.not) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) StreamsException(org.apache.kafka.streams.errors.StreamsException) ValidationResult(org.apache.kafka.streams.processor.internals.InternalTopicManager.ValidationResult) Matchers.hasKey(org.hamcrest.Matchers.hasKey) AdminClient(org.apache.kafka.clients.admin.AdminClient) Utils.mkMap(org.apache.kafka.common.utils.Utils.mkMap) After(org.junit.After) Map(java.util.Map) DeleteTopicsResult(org.apache.kafka.clients.admin.DeleteTopicsResult) Assert.fail(org.junit.Assert.fail) TopicConfig(org.apache.kafka.common.config.TopicConfig) CreatableReplicaAssignmentCollection(org.apache.kafka.common.message.CreateTopicsRequestData.CreatableReplicaAssignmentCollection) Collection(java.util.Collection) Utils.mkSet(org.apache.kafka.common.utils.Utils.mkSet) Set(java.util.Set) ConsumerConfig(org.apache.kafka.clients.consumer.ConsumerConfig) KafkaFuture(org.apache.kafka.common.KafkaFuture) Collectors(java.util.stream.Collectors) MockAdminClient(org.apache.kafka.clients.admin.MockAdminClient) TopicExistsException(org.apache.kafka.common.errors.TopicExistsException) List(java.util.List) Utils.mkEntry(org.apache.kafka.common.utils.Utils.mkEntry) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Optional(java.util.Optional) Node(org.apache.kafka.common.Node) UnknownTopicOrPartitionException(org.apache.kafka.common.errors.UnknownTopicOrPartitionException) Matchers.is(org.hamcrest.Matchers.is) CreateTopicsOptions(org.apache.kafka.clients.admin.CreateTopicsOptions) Type(org.apache.kafka.common.config.ConfigResource.Type) Config(org.apache.kafka.clients.admin.Config) Uuid(org.apache.kafka.common.Uuid) StreamsConfig(org.apache.kafka.streams.StreamsConfig) CreateTopicsRequest(org.apache.kafka.common.requests.CreateTopicsRequest) Assert.assertThrows(org.junit.Assert.assertThrows) HashMap(java.util.HashMap) LeaderNotAvailableException(org.apache.kafka.common.errors.LeaderNotAvailableException) CreatableTopic(org.apache.kafka.common.message.CreateTopicsRequestData.CreatableTopic) ArrayList(java.util.ArrayList) ConfigResource(org.apache.kafka.common.config.ConfigResource) DescribeTopicsResult(org.apache.kafka.clients.admin.DescribeTopicsResult) CreateTopicsResult(org.apache.kafka.clients.admin.CreateTopicsResult) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) ProducerConfig(org.apache.kafka.clients.producer.ProducerConfig) Before(org.junit.Before) TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TimeoutException(org.apache.kafka.common.errors.TimeoutException) Matchers.empty(org.hamcrest.Matchers.empty) NewTopic(org.apache.kafka.clients.admin.NewTopic) Test(org.junit.Test) EasyMock(org.easymock.EasyMock) TopicMetadataAndConfig(org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig) CreatableTopicCollection(org.apache.kafka.common.message.CreateTopicsRequestData.CreatableTopicCollection) CreateTopicsRequestData(org.apache.kafka.common.message.CreateTopicsRequestData) Matchers.hasItem(org.hamcrest.Matchers.hasItem) Assert.assertNull(org.junit.Assert.assertNull) LogCaptureAppender(org.apache.kafka.streams.processor.internals.testutil.LogCaptureAppender) UnsupportedVersionException(org.apache.kafka.common.errors.UnsupportedVersionException) Matchers.anEmptyMap(org.hamcrest.Matchers.anEmptyMap) Collections(java.util.Collections) Assert.assertEquals(org.junit.Assert.assertEquals) TopicConfig(org.apache.kafka.common.config.TopicConfig) ConsumerConfig(org.apache.kafka.clients.consumer.ConsumerConfig) Config(org.apache.kafka.clients.admin.Config) StreamsConfig(org.apache.kafka.streams.StreamsConfig) ProducerConfig(org.apache.kafka.clients.producer.ProducerConfig) TopicMetadataAndConfig(org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig) LeaderNotAvailableException(org.apache.kafka.common.errors.LeaderNotAvailableException) KafkaFutureImpl(org.apache.kafka.common.internals.KafkaFutureImpl) ValidationResult(org.apache.kafka.streams.processor.internals.InternalTopicManager.ValidationResult) ConfigResource(org.apache.kafka.common.config.ConfigResource) ConfigEntry(org.apache.kafka.clients.admin.ConfigEntry) TopicPartitionInfo(org.apache.kafka.common.TopicPartitionInfo) TopicDescription(org.apache.kafka.clients.admin.TopicDescription) AdminClient(org.apache.kafka.clients.admin.AdminClient) MockAdminClient(org.apache.kafka.clients.admin.MockAdminClient) StreamsConfig(org.apache.kafka.streams.StreamsConfig) Test(org.junit.Test)

Example 60 with ConfigResource

use of org.apache.kafka.common.config.ConfigResource in project kafka by apache.

the class ReplicationControlManagerTest method assertEmptyTopicConfigs.

private void assertEmptyTopicConfigs(ReplicationControlTestContext ctx, String topic) {
    Map<String, String> configs = ctx.configurationControl.getConfigs(new ConfigResource(ConfigResource.Type.TOPIC, topic));
    assertEquals(Collections.emptyMap(), configs);
}
Also used : ConfigResource(org.apache.kafka.common.config.ConfigResource)

Aggregations

ConfigResource (org.apache.kafka.common.config.ConfigResource)64 HashMap (java.util.HashMap)32 Config (org.apache.kafka.clients.admin.Config)23 Map (java.util.Map)22 KafkaFuture (org.apache.kafka.common.KafkaFuture)20 KafkaFutureImpl (org.apache.kafka.common.internals.KafkaFutureImpl)20 TopicConfig (org.apache.kafka.common.config.TopicConfig)18 ArrayList (java.util.ArrayList)17 ConfigEntry (org.apache.kafka.clients.admin.ConfigEntry)16 Test (org.junit.Test)15 Collection (java.util.Collection)14 TopicDescription (org.apache.kafka.clients.admin.TopicDescription)13 ConsumerConfig (org.apache.kafka.clients.consumer.ConsumerConfig)13 Node (org.apache.kafka.common.Node)13 AdminClient (org.apache.kafka.clients.admin.AdminClient)12 ProducerConfig (org.apache.kafka.clients.producer.ProducerConfig)12 Collections (java.util.Collections)11 Collectors (java.util.stream.Collectors)11 TopicMetadataAndConfig (org.apache.kafka.clients.admin.CreateTopicsResult.TopicMetadataAndConfig)11 StreamsConfig (org.apache.kafka.streams.StreamsConfig)11