Search in sources :

Example 1 with CONNECT

use of io.strimzi.systemtest.Constants.CONNECT in project strimzi by strimzi.

the class MetricsIsolatedST method testKafkaConnectResponse.

@ParallelTest
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectResponse() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectResponse = Pattern.compile("kafka_connect_node_response_total\\{clientid=\".*\",.*} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectResponse, kafkaConnectMetricsData);
    assertThat("KafkaConnect response count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 2 with CONNECT

use of io.strimzi.systemtest.Constants.CONNECT in project strimzi by strimzi.

the class MetricsIsolatedST method testKafkaConnectRequests.

@ParallelTest
@Tag(ACCEPTANCE)
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectRequests() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectRequests = Pattern.compile("kafka_connect_node_request_total\\{clientid=\".*\",} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectRequests, kafkaConnectMetricsData);
    assertThat("KafkaConnect requests count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 3 with CONNECT

use of io.strimzi.systemtest.Constants.CONNECT in project strimzi-kafka-operator by strimzi.

the class ConnectBuilderIsolatedST method testBuildFailsWithWrongChecksumOfArtifact.

@ParallelTest
void testBuildFailsWithWrongChecksumOfArtifact(ExtensionContext extensionContext) {
    String connectClusterName = mapWithClusterNames.get(extensionContext.getDisplayName()) + "-connect";
    String kafkaClientsName = mapWithKafkaClientNames.get(extensionContext.getDisplayName());
    final String imageName = getImageNameForTestCase();
    Plugin pluginWithWrongChecksum = new PluginBuilder().withName("connector-with-wrong-checksum").withArtifacts(new JarArtifactBuilder().withUrl(ECHO_SINK_JAR_URL).withSha512sum(ECHO_SINK_JAR_WRONG_CHECKSUM).build()).build();
    resourceManager.createResource(extensionContext, KafkaClientsTemplates.kafkaClients(false, kafkaClientsName).build());
    String kafkaClientsPodName = kubeClient(INFRA_NAMESPACE).listPodsByPrefixInName(kafkaClientsName).get(0).getMetadata().getName();
    resourceManager.createResource(extensionContext, false, KafkaConnectTemplates.kafkaConnect(extensionContext, connectClusterName, INFRA_NAMESPACE, INFRA_NAMESPACE, 1).editMetadata().addToAnnotations(Annotations.STRIMZI_IO_USE_CONNECTOR_RESOURCES, "true").endMetadata().editOrNewSpec().withNewBuild().withPlugins(pluginWithWrongChecksum).withNewDockerOutput().withImage(imageName).endDockerOutput().endBuild().endSpec().build());
    KafkaConnectUtils.waitForConnectNotReady(connectClusterName);
    KafkaConnectUtils.waitUntilKafkaConnectStatusConditionContainsMessage(connectClusterName, INFRA_NAMESPACE, "The Kafka Connect build failed(.*)?");
    LOGGER.info("Checking if KafkaConnect status condition contains message about build failure");
    KafkaConnect kafkaConnect = KafkaConnectResource.kafkaConnectClient().inNamespace(INFRA_NAMESPACE).withName(connectClusterName).get();
    LOGGER.info("Deploying network policies for KafkaConnect");
    NetworkPolicyResource.deployNetworkPolicyForResource(extensionContext, kafkaConnect, KafkaConnectResources.deploymentName(connectClusterName));
    Condition connectCondition = kafkaConnect.getStatus().getConditions().stream().findFirst().orElseThrow();
    assertTrue(connectCondition.getMessage().matches("The Kafka Connect build failed(.*)?"));
    assertThat(connectCondition.getType(), is(NotReady.toString()));
    LOGGER.info("Replacing plugin's checksum with right one");
    KafkaConnectResource.replaceKafkaConnectResource(connectClusterName, kC -> {
        Plugin pluginWithRightChecksum = new PluginBuilder().withName("connector-with-right-checksum").withArtifacts(new JarArtifactBuilder().withUrl(ECHO_SINK_JAR_URL).withSha512sum(ECHO_SINK_JAR_CHECKSUM).build()).build();
        kC.getSpec().getBuild().getPlugins().remove(0);
        kC.getSpec().getBuild().getPlugins().add(pluginWithRightChecksum);
    });
    KafkaConnectUtils.waitForConnectReady(connectClusterName);
    LOGGER.info("Checking if KafkaConnect API contains EchoSink connector");
    String plugins = cmdKubeClient().execInPod(kafkaClientsPodName, "curl", "-X", "GET", "http://" + KafkaConnectResources.serviceName(connectClusterName) + ":8083/connector-plugins").out();
    assertTrue(plugins.contains(ECHO_SINK_CLASS_NAME));
    LOGGER.info("Checking if KafkaConnect resource contains EchoSink connector in status");
    kafkaConnect = KafkaConnectResource.kafkaConnectClient().inNamespace(INFRA_NAMESPACE).withName(connectClusterName).get();
    assertTrue(kafkaConnect.getStatus().getConnectorPlugins().stream().anyMatch(connectorPlugin -> connectorPlugin.getConnectorClass().contains(ECHO_SINK_CLASS_NAME)));
}
Also used : Condition(io.strimzi.api.kafka.model.status.Condition) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Assertions.assertNotEquals(org.junit.jupiter.api.Assertions.assertNotEquals) Annotations(io.strimzi.operator.common.Annotations) Random(java.util.Random) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnectResource(io.strimzi.systemtest.resources.crd.KafkaConnectResource) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) Assertions.assertFalse(org.junit.jupiter.api.Assertions.assertFalse) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) OtherArtifactBuilder(io.strimzi.api.kafka.model.connect.build.OtherArtifactBuilder) KafkaConnectUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaConnectUtils) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaClients(io.strimzi.systemtest.kafkaclients.internalClients.KafkaClients) Collectors(java.util.stream.Collectors) ClientUtils(io.strimzi.systemtest.utils.ClientUtils) Labels(io.strimzi.operator.common.model.Labels) Logger(org.apache.logging.log4j.Logger) Plugin(io.strimzi.api.kafka.model.connect.build.Plugin) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) KafkaConnectorTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectorTemplates) Assertions.assertTrue(org.junit.jupiter.api.Assertions.assertTrue) Matchers.is(org.hamcrest.Matchers.is) Condition(io.strimzi.api.kafka.model.status.Condition) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OpenShiftOnly(io.strimzi.systemtest.annotations.OpenShiftOnly) HashMap(java.util.HashMap) PluginBuilder(io.strimzi.api.kafka.model.connect.build.PluginBuilder) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ZipArtifactBuilder(io.strimzi.api.kafka.model.connect.build.ZipArtifactBuilder) KafkaConnectorResource(io.strimzi.systemtest.resources.crd.KafkaConnectorResource) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) NotReady(io.strimzi.systemtest.enums.CustomResourceStatus.NotReady) Service(io.fabric8.kubernetes.api.model.Service) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) TgzArtifactBuilder(io.strimzi.api.kafka.model.connect.build.TgzArtifactBuilder) ImageStream(io.fabric8.openshift.api.model.ImageStream) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) Ready(io.strimzi.systemtest.enums.CustomResourceStatus.Ready) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) Constants(io.strimzi.systemtest.Constants) KafkaClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.KafkaClientsBuilder) OpenShiftClient(io.fabric8.openshift.client.OpenShiftClient) JarArtifactBuilder(io.strimzi.api.kafka.model.connect.build.JarArtifactBuilder) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) Util(io.strimzi.operator.common.Util) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ImageStreamBuilder(io.fabric8.openshift.api.model.ImageStreamBuilder) MavenArtifactBuilder(io.strimzi.api.kafka.model.connect.build.MavenArtifactBuilder) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) KafkaConnectResources(io.strimzi.api.kafka.model.KafkaConnectResources) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) PluginBuilder(io.strimzi.api.kafka.model.connect.build.PluginBuilder) Plugin(io.strimzi.api.kafka.model.connect.build.Plugin) JarArtifactBuilder(io.strimzi.api.kafka.model.connect.build.JarArtifactBuilder) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest)

Example 4 with CONNECT

use of io.strimzi.systemtest.Constants.CONNECT in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testKafkaConnectResponse.

@ParallelTest
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectResponse() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectResponse = Pattern.compile("kafka_connect_node_response_total\\{clientid=\".*\",.*} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectResponse, kafkaConnectMetricsData);
    assertThat("KafkaConnect response count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 5 with CONNECT

use of io.strimzi.systemtest.Constants.CONNECT in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testKafkaConnectIoNetwork.

@ParallelTest
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectIoNetwork() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectIoNetwork = Pattern.compile("kafka_connect_network_io_total\\{clientid=\".*\",} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectIoNetwork, kafkaConnectMetricsData);
    assertThat("KafkaConnect IO network count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Aggregations

KafkaConnect (io.strimzi.api.kafka.model.KafkaConnect)12 KafkaConnector (io.strimzi.api.kafka.model.KafkaConnector)12 KafkaResources (io.strimzi.api.kafka.model.KafkaResources)12 AbstractST (io.strimzi.systemtest.AbstractST)12 Constants (io.strimzi.systemtest.Constants)12 CONNECT (io.strimzi.systemtest.Constants.CONNECT)12 CONNECT_COMPONENTS (io.strimzi.systemtest.Constants.CONNECT_COMPONENTS)12 INFRA_NAMESPACE (io.strimzi.systemtest.Constants.INFRA_NAMESPACE)12 REGRESSION (io.strimzi.systemtest.Constants.REGRESSION)12 IsolatedSuite (io.strimzi.systemtest.annotations.IsolatedSuite)12 ParallelTest (io.strimzi.systemtest.annotations.ParallelTest)12 InternalKafkaClient (io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient)12 NetworkPolicyResource (io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource)12 KafkaClientsTemplates (io.strimzi.systemtest.templates.crd.KafkaClientsTemplates)12 KafkaConnectTemplates (io.strimzi.systemtest.templates.crd.KafkaConnectTemplates)12 KafkaTemplates (io.strimzi.systemtest.templates.crd.KafkaTemplates)12 KafkaTopicTemplates (io.strimzi.systemtest.templates.crd.KafkaTopicTemplates)12 KafkaTopicUtils (io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils)12 DeploymentUtils (io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils)12 PodUtils (io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils)12