Search in sources :

Example 66 with ParallelTest

use of io.strimzi.systemtest.annotations.ParallelTest in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testKafkaConnectResponse.

@ParallelTest
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectResponse() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectResponse = Pattern.compile("kafka_connect_node_response_total\\{clientid=\".*\",.*} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectResponse, kafkaConnectMetricsData);
    assertThat("KafkaConnect response count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 67 with ParallelTest

use of io.strimzi.systemtest.annotations.ParallelTest in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testClusterOperatorMetrics.

@ParallelTest
@Tag(ACCEPTANCE)
void testClusterOperatorMetrics(ExtensionContext extensionContext) {
    clusterOperatorMetricsData = collector.toBuilder().withNamespaceName(clusterOperator.getDeploymentNamespace()).withComponentType(ComponentType.ClusterOperator).withComponentName("").build().collectMetricsFromPods();
    List<String> resourcesList = Arrays.asList("Kafka", "KafkaBridge", "KafkaConnect", "KafkaConnector", "KafkaMirrorMaker", "KafkaMirrorMaker2", "KafkaRebalance");
    for (String resource : resourcesList) {
        assertCoMetricNotNull("strimzi_reconciliations_periodical_total", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_duration_seconds_count", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_duration_seconds_sum", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_duration_seconds_max", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_locked_total", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_successful_total", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_periodical_total", resource, clusterOperatorMetricsData);
        assertCoMetricNotNull("strimzi_reconciliations_failed_total", resource, clusterOperatorMetricsData);
    }
    assertCoMetricResources(Kafka.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 1);
    assertCoMetricResources(Kafka.RESOURCE_KIND, SECOND_NAMESPACE, 1);
    assertCoMetricResourceState(Kafka.RESOURCE_KIND, metricsClusterName, clusterOperator.getDeploymentNamespace(), 1, "none");
    assertCoMetricResourceState(Kafka.RESOURCE_KIND, SECOND_CLUSTER, SECOND_NAMESPACE, 1, "none");
    assertCoMetricResources(KafkaBridge.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 1);
    assertCoMetricResources(KafkaBridge.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceState(KafkaBridge.RESOURCE_KIND, BRIDGE_CLUSTER, clusterOperator.getDeploymentNamespace(), 1, "none");
    assertCoMetricResources(KafkaConnect.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 1);
    assertCoMetricResources(KafkaConnect.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceState(KafkaConnect.RESOURCE_KIND, metricsClusterName, clusterOperator.getDeploymentNamespace(), 1, "none");
    assertCoMetricResources(KafkaMirrorMaker.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 0);
    assertCoMetricResources(KafkaMirrorMaker.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceStateNotExists(KafkaMirrorMaker.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), metricsClusterName);
    assertCoMetricResources(KafkaMirrorMaker2.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 1);
    assertCoMetricResources(KafkaMirrorMaker2.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceState(KafkaMirrorMaker2.RESOURCE_KIND, MIRROR_MAKER_CLUSTER, clusterOperator.getDeploymentNamespace(), 1, "none");
    assertCoMetricResources(KafkaConnector.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 0);
    assertCoMetricResources(KafkaConnector.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceStateNotExists(KafkaConnector.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), metricsClusterName);
    assertCoMetricResources(KafkaRebalance.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), 0);
    assertCoMetricResources(KafkaRebalance.RESOURCE_KIND, SECOND_NAMESPACE, 0);
    assertCoMetricResourceStateNotExists(KafkaRebalance.RESOURCE_KIND, clusterOperator.getDeploymentNamespace(), metricsClusterName);
}
Also used : Matchers.emptyString(org.hamcrest.Matchers.emptyString) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 68 with ParallelTest

use of io.strimzi.systemtest.annotations.ParallelTest in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testCruiseControlMetrics.

@ParallelTest
@Tag(ACCEPTANCE)
void testCruiseControlMetrics() {
    String cruiseControlMetrics = CruiseControlUtils.callApi(INFRA_NAMESPACE, CruiseControlUtils.SupportedHttpMethods.GET, "/metrics");
    Matcher regex = Pattern.compile("^([^#].*)\\s+([^\\s]*)$", Pattern.MULTILINE).matcher(cruiseControlMetrics);
    LOGGER.info("Verifying that we have more than 0 groups");
    assertThat(regex.groupCount(), greaterThan(0));
    while (regex.find()) {
        String metricKey = regex.group(1);
        Object metricValue = regex.group(2);
        LOGGER.debug("{} -> {}", metricKey, metricValue);
        assertThat(metricKey, not(nullValue()));
        assertThat(metricValue, not(nullValue()));
    }
}
Also used : Matcher(java.util.regex.Matcher) Matchers.emptyString(org.hamcrest.Matchers.emptyString) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 69 with ParallelTest

use of io.strimzi.systemtest.annotations.ParallelTest in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testKafkaConnectIoNetwork.

@ParallelTest
@Tag(CONNECT)
@Tag(CONNECT_COMPONENTS)
void testKafkaConnectIoNetwork() {
    kafkaConnectMetricsData = collector.toBuilder().withComponentType(ComponentType.KafkaConnect).build().collectMetricsFromPods();
    Pattern connectIoNetwork = Pattern.compile("kafka_connect_network_io_total\\{clientid=\".*\",} ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(connectIoNetwork, kafkaConnectMetricsData);
    assertThat("KafkaConnect IO network count doesn't match expected value", values.stream().mapToDouble(i -> i).sum() > 0);
}
Also used : ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) CoreMatchers(org.hamcrest.CoreMatchers) Arrays(java.util.Arrays) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) KafkaExporterResources(io.strimzi.api.kafka.model.KafkaExporterResources) CruiseControlUtils(io.strimzi.systemtest.utils.specific.CruiseControlUtils) Matchers.emptyString(org.hamcrest.Matchers.emptyString) KafkaResource(io.strimzi.systemtest.resources.crd.KafkaResource) GLOBAL_TIMEOUT(io.strimzi.systemtest.Constants.GLOBAL_TIMEOUT) KubeClusterResource.cmdKubeClient(io.strimzi.test.k8s.KubeClusterResource.cmdKubeClient) Matchers.not(org.hamcrest.Matchers.not) KafkaTopicUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaTopicUtils) INFRA_NAMESPACE(io.strimzi.systemtest.Constants.INFRA_NAMESPACE) KafkaConnector(io.strimzi.api.kafka.model.KafkaConnector) JsonReadFeature(com.fasterxml.jackson.core.json.JsonReadFeature) Matcher(java.util.regex.Matcher) KafkaBridge(io.strimzi.api.kafka.model.KafkaBridge) ConfigMapKeySelector(io.fabric8.kubernetes.api.model.ConfigMapKeySelector) KafkaResources(io.strimzi.api.kafka.model.KafkaResources) BeforeAll(org.junit.jupiter.api.BeforeAll) Map(java.util.Map) Matchers.nullValue(org.hamcrest.Matchers.nullValue) Tag(org.junit.jupiter.api.Tag) KafkaConnect(io.strimzi.api.kafka.model.KafkaConnect) ACCEPTANCE(io.strimzi.systemtest.Constants.ACCEPTANCE) IsolatedSuite(io.strimzi.systemtest.annotations.IsolatedSuite) KafkaRebalance(io.strimzi.api.kafka.model.KafkaRebalance) KafkaMirrorMaker2Templates(io.strimzi.systemtest.templates.crd.KafkaMirrorMaker2Templates) Matchers.notNullValue(org.hamcrest.Matchers.notNullValue) KafkaMirrorMaker(io.strimzi.api.kafka.model.KafkaMirrorMaker) INTERNAL_CLIENTS_USED(io.strimzi.systemtest.Constants.INTERNAL_CLIENTS_USED) KafkaMirrorMaker2(io.strimzi.api.kafka.model.KafkaMirrorMaker2) List(java.util.List) Exec(io.strimzi.test.executor.Exec) Logger(org.apache.logging.log4j.Logger) KafkaTopicTemplates(io.strimzi.systemtest.templates.crd.KafkaTopicTemplates) BRIDGE(io.strimzi.systemtest.Constants.BRIDGE) BridgeClients(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients) BridgeClientsBuilder(io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.is(org.hamcrest.Matchers.is) KafkaTopicResource(io.strimzi.systemtest.resources.crd.KafkaTopicResource) Pattern(java.util.regex.Pattern) CRUISE_CONTROL(io.strimzi.systemtest.Constants.CRUISE_CONTROL) AbstractST(io.strimzi.systemtest.AbstractST) KafkaClientsTemplates(io.strimzi.systemtest.templates.crd.KafkaClientsTemplates) KafkaConnectTemplates(io.strimzi.systemtest.templates.crd.KafkaConnectTemplates) NetworkPolicyResource(io.strimzi.systemtest.resources.kubernetes.NetworkPolicyResource) CONNECT(io.strimzi.systemtest.Constants.CONNECT) OptionalDouble(java.util.OptionalDouble) ExtensionContext(org.junit.jupiter.api.extension.ExtensionContext) ArrayList(java.util.ArrayList) JmxPrometheusExporterMetrics(io.strimzi.api.kafka.model.JmxPrometheusExporterMetrics) PodUtils(io.strimzi.systemtest.utils.kubeUtils.objects.PodUtils) JmxPrometheusExporterMetricsBuilder(io.strimzi.api.kafka.model.JmxPrometheusExporterMetricsBuilder) TestUtils(io.strimzi.test.TestUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Assertions.assertEquals(org.junit.jupiter.api.Assertions.assertEquals) MIRROR_MAKER2(io.strimzi.systemtest.Constants.MIRROR_MAKER2) KafkaBridgeResources(io.strimzi.api.kafka.model.KafkaBridgeResources) KafkaTemplates(io.strimzi.systemtest.templates.crd.KafkaTemplates) InternalKafkaClient(io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient) ComponentType(io.strimzi.systemtest.resources.ComponentType) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Constants(io.strimzi.systemtest.Constants) IOException(java.io.IOException) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KubeClusterResource.kubeClient(io.strimzi.test.k8s.KubeClusterResource.kubeClient) ConfigMapBuilder(io.fabric8.kubernetes.api.model.ConfigMapBuilder) IsolatedTest(io.strimzi.systemtest.annotations.IsolatedTest) ExecutionException(java.util.concurrent.ExecutionException) DeploymentUtils(io.strimzi.systemtest.utils.kubeUtils.controllers.DeploymentUtils) ResourceManager(io.strimzi.systemtest.resources.ResourceManager) KafkaUserTemplates(io.strimzi.systemtest.templates.crd.KafkaUserTemplates) KafkaUserUtils(io.strimzi.systemtest.utils.kafkaUtils.KafkaUserUtils) METRICS(io.strimzi.systemtest.Constants.METRICS) CONNECT_COMPONENTS(io.strimzi.systemtest.Constants.CONNECT_COMPONENTS) ConfigMapKeySelectorBuilder(io.fabric8.kubernetes.api.model.ConfigMapKeySelectorBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) KafkaBridgeTemplates(io.strimzi.systemtest.templates.crd.KafkaBridgeTemplates) LogManager(org.apache.logging.log4j.LogManager) Collections(java.util.Collections) REGRESSION(io.strimzi.systemtest.Constants.REGRESSION) Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest) Tag(org.junit.jupiter.api.Tag)

Example 70 with ParallelTest

use of io.strimzi.systemtest.annotations.ParallelTest in project strimzi-kafka-operator by strimzi.

the class MetricsIsolatedST method testZookeeperAliveConnections.

@ParallelTest
void testZookeeperAliveConnections() {
    Pattern numAliveConnections = Pattern.compile("zookeeper_numaliveconnections ([\\d.][^\\n]+)", Pattern.CASE_INSENSITIVE);
    ArrayList<Double> values = MetricsCollector.collectSpecificMetric(numAliveConnections, zookeeperMetricsData);
    assertThat("Zookeeper alive connections count doesn't match expected value", values.stream().mapToDouble(i -> i).count(), is(0L));
}
Also used : Pattern(java.util.regex.Pattern) OptionalDouble(java.util.OptionalDouble) ParallelTest(io.strimzi.systemtest.annotations.ParallelTest)

Aggregations

ParallelTest (io.strimzi.systemtest.annotations.ParallelTest)192 Tag (org.junit.jupiter.api.Tag)62 InternalKafkaClient (io.strimzi.systemtest.kafkaclients.clients.InternalKafkaClient)38 Matchers.containsString (org.hamcrest.Matchers.containsString)38 KafkaOauthClients (io.strimzi.systemtest.kafkaclients.internalClients.KafkaOauthClients)32 KafkaOauthClientsBuilder (io.strimzi.systemtest.kafkaclients.internalClients.KafkaOauthClientsBuilder)32 OptionalDouble (java.util.OptionalDouble)28 Pattern (java.util.regex.Pattern)28 StringContains.containsString (org.hamcrest.core.StringContains.containsString)28 BridgeClients (io.strimzi.systemtest.kafkaclients.internalClients.BridgeClients)26 BridgeClientsBuilder (io.strimzi.systemtest.kafkaclients.internalClients.BridgeClientsBuilder)26 Description (io.vertx.core.cli.annotations.Description)24 KafkaUser (io.strimzi.api.kafka.model.KafkaUser)16 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)16 KafkaResources (io.strimzi.api.kafka.model.KafkaResources)14 AbstractST (io.strimzi.systemtest.AbstractST)14 Constants (io.strimzi.systemtest.Constants)14 INFRA_NAMESPACE (io.strimzi.systemtest.Constants.INFRA_NAMESPACE)14 REGRESSION (io.strimzi.systemtest.Constants.REGRESSION)14 IsolatedSuite (io.strimzi.systemtest.annotations.IsolatedSuite)14