Examples with ZookeeperClusterSpec io.strimzi.api.kafka.model.ZookeeperClusterSpec used on opensource projects

Search in sources :

Example 1 with ZookeeperClusterSpec

use of io.strimzi.api.kafka.model.ZookeeperClusterSpec in project strimzi by strimzi.

the class KafkaST method testCustomAndUpdatedValues.

@ParallelNamespaceTest
@SuppressWarnings({ "checkstyle:MethodLength", "checkstyle:JavaNCSS" })
void testCustomAndUpdatedValues(ExtensionContext extensionContext) {
    final String namespaceName = StUtils.getNamespaceBasedOnRbac(namespace, extensionContext);
    final String clusterName = mapWithClusterNames.get(extensionContext.getDisplayName());
    final LabelSelector kafkaSelector = KafkaResource.getLabelSelector(clusterName, KafkaResources.kafkaStatefulSetName(clusterName));
    final LabelSelector zkSelector = KafkaResource.getLabelSelector(clusterName, KafkaResources.zookeeperStatefulSetName(clusterName));
    LinkedHashMap<String, String> envVarGeneral = new LinkedHashMap<>();
    envVarGeneral.put("TEST_ENV_1", "test.env.one");
    envVarGeneral.put("TEST_ENV_2", "test.env.two");
    LinkedHashMap<String, String> envVarUpdated = new LinkedHashMap<>();
    envVarUpdated.put("TEST_ENV_2", "updated.test.env.two");
    envVarUpdated.put("TEST_ENV_3", "test.env.three");
    // Kafka Broker config
    Map<String, Object> kafkaConfig = new HashMap<>();
    kafkaConfig.put("offsets.topic.replication.factor", "1");
    kafkaConfig.put("transaction.state.log.replication.factor", "1");
    kafkaConfig.put("default.replication.factor", "1");
    Map<String, Object> updatedKafkaConfig = new HashMap<>();
    updatedKafkaConfig.put("offsets.topic.replication.factor", "2");
    updatedKafkaConfig.put("transaction.state.log.replication.factor", "2");
    updatedKafkaConfig.put("default.replication.factor", "2");
    // Zookeeper Config
    Map<String, Object> zookeeperConfig = new HashMap<>();
    zookeeperConfig.put("tickTime", "2000");
    zookeeperConfig.put("initLimit", "5");
    zookeeperConfig.put("syncLimit", "2");
    zookeeperConfig.put("autopurge.purgeInterval", "1");
    Map<String, Object> updatedZookeeperConfig = new HashMap<>();
    updatedZookeeperConfig.put("tickTime", "2500");
    updatedZookeeperConfig.put("initLimit", "3");
    updatedZookeeperConfig.put("syncLimit", "5");
    final int initialDelaySeconds = 30;
    final int timeoutSeconds = 10;
    final int updatedInitialDelaySeconds = 31;
    final int updatedTimeoutSeconds = 11;
    final int periodSeconds = 10;
    final int successThreshold = 1;
    final int failureThreshold = 3;
    final int updatedPeriodSeconds = 5;
    final int updatedFailureThreshold = 1;
    resourceManager.createResource(extensionContext, KafkaTemplates.kafkaPersistent(clusterName, 2).editSpec().editKafka().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().withConfig(kafkaConfig).withNewTemplate().withNewKafkaContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endKafkaContainer().endTemplate().endKafka().editZookeeper().withReplicas(2).withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).endLivenessProbe().withConfig(zookeeperConfig).withNewTemplate().withNewZookeeperContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endZookeeperContainer().endTemplate().endZookeeper().editEntityOperator().withNewTemplate().withNewTopicOperatorContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endTopicOperatorContainer().withNewUserOperatorContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endUserOperatorContainer().withNewTlsSidecarContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endTlsSidecarContainer().endTemplate().editUserOperator().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endUserOperator().editTopicOperator().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endTopicOperator().withNewTlsSidecar().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endTlsSidecar().endEntityOperator().endSpec().build());
    final Map<String, String> kafkaSnapshot = PodUtils.podSnapshot(namespaceName, kafkaSelector);
    final Map<String, String> zkSnapshot = PodUtils.podSnapshot(namespaceName, zkSelector);
    final Map<String, String> eoPod = DeploymentUtils.depSnapshot(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName));
    LOGGER.info("Verify values before update");
    checkReadinessLivenessProbe(namespaceName, kafkaStatefulSetName(clusterName), "kafka", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkKafkaConfiguration(namespaceName, kafkaStatefulSetName(clusterName), kafkaConfig, clusterName);
    checkSpecificVariablesInContainer(namespaceName, kafkaStatefulSetName(clusterName), "kafka", envVarGeneral);
    String kafkaConfiguration = kubeClient().getConfigMap(namespaceName, KafkaResources.kafkaMetricsAndLogConfigMapName(clusterName)).getData().get("server.config");
    assertThat(kafkaConfiguration, containsString("offsets.topic.replication.factor=1"));
    assertThat(kafkaConfiguration, containsString("transaction.state.log.replication.factor=1"));
    assertThat(kafkaConfiguration, containsString("default.replication.factor=1"));
    String kafkaConfigurationFromPod = cmdKubeClient(namespaceName).execInPod(KafkaResources.kafkaPodName(clusterName, 0), "cat", "/tmp/strimzi.properties").out();
    assertThat(kafkaConfigurationFromPod, containsString("offsets.topic.replication.factor=1"));
    assertThat(kafkaConfigurationFromPod, containsString("transaction.state.log.replication.factor=1"));
    assertThat(kafkaConfigurationFromPod, containsString("default.replication.factor=1"));
    LOGGER.info("Testing Zookeepers");
    checkReadinessLivenessProbe(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkComponentConfiguration(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", "ZOOKEEPER_CONFIGURATION", zookeeperConfig);
    checkSpecificVariablesInContainer(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", envVarGeneral);
    LOGGER.info("Checking configuration of TO and UO");
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", envVarGeneral);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", envVarGeneral);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", envVarGeneral);
    LOGGER.info("Updating configuration of Kafka cluster");
    KafkaResource.replaceKafkaResourceInSpecificNamespace(clusterName, k -> {
        KafkaClusterSpec kafkaClusterSpec = k.getSpec().getKafka();
        kafkaClusterSpec.getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        kafkaClusterSpec.getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        kafkaClusterSpec.getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        kafkaClusterSpec.getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        kafkaClusterSpec.getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        kafkaClusterSpec.getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        kafkaClusterSpec.getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        kafkaClusterSpec.getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        kafkaClusterSpec.setConfig(updatedKafkaConfig);
        kafkaClusterSpec.getTemplate().getKafkaContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        ZookeeperClusterSpec zookeeperClusterSpec = k.getSpec().getZookeeper();
        zookeeperClusterSpec.getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        zookeeperClusterSpec.getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        zookeeperClusterSpec.getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        zookeeperClusterSpec.getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        zookeeperClusterSpec.getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        zookeeperClusterSpec.getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        zookeeperClusterSpec.getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        zookeeperClusterSpec.getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        zookeeperClusterSpec.setConfig(updatedZookeeperConfig);
        zookeeperClusterSpec.getTemplate().getZookeeperContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        // Configuring TO and UO to use new values for InitialDelaySeconds and TimeoutSeconds
        EntityOperatorSpec entityOperatorSpec = k.getSpec().getEntityOperator();
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTemplate().getTopicOperatorContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        entityOperatorSpec.getTemplate().getUserOperatorContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        entityOperatorSpec.getTemplate().getTlsSidecarContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
    }, namespaceName);
    RollingUpdateUtils.waitTillComponentHasRolled(namespaceName, zkSelector, 2, zkSnapshot);
    RollingUpdateUtils.waitTillComponentHasRolled(namespaceName, kafkaSelector, 2, kafkaSnapshot);
    DeploymentUtils.waitTillDepHasRolled(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), 1, eoPod);
    KafkaUtils.waitForKafkaReady(namespaceName, clusterName);
    LOGGER.info("Verify values after update");
    checkReadinessLivenessProbe(namespaceName, kafkaStatefulSetName(clusterName), "kafka", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkKafkaConfiguration(namespaceName, kafkaStatefulSetName(clusterName), updatedKafkaConfig, clusterName);
    checkSpecificVariablesInContainer(namespaceName, kafkaStatefulSetName(clusterName), "kafka", envVarUpdated);
    kafkaConfiguration = kubeClient(namespaceName).getConfigMap(namespaceName, KafkaResources.kafkaMetricsAndLogConfigMapName(clusterName)).getData().get("server.config");
    assertThat(kafkaConfiguration, containsString("offsets.topic.replication.factor=2"));
    assertThat(kafkaConfiguration, containsString("transaction.state.log.replication.factor=2"));
    assertThat(kafkaConfiguration, containsString("default.replication.factor=2"));
    kafkaConfigurationFromPod = cmdKubeClient(namespaceName).execInPod(KafkaResources.kafkaPodName(clusterName, 0), "cat", "/tmp/strimzi.properties").out();
    assertThat(kafkaConfigurationFromPod, containsString("offsets.topic.replication.factor=2"));
    assertThat(kafkaConfigurationFromPod, containsString("transaction.state.log.replication.factor=2"));
    assertThat(kafkaConfigurationFromPod, containsString("default.replication.factor=2"));
    LOGGER.info("Testing Zookeepers");
    checkReadinessLivenessProbe(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkComponentConfiguration(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", "ZOOKEEPER_CONFIGURATION", updatedZookeeperConfig);
    checkSpecificVariablesInContainer(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", envVarUpdated);
    LOGGER.info("Getting entity operator to check configuration of TO and UO");
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", envVarUpdated);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", envVarUpdated);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", envVarUpdated);
}
Also used : KafkaClusterSpec(io.strimzi.api.kafka.model.KafkaClusterSpec) ZookeeperClusterSpec(io.strimzi.api.kafka.model.ZookeeperClusterSpec) EntityOperatorSpec(io.strimzi.api.kafka.model.EntityOperatorSpec) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) Matchers.containsString(org.hamcrest.Matchers.containsString) Matchers.emptyOrNullString(org.hamcrest.Matchers.emptyOrNullString) TestUtils.fromYamlString(io.strimzi.test.TestUtils.fromYamlString) LinkedHashMap(java.util.LinkedHashMap) ParallelNamespaceTest(io.strimzi.systemtest.annotations.ParallelNamespaceTest)

Example 2 with ZookeeperClusterSpec

use of io.strimzi.api.kafka.model.ZookeeperClusterSpec in project strimzi-kafka-operator by strimzi.

the class KafkaST method testCustomAndUpdatedValues.

@ParallelNamespaceTest
@SuppressWarnings({ "checkstyle:MethodLength", "checkstyle:JavaNCSS" })
void testCustomAndUpdatedValues(ExtensionContext extensionContext) {
    final String namespaceName = StUtils.getNamespaceBasedOnRbac(namespace, extensionContext);
    final String clusterName = mapWithClusterNames.get(extensionContext.getDisplayName());
    final LabelSelector kafkaSelector = KafkaResource.getLabelSelector(clusterName, KafkaResources.kafkaStatefulSetName(clusterName));
    final LabelSelector zkSelector = KafkaResource.getLabelSelector(clusterName, KafkaResources.zookeeperStatefulSetName(clusterName));
    LinkedHashMap<String, String> envVarGeneral = new LinkedHashMap<>();
    envVarGeneral.put("TEST_ENV_1", "test.env.one");
    envVarGeneral.put("TEST_ENV_2", "test.env.two");
    LinkedHashMap<String, String> envVarUpdated = new LinkedHashMap<>();
    envVarUpdated.put("TEST_ENV_2", "updated.test.env.two");
    envVarUpdated.put("TEST_ENV_3", "test.env.three");
    // Kafka Broker config
    Map<String, Object> kafkaConfig = new HashMap<>();
    kafkaConfig.put("offsets.topic.replication.factor", "1");
    kafkaConfig.put("transaction.state.log.replication.factor", "1");
    kafkaConfig.put("default.replication.factor", "1");
    Map<String, Object> updatedKafkaConfig = new HashMap<>();
    updatedKafkaConfig.put("offsets.topic.replication.factor", "2");
    updatedKafkaConfig.put("transaction.state.log.replication.factor", "2");
    updatedKafkaConfig.put("default.replication.factor", "2");
    // Zookeeper Config
    Map<String, Object> zookeeperConfig = new HashMap<>();
    zookeeperConfig.put("tickTime", "2000");
    zookeeperConfig.put("initLimit", "5");
    zookeeperConfig.put("syncLimit", "2");
    zookeeperConfig.put("autopurge.purgeInterval", "1");
    Map<String, Object> updatedZookeeperConfig = new HashMap<>();
    updatedZookeeperConfig.put("tickTime", "2500");
    updatedZookeeperConfig.put("initLimit", "3");
    updatedZookeeperConfig.put("syncLimit", "5");
    final int initialDelaySeconds = 30;
    final int timeoutSeconds = 10;
    final int updatedInitialDelaySeconds = 31;
    final int updatedTimeoutSeconds = 11;
    final int periodSeconds = 10;
    final int successThreshold = 1;
    final int failureThreshold = 3;
    final int updatedPeriodSeconds = 5;
    final int updatedFailureThreshold = 1;
    resourceManager.createResource(extensionContext, KafkaTemplates.kafkaPersistent(clusterName, 2).editSpec().editKafka().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().withConfig(kafkaConfig).withNewTemplate().withNewKafkaContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endKafkaContainer().endTemplate().endKafka().editZookeeper().withReplicas(2).withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).endLivenessProbe().withConfig(zookeeperConfig).withNewTemplate().withNewZookeeperContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endZookeeperContainer().endTemplate().endZookeeper().editEntityOperator().withNewTemplate().withNewTopicOperatorContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endTopicOperatorContainer().withNewUserOperatorContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endUserOperatorContainer().withNewTlsSidecarContainer().withEnv(StUtils.createContainerEnvVarsFromMap(envVarGeneral)).endTlsSidecarContainer().endTemplate().editUserOperator().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endUserOperator().editTopicOperator().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endTopicOperator().withNewTlsSidecar().withNewReadinessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endReadinessProbe().withNewLivenessProbe().withInitialDelaySeconds(initialDelaySeconds).withTimeoutSeconds(timeoutSeconds).withPeriodSeconds(periodSeconds).withSuccessThreshold(successThreshold).withFailureThreshold(failureThreshold).endLivenessProbe().endTlsSidecar().endEntityOperator().endSpec().build());
    final Map<String, String> kafkaSnapshot = PodUtils.podSnapshot(namespaceName, kafkaSelector);
    final Map<String, String> zkSnapshot = PodUtils.podSnapshot(namespaceName, zkSelector);
    final Map<String, String> eoPod = DeploymentUtils.depSnapshot(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName));
    LOGGER.info("Verify values before update");
    checkReadinessLivenessProbe(namespaceName, kafkaStatefulSetName(clusterName), "kafka", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkKafkaConfiguration(namespaceName, kafkaStatefulSetName(clusterName), kafkaConfig, clusterName);
    checkSpecificVariablesInContainer(namespaceName, kafkaStatefulSetName(clusterName), "kafka", envVarGeneral);
    String kafkaConfiguration = kubeClient().getConfigMap(namespaceName, KafkaResources.kafkaMetricsAndLogConfigMapName(clusterName)).getData().get("server.config");
    assertThat(kafkaConfiguration, containsString("offsets.topic.replication.factor=1"));
    assertThat(kafkaConfiguration, containsString("transaction.state.log.replication.factor=1"));
    assertThat(kafkaConfiguration, containsString("default.replication.factor=1"));
    String kafkaConfigurationFromPod = cmdKubeClient(namespaceName).execInPod(KafkaResources.kafkaPodName(clusterName, 0), "cat", "/tmp/strimzi.properties").out();
    assertThat(kafkaConfigurationFromPod, containsString("offsets.topic.replication.factor=1"));
    assertThat(kafkaConfigurationFromPod, containsString("transaction.state.log.replication.factor=1"));
    assertThat(kafkaConfigurationFromPod, containsString("default.replication.factor=1"));
    LOGGER.info("Testing Zookeepers");
    checkReadinessLivenessProbe(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkComponentConfiguration(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", "ZOOKEEPER_CONFIGURATION", zookeeperConfig);
    checkSpecificVariablesInContainer(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", envVarGeneral);
    LOGGER.info("Checking configuration of TO and UO");
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", envVarGeneral);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", envVarGeneral);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", initialDelaySeconds, timeoutSeconds, periodSeconds, successThreshold, failureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", envVarGeneral);
    LOGGER.info("Updating configuration of Kafka cluster");
    KafkaResource.replaceKafkaResourceInSpecificNamespace(clusterName, k -> {
        KafkaClusterSpec kafkaClusterSpec = k.getSpec().getKafka();
        kafkaClusterSpec.getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        kafkaClusterSpec.getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        kafkaClusterSpec.getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        kafkaClusterSpec.getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        kafkaClusterSpec.getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        kafkaClusterSpec.getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        kafkaClusterSpec.getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        kafkaClusterSpec.getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        kafkaClusterSpec.setConfig(updatedKafkaConfig);
        kafkaClusterSpec.getTemplate().getKafkaContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        ZookeeperClusterSpec zookeeperClusterSpec = k.getSpec().getZookeeper();
        zookeeperClusterSpec.getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        zookeeperClusterSpec.getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        zookeeperClusterSpec.getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        zookeeperClusterSpec.getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        zookeeperClusterSpec.getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        zookeeperClusterSpec.getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        zookeeperClusterSpec.getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        zookeeperClusterSpec.getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        zookeeperClusterSpec.setConfig(updatedZookeeperConfig);
        zookeeperClusterSpec.getTemplate().getZookeeperContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        // Configuring TO and UO to use new values for InitialDelaySeconds and TimeoutSeconds
        EntityOperatorSpec entityOperatorSpec = k.getSpec().getEntityOperator();
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTopicOperator().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTopicOperator().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getUserOperator().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getUserOperator().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setInitialDelaySeconds(updatedInitialDelaySeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setTimeoutSeconds(updatedTimeoutSeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setPeriodSeconds(updatedPeriodSeconds);
        entityOperatorSpec.getTlsSidecar().getLivenessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTlsSidecar().getReadinessProbe().setFailureThreshold(updatedFailureThreshold);
        entityOperatorSpec.getTemplate().getTopicOperatorContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        entityOperatorSpec.getTemplate().getUserOperatorContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
        entityOperatorSpec.getTemplate().getTlsSidecarContainer().setEnv(StUtils.createContainerEnvVarsFromMap(envVarUpdated));
    }, namespaceName);
    RollingUpdateUtils.waitTillComponentHasRolled(namespaceName, zkSelector, 2, zkSnapshot);
    RollingUpdateUtils.waitTillComponentHasRolled(namespaceName, kafkaSelector, 2, kafkaSnapshot);
    DeploymentUtils.waitTillDepHasRolled(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), 1, eoPod);
    KafkaUtils.waitForKafkaReady(namespaceName, clusterName);
    LOGGER.info("Verify values after update");
    checkReadinessLivenessProbe(namespaceName, kafkaStatefulSetName(clusterName), "kafka", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkKafkaConfiguration(namespaceName, kafkaStatefulSetName(clusterName), updatedKafkaConfig, clusterName);
    checkSpecificVariablesInContainer(namespaceName, kafkaStatefulSetName(clusterName), "kafka", envVarUpdated);
    kafkaConfiguration = kubeClient(namespaceName).getConfigMap(namespaceName, KafkaResources.kafkaMetricsAndLogConfigMapName(clusterName)).getData().get("server.config");
    assertThat(kafkaConfiguration, containsString("offsets.topic.replication.factor=2"));
    assertThat(kafkaConfiguration, containsString("transaction.state.log.replication.factor=2"));
    assertThat(kafkaConfiguration, containsString("default.replication.factor=2"));
    kafkaConfigurationFromPod = cmdKubeClient(namespaceName).execInPod(KafkaResources.kafkaPodName(clusterName, 0), "cat", "/tmp/strimzi.properties").out();
    assertThat(kafkaConfigurationFromPod, containsString("offsets.topic.replication.factor=2"));
    assertThat(kafkaConfigurationFromPod, containsString("transaction.state.log.replication.factor=2"));
    assertThat(kafkaConfigurationFromPod, containsString("default.replication.factor=2"));
    LOGGER.info("Testing Zookeepers");
    checkReadinessLivenessProbe(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkComponentConfiguration(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", "ZOOKEEPER_CONFIGURATION", updatedZookeeperConfig);
    checkSpecificVariablesInContainer(namespaceName, zookeeperStatefulSetName(clusterName), "zookeeper", envVarUpdated);
    LOGGER.info("Getting entity operator to check configuration of TO and UO");
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "topic-operator", envVarUpdated);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "user-operator", envVarUpdated);
    checkReadinessLivenessProbe(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", updatedInitialDelaySeconds, updatedTimeoutSeconds, updatedPeriodSeconds, successThreshold, updatedFailureThreshold);
    checkSpecificVariablesInContainer(namespaceName, KafkaResources.entityOperatorDeploymentName(clusterName), "tls-sidecar", envVarUpdated);
}
Also used : KafkaClusterSpec(io.strimzi.api.kafka.model.KafkaClusterSpec) ZookeeperClusterSpec(io.strimzi.api.kafka.model.ZookeeperClusterSpec) EntityOperatorSpec(io.strimzi.api.kafka.model.EntityOperatorSpec) HashMap(java.util.HashMap) LinkedHashMap(java.util.LinkedHashMap) LabelSelector(io.fabric8.kubernetes.api.model.LabelSelector) Matchers.containsString(org.hamcrest.Matchers.containsString) Matchers.emptyOrNullString(org.hamcrest.Matchers.emptyOrNullString) TestUtils.fromYamlString(io.strimzi.test.TestUtils.fromYamlString) LinkedHashMap(java.util.LinkedHashMap) ParallelNamespaceTest(io.strimzi.systemtest.annotations.ParallelNamespaceTest)

Example 3 with ZookeeperClusterSpec

use of io.strimzi.api.kafka.model.ZookeeperClusterSpec in project strimzi by strimzi.

the class ResourceUtils method createKafka.

public static Kafka createKafka(String namespace, String name, int replicas, String image, int healthDelay, int healthTimeout, MetricsConfig metricsConfig, Map<String, Object> kafkaConfiguration, Map<String, Object> zooConfiguration, Storage kafkaStorage, SingleVolumeStorage zkStorage, Logging kafkaLogging, Logging zkLogging, KafkaExporterSpec keSpec, CruiseControlSpec ccSpec) {
    Kafka result = new Kafka();
    ObjectMeta meta = new ObjectMetaBuilder().withNamespace(namespace).withName(name).withLabels(Labels.fromMap(TestUtils.map(Labels.KUBERNETES_DOMAIN + "part-of", "tests", "my-user-label", "cromulent")).toMap()).build();
    result.setMetadata(meta);
    KafkaSpec spec = new KafkaSpec();
    KafkaClusterSpec kafkaClusterSpec = new KafkaClusterSpec();
    kafkaClusterSpec.setReplicas(replicas);
    kafkaClusterSpec.setListeners(singletonList(new GenericKafkaListenerBuilder().withName("plain").withPort(9092).withTls(false).withType(KafkaListenerType.INTERNAL).build()));
    kafkaClusterSpec.setImage(image);
    if (kafkaLogging != null) {
        kafkaClusterSpec.setLogging(kafkaLogging);
    }
    Probe livenessProbe = new Probe();
    livenessProbe.setInitialDelaySeconds(healthDelay);
    livenessProbe.setTimeoutSeconds(healthTimeout);
    livenessProbe.setSuccessThreshold(4);
    livenessProbe.setFailureThreshold(10);
    livenessProbe.setPeriodSeconds(33);
    kafkaClusterSpec.setLivenessProbe(livenessProbe);
    kafkaClusterSpec.setReadinessProbe(livenessProbe);
    kafkaClusterSpec.setMetricsConfig(metricsConfig);
    if (kafkaConfiguration != null) {
        kafkaClusterSpec.setConfig(kafkaConfiguration);
    }
    kafkaClusterSpec.setStorage(kafkaStorage);
    spec.setKafka(kafkaClusterSpec);
    ZookeeperClusterSpec zk = new ZookeeperClusterSpec();
    zk.setReplicas(replicas);
    zk.setImage(image + "-zk");
    if (zkLogging != null) {
        zk.setLogging(zkLogging);
    }
    zk.setLivenessProbe(livenessProbe);
    zk.setReadinessProbe(livenessProbe);
    if (zooConfiguration != null) {
        zk.setConfig(zooConfiguration);
    }
    zk.setStorage(zkStorage);
    zk.setMetricsConfig(metricsConfig);
    spec.setKafkaExporter(keSpec);
    spec.setCruiseControl(ccSpec);
    spec.setZookeeper(zk);
    result.setSpec(spec);
    return result;
}
Also used : KafkaClusterSpec(io.strimzi.api.kafka.model.KafkaClusterSpec) ObjectMeta(io.fabric8.kubernetes.api.model.ObjectMeta) ZookeeperClusterSpec(io.strimzi.api.kafka.model.ZookeeperClusterSpec) GenericKafkaListenerBuilder(io.strimzi.api.kafka.model.listener.arraylistener.GenericKafkaListenerBuilder) Kafka(io.strimzi.api.kafka.model.Kafka) ObjectMetaBuilder(io.fabric8.kubernetes.api.model.ObjectMetaBuilder) Probe(io.strimzi.api.kafka.model.Probe) KafkaSpec(io.strimzi.api.kafka.model.KafkaSpec)

Example 4 with ZookeeperClusterSpec

use of io.strimzi.api.kafka.model.ZookeeperClusterSpec in project strimzi by strimzi.

the class ZookeeperCluster method fromCrd.

@SuppressWarnings({ "checkstyle:MethodLength", "checkstyle:CyclomaticComplexity" })
public static ZookeeperCluster fromCrd(Reconciliation reconciliation, Kafka kafkaAssembly, KafkaVersion.Lookup versions, Storage oldStorage, int oldReplicas) {
    ZookeeperCluster zk = new ZookeeperCluster(reconciliation, kafkaAssembly);
    zk.setOwnerReference(kafkaAssembly);
    ZookeeperClusterSpec zookeeperClusterSpec = kafkaAssembly.getSpec().getZookeeper();
    int replicas = zookeeperClusterSpec.getReplicas();
    if (replicas <= 0) {
        replicas = ZookeeperClusterSpec.DEFAULT_REPLICAS;
    }
    if (replicas == 1 && zookeeperClusterSpec.getStorage() != null && "ephemeral".equals(zookeeperClusterSpec.getStorage().getType())) {
        LOGGER.warnCr(reconciliation, "A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used.");
    }
    zk.setReplicas(replicas);
    // Get the ZK version information from either the CRD or from the default setting
    KafkaClusterSpec kafkaClusterSpec = kafkaAssembly.getSpec().getKafka();
    String version = versions.supportedVersion(kafkaClusterSpec != null ? kafkaClusterSpec.getVersion() : null).zookeeperVersion();
    zk.setVersion(version);
    String image = zookeeperClusterSpec.getImage();
    if (image == null) {
        image = versions.kafkaImage(kafkaClusterSpec != null ? kafkaClusterSpec.getImage() : null, kafkaClusterSpec != null ? kafkaClusterSpec.getVersion() : null);
    }
    zk.setImage(image);
    if (zookeeperClusterSpec.getReadinessProbe() != null) {
        zk.setReadinessProbe(zookeeperClusterSpec.getReadinessProbe());
    }
    if (zookeeperClusterSpec.getLivenessProbe() != null) {
        zk.setLivenessProbe(zookeeperClusterSpec.getLivenessProbe());
    }
    Logging logging = zookeeperClusterSpec.getLogging();
    zk.setLogging(logging == null ? new InlineLogging() : logging);
    zk.setGcLoggingEnabled(zookeeperClusterSpec.getJvmOptions() == null ? DEFAULT_JVM_GC_LOGGING_ENABLED : zookeeperClusterSpec.getJvmOptions().isGcLoggingEnabled());
    if (zookeeperClusterSpec.getJvmOptions() != null) {
        zk.setJavaSystemProperties(zookeeperClusterSpec.getJvmOptions().getJavaSystemProperties());
    }
    // Parse different types of metrics configurations
    ModelUtils.parseMetrics(zk, zookeeperClusterSpec);
    if (oldStorage != null) {
        Storage newStorage = zookeeperClusterSpec.getStorage();
        AbstractModel.validatePersistentStorage(newStorage);
        StorageDiff diff = new StorageDiff(reconciliation, oldStorage, newStorage, oldReplicas, zookeeperClusterSpec.getReplicas());
        if (!diff.isEmpty()) {
            LOGGER.warnCr(reconciliation, "Only the following changes to Zookeeper storage are allowed: " + "changing the deleteClaim flag, " + "changing overrides to nodes which do not exist yet " + "and increasing size of persistent claim volumes (depending on the volume type and used storage class).");
            LOGGER.warnCr(reconciliation, "The desired ZooKeeper storage configuration in the custom resource {}/{} contains changes which are not allowed. As " + "a result, all storage changes will be ignored. Use DEBUG level logging for more information " + "about the detected changes.", kafkaAssembly.getMetadata().getNamespace(), kafkaAssembly.getMetadata().getName());
            Condition warning = StatusUtils.buildWarningCondition("ZooKeeperStorage", "The desired ZooKeeper storage configuration contains changes which are not allowed. As a " + "result, all storage changes will be ignored. Use DEBUG level logging for more information " + "about the detected changes.");
            zk.addWarningCondition(warning);
            zk.setStorage(oldStorage);
        } else {
            zk.setStorage(newStorage);
        }
    } else {
        zk.setStorage(zookeeperClusterSpec.getStorage());
    }
    zk.setConfiguration(new ZookeeperConfiguration(reconciliation, zookeeperClusterSpec.getConfig().entrySet()));
    zk.setResources(zookeeperClusterSpec.getResources());
    zk.setJvmOptions(zookeeperClusterSpec.getJvmOptions());
    if (zookeeperClusterSpec.getJmxOptions() != null) {
        zk.setJmxEnabled(Boolean.TRUE);
        AuthenticationUtils.configureZookeeperJmxOptions(zookeeperClusterSpec.getJmxOptions().getAuthentication(), zk);
    }
    if (zookeeperClusterSpec.getTemplate() != null) {
        ZookeeperClusterTemplate template = zookeeperClusterSpec.getTemplate();
        if (template.getStatefulset() != null) {
            if (template.getStatefulset().getPodManagementPolicy() != null) {
                zk.templatePodManagementPolicy = template.getStatefulset().getPodManagementPolicy();
            }
            if (template.getStatefulset().getMetadata() != null) {
                zk.templateStatefulSetLabels = template.getStatefulset().getMetadata().getLabels();
                zk.templateStatefulSetAnnotations = template.getStatefulset().getMetadata().getAnnotations();
            }
        }
        if (template.getPodSet() != null && template.getPodSet().getMetadata() != null) {
            zk.templatePodSetLabels = template.getPodSet().getMetadata().getLabels();
            zk.templatePodSetAnnotations = template.getPodSet().getMetadata().getAnnotations();
        }
        ModelUtils.parsePodTemplate(zk, template.getPod());
        ModelUtils.parseInternalServiceTemplate(zk, template.getClientService());
        ModelUtils.parseInternalHeadlessServiceTemplate(zk, template.getNodesService());
        if (template.getPersistentVolumeClaim() != null && template.getPersistentVolumeClaim().getMetadata() != null) {
            zk.templatePersistentVolumeClaimLabels = Util.mergeLabelsOrAnnotations(template.getPersistentVolumeClaim().getMetadata().getLabels(), zk.templateStatefulSetLabels);
            zk.templatePersistentVolumeClaimAnnotations = template.getPersistentVolumeClaim().getMetadata().getAnnotations();
        }
        if (template.getZookeeperContainer() != null && template.getZookeeperContainer().getEnv() != null) {
            zk.templateZookeeperContainerEnvVars = template.getZookeeperContainer().getEnv();
        }
        if (template.getZookeeperContainer() != null && template.getZookeeperContainer().getSecurityContext() != null) {
            zk.templateZookeeperContainerSecurityContext = template.getZookeeperContainer().getSecurityContext();
        }
        if (template.getServiceAccount() != null && template.getServiceAccount().getMetadata() != null) {
            zk.templateServiceAccountLabels = template.getServiceAccount().getMetadata().getLabels();
            zk.templateServiceAccountAnnotations = template.getServiceAccount().getMetadata().getAnnotations();
        }
        if (template.getJmxSecret() != null && template.getJmxSecret().getMetadata() != null) {
            zk.templateJmxSecretLabels = template.getJmxSecret().getMetadata().getLabels();
            zk.templateJmxSecretAnnotations = template.getJmxSecret().getMetadata().getAnnotations();
        }
        ModelUtils.parsePodDisruptionBudgetTemplate(zk, template.getPodDisruptionBudget());
    }
    zk.templatePodLabels = Util.mergeLabelsOrAnnotations(zk.templatePodLabels, DEFAULT_POD_LABELS);
    return zk;
}
Also used : KafkaClusterSpec(io.strimzi.api.kafka.model.KafkaClusterSpec) MetricsAndLogging(io.strimzi.operator.common.MetricsAndLogging) InlineLogging(io.strimzi.api.kafka.model.InlineLogging) Logging(io.strimzi.api.kafka.model.Logging) Condition(io.strimzi.api.kafka.model.status.Condition) ZookeeperClusterSpec(io.strimzi.api.kafka.model.ZookeeperClusterSpec) Storage(io.strimzi.api.kafka.model.storage.Storage) ZookeeperClusterTemplate(io.strimzi.api.kafka.model.template.ZookeeperClusterTemplate) IntOrString(io.fabric8.kubernetes.api.model.IntOrString) InlineLogging(io.strimzi.api.kafka.model.InlineLogging)

Example 5 with ZookeeperClusterSpec

use of io.strimzi.api.kafka.model.ZookeeperClusterSpec in project strimzi-kafka-operator by strimzi.

the class ZookeeperCluster method fromCrd.

@SuppressWarnings({ "checkstyle:MethodLength", "checkstyle:CyclomaticComplexity" })
public static ZookeeperCluster fromCrd(Reconciliation reconciliation, Kafka kafkaAssembly, KafkaVersion.Lookup versions, Storage oldStorage, int oldReplicas) {
    ZookeeperCluster zk = new ZookeeperCluster(reconciliation, kafkaAssembly);
    zk.setOwnerReference(kafkaAssembly);
    ZookeeperClusterSpec zookeeperClusterSpec = kafkaAssembly.getSpec().getZookeeper();
    int replicas = zookeeperClusterSpec.getReplicas();
    if (replicas <= 0) {
        replicas = ZookeeperClusterSpec.DEFAULT_REPLICAS;
    }
    if (replicas == 1 && zookeeperClusterSpec.getStorage() != null && "ephemeral".equals(zookeeperClusterSpec.getStorage().getType())) {
        LOGGER.warnCr(reconciliation, "A ZooKeeper cluster with a single replica and ephemeral storage will be in a defective state after any restart or rolling update. It is recommended that a minimum of three replicas are used.");
    }
    zk.setReplicas(replicas);
    // Get the ZK version information from either the CRD or from the default setting
    KafkaClusterSpec kafkaClusterSpec = kafkaAssembly.getSpec().getKafka();
    String version = versions.supportedVersion(kafkaClusterSpec != null ? kafkaClusterSpec.getVersion() : null).zookeeperVersion();
    zk.setVersion(version);
    String image = zookeeperClusterSpec.getImage();
    if (image == null) {
        image = versions.kafkaImage(kafkaClusterSpec != null ? kafkaClusterSpec.getImage() : null, kafkaClusterSpec != null ? kafkaClusterSpec.getVersion() : null);
    }
    zk.setImage(image);
    if (zookeeperClusterSpec.getReadinessProbe() != null) {
        zk.setReadinessProbe(zookeeperClusterSpec.getReadinessProbe());
    }
    if (zookeeperClusterSpec.getLivenessProbe() != null) {
        zk.setLivenessProbe(zookeeperClusterSpec.getLivenessProbe());
    }
    Logging logging = zookeeperClusterSpec.getLogging();
    zk.setLogging(logging == null ? new InlineLogging() : logging);
    zk.setGcLoggingEnabled(zookeeperClusterSpec.getJvmOptions() == null ? DEFAULT_JVM_GC_LOGGING_ENABLED : zookeeperClusterSpec.getJvmOptions().isGcLoggingEnabled());
    if (zookeeperClusterSpec.getJvmOptions() != null) {
        zk.setJavaSystemProperties(zookeeperClusterSpec.getJvmOptions().getJavaSystemProperties());
    }
    // Parse different types of metrics configurations
    ModelUtils.parseMetrics(zk, zookeeperClusterSpec);
    if (oldStorage != null) {
        Storage newStorage = zookeeperClusterSpec.getStorage();
        AbstractModel.validatePersistentStorage(newStorage);
        StorageDiff diff = new StorageDiff(reconciliation, oldStorage, newStorage, oldReplicas, zookeeperClusterSpec.getReplicas());
        if (!diff.isEmpty()) {
            LOGGER.warnCr(reconciliation, "Only the following changes to Zookeeper storage are allowed: " + "changing the deleteClaim flag, " + "changing overrides to nodes which do not exist yet " + "and increasing size of persistent claim volumes (depending on the volume type and used storage class).");
            LOGGER.warnCr(reconciliation, "The desired ZooKeeper storage configuration in the custom resource {}/{} contains changes which are not allowed. As " + "a result, all storage changes will be ignored. Use DEBUG level logging for more information " + "about the detected changes.", kafkaAssembly.getMetadata().getNamespace(), kafkaAssembly.getMetadata().getName());
            Condition warning = StatusUtils.buildWarningCondition("ZooKeeperStorage", "The desired ZooKeeper storage configuration contains changes which are not allowed. As a " + "result, all storage changes will be ignored. Use DEBUG level logging for more information " + "about the detected changes.");
            zk.addWarningCondition(warning);
            zk.setStorage(oldStorage);
        } else {
            zk.setStorage(newStorage);
        }
    } else {
        zk.setStorage(zookeeperClusterSpec.getStorage());
    }
    zk.setConfiguration(new ZookeeperConfiguration(reconciliation, zookeeperClusterSpec.getConfig().entrySet()));
    zk.setResources(zookeeperClusterSpec.getResources());
    zk.setJvmOptions(zookeeperClusterSpec.getJvmOptions());
    if (zookeeperClusterSpec.getJmxOptions() != null) {
        zk.setJmxEnabled(Boolean.TRUE);
        AuthenticationUtils.configureZookeeperJmxOptions(zookeeperClusterSpec.getJmxOptions().getAuthentication(), zk);
    }
    if (zookeeperClusterSpec.getTemplate() != null) {
        ZookeeperClusterTemplate template = zookeeperClusterSpec.getTemplate();
        if (template.getStatefulset() != null) {
            if (template.getStatefulset().getPodManagementPolicy() != null) {
                zk.templatePodManagementPolicy = template.getStatefulset().getPodManagementPolicy();
            }
            if (template.getStatefulset().getMetadata() != null) {
                zk.templateStatefulSetLabels = template.getStatefulset().getMetadata().getLabels();
                zk.templateStatefulSetAnnotations = template.getStatefulset().getMetadata().getAnnotations();
            }
        }
        if (template.getPodSet() != null && template.getPodSet().getMetadata() != null) {
            zk.templatePodSetLabels = template.getPodSet().getMetadata().getLabels();
            zk.templatePodSetAnnotations = template.getPodSet().getMetadata().getAnnotations();
        }
        ModelUtils.parsePodTemplate(zk, template.getPod());
        ModelUtils.parseInternalServiceTemplate(zk, template.getClientService());
        ModelUtils.parseInternalHeadlessServiceTemplate(zk, template.getNodesService());
        if (template.getPersistentVolumeClaim() != null && template.getPersistentVolumeClaim().getMetadata() != null) {
            zk.templatePersistentVolumeClaimLabels = Util.mergeLabelsOrAnnotations(template.getPersistentVolumeClaim().getMetadata().getLabels(), zk.templateStatefulSetLabels);
            zk.templatePersistentVolumeClaimAnnotations = template.getPersistentVolumeClaim().getMetadata().getAnnotations();
        }
        if (template.getZookeeperContainer() != null && template.getZookeeperContainer().getEnv() != null) {
            zk.templateZookeeperContainerEnvVars = template.getZookeeperContainer().getEnv();
        }
        if (template.getZookeeperContainer() != null && template.getZookeeperContainer().getSecurityContext() != null) {
            zk.templateZookeeperContainerSecurityContext = template.getZookeeperContainer().getSecurityContext();
        }
        if (template.getServiceAccount() != null && template.getServiceAccount().getMetadata() != null) {
            zk.templateServiceAccountLabels = template.getServiceAccount().getMetadata().getLabels();
            zk.templateServiceAccountAnnotations = template.getServiceAccount().getMetadata().getAnnotations();
        }
        if (template.getJmxSecret() != null && template.getJmxSecret().getMetadata() != null) {
            zk.templateJmxSecretLabels = template.getJmxSecret().getMetadata().getLabels();
            zk.templateJmxSecretAnnotations = template.getJmxSecret().getMetadata().getAnnotations();
        }
        ModelUtils.parsePodDisruptionBudgetTemplate(zk, template.getPodDisruptionBudget());
    }
    zk.templatePodLabels = Util.mergeLabelsOrAnnotations(zk.templatePodLabels, DEFAULT_POD_LABELS);
    return zk;
}
Also used : KafkaClusterSpec(io.strimzi.api.kafka.model.KafkaClusterSpec) MetricsAndLogging(io.strimzi.operator.common.MetricsAndLogging) InlineLogging(io.strimzi.api.kafka.model.InlineLogging) Logging(io.strimzi.api.kafka.model.Logging) Condition(io.strimzi.api.kafka.model.status.Condition) ZookeeperClusterSpec(io.strimzi.api.kafka.model.ZookeeperClusterSpec) Storage(io.strimzi.api.kafka.model.storage.Storage) ZookeeperClusterTemplate(io.strimzi.api.kafka.model.template.ZookeeperClusterTemplate) IntOrString(io.fabric8.kubernetes.api.model.IntOrString) InlineLogging(io.strimzi.api.kafka.model.InlineLogging)

Aggregations

KafkaClusterSpec (io.strimzi.api.kafka.model.KafkaClusterSpec)6 ZookeeperClusterSpec (io.strimzi.api.kafka.model.ZookeeperClusterSpec)6 IntOrString (io.fabric8.kubernetes.api.model.IntOrString)2 LabelSelector (io.fabric8.kubernetes.api.model.LabelSelector)2 ObjectMeta (io.fabric8.kubernetes.api.model.ObjectMeta)2 ObjectMetaBuilder (io.fabric8.kubernetes.api.model.ObjectMetaBuilder)2 EntityOperatorSpec (io.strimzi.api.kafka.model.EntityOperatorSpec)2 InlineLogging (io.strimzi.api.kafka.model.InlineLogging)2 Kafka (io.strimzi.api.kafka.model.Kafka)2 KafkaSpec (io.strimzi.api.kafka.model.KafkaSpec)2 Logging (io.strimzi.api.kafka.model.Logging)2 Probe (io.strimzi.api.kafka.model.Probe)2 GenericKafkaListenerBuilder (io.strimzi.api.kafka.model.listener.arraylistener.GenericKafkaListenerBuilder)2 Condition (io.strimzi.api.kafka.model.status.Condition)2 Storage (io.strimzi.api.kafka.model.storage.Storage)2 ZookeeperClusterTemplate (io.strimzi.api.kafka.model.template.ZookeeperClusterTemplate)2 MetricsAndLogging (io.strimzi.operator.common.MetricsAndLogging)2 ParallelNamespaceTest (io.strimzi.systemtest.annotations.ParallelNamespaceTest)2 TestUtils.fromYamlString (io.strimzi.test.TestUtils.fromYamlString)2 HashMap (java.util.HashMap)2
About Me
UseOf

Java Tutorials :

Simple Java RabbitMQ Example with Spring
Simple Springboot JPA Eclipeslink Example
Simple SpringBoot JPA Hibernate Example
Jackson JSON @JsonIgnore and @JsonIgnoreProperties Examples
Java Infinite recursion (StackOverflowError) Jackson solutions
Simple Java Jackson Serialize Objects to JSON and convert them back To Object
ElasticSearch 5 - Upload files using Java API in binary field Example
Java JAXB marshall unmarshall Examples from String and Stream
Java 8 forEach List and Map examples
ElasticSearch 5 Java API SearchRequestBuilder examples
Java ElasticSearch 5 examples with node index and mapping - running local
Convert String to int or Integer Java 8
Java 9 in action - JShell - Ubuntu
Java - removing invalid characters from a file name
Hello world!? or Hello Tutorial