Search in sources :

Example 6 with FlinkPod

use of org.apache.flink.kubernetes.kubeclient.FlinkPod in project flink by apache.

the class KubernetesJobManagerFactory method createJobManagerDeployment.

private static Deployment createJobManagerDeployment(FlinkPod flinkPod, KubernetesJobManagerParameters kubernetesJobManagerParameters) {
    final Container resolvedMainContainer = flinkPod.getMainContainer();
    final Pod resolvedPod = new PodBuilder(flinkPod.getPodWithoutMainContainer()).editOrNewSpec().addToContainers(resolvedMainContainer).endSpec().build();
    return new DeploymentBuilder().withApiVersion(Constants.APPS_API_VERSION).editOrNewMetadata().withName(KubernetesUtils.getDeploymentName(kubernetesJobManagerParameters.getClusterId())).withAnnotations(kubernetesJobManagerParameters.getAnnotations()).withLabels(kubernetesJobManagerParameters.getLabels()).withOwnerReferences(kubernetesJobManagerParameters.getOwnerReference().stream().map(e -> KubernetesOwnerReference.fromMap(e).getInternalResource()).collect(Collectors.toList())).endMetadata().editOrNewSpec().withReplicas(kubernetesJobManagerParameters.getReplicas()).editOrNewTemplate().withMetadata(resolvedPod.getMetadata()).withSpec(resolvedPod.getSpec()).endTemplate().editOrNewSelector().addToMatchLabels(kubernetesJobManagerParameters.getSelectors()).endSelector().endSpec().build();
}
Also used : Container(io.fabric8.kubernetes.api.model.Container) Pod(io.fabric8.kubernetes.api.model.Pod) FlinkPod(org.apache.flink.kubernetes.kubeclient.FlinkPod) PodBuilder(io.fabric8.kubernetes.api.model.PodBuilder) DeploymentBuilder(io.fabric8.kubernetes.api.model.apps.DeploymentBuilder)

Example 7 with FlinkPod

use of org.apache.flink.kubernetes.kubeclient.FlinkPod in project flink by apache.

the class InitTaskManagerDecoratorAccountTest method onSetup.

@Override
protected void onSetup() throws Exception {
    super.onSetup();
    final InitTaskManagerDecorator initTaskManagerDecorator = new InitTaskManagerDecorator(kubernetesTaskManagerParameters);
    final FlinkPod resultFlinkPod = initTaskManagerDecorator.decorateFlinkPod(this.baseFlinkPod);
    this.resultPod = resultFlinkPod.getPodWithoutMainContainer();
}
Also used : FlinkPod(org.apache.flink.kubernetes.kubeclient.FlinkPod)

Example 8 with FlinkPod

use of org.apache.flink.kubernetes.kubeclient.FlinkPod in project flink by apache.

the class PodTemplateMountDecoratorTest method testDecoratedFlinkPodWithTaskManagerPodTemplate.

@Test
public void testDecoratedFlinkPodWithTaskManagerPodTemplate() throws Exception {
    KubernetesTestUtils.createTemporyFile(POD_TEMPLATE_DATA, flinkConfDir, POD_TEMPLATE_FILE_NAME);
    final FlinkPod resultFlinkPod = podTemplateMountDecorator.decorateFlinkPod(baseFlinkPod);
    final List<KeyToPath> expectedKeyToPaths = Collections.singletonList(new KeyToPathBuilder().withKey(TASK_MANAGER_POD_TEMPLATE_FILE_NAME).withPath(TASK_MANAGER_POD_TEMPLATE_FILE_NAME).build());
    final List<Volume> expectedVolumes = getExpectedVolumes(expectedKeyToPaths);
    assertThat(resultFlinkPod.getPodWithoutMainContainer().getSpec().getVolumes(), containsInAnyOrder(expectedVolumes.toArray()));
    final List<VolumeMount> expectedVolumeMounts = Collections.singletonList(new VolumeMountBuilder().withName(Constants.POD_TEMPLATE_VOLUME).withMountPath(Constants.POD_TEMPLATE_DIR_IN_POD).build());
    assertThat(resultFlinkPod.getMainContainer().getVolumeMounts(), containsInAnyOrder(expectedVolumeMounts.toArray()));
}
Also used : KeyToPath(io.fabric8.kubernetes.api.model.KeyToPath) FlinkPod(org.apache.flink.kubernetes.kubeclient.FlinkPod) Volume(io.fabric8.kubernetes.api.model.Volume) KeyToPathBuilder(io.fabric8.kubernetes.api.model.KeyToPathBuilder) VolumeMount(io.fabric8.kubernetes.api.model.VolumeMount) VolumeMountBuilder(io.fabric8.kubernetes.api.model.VolumeMountBuilder) Test(org.junit.Test)

Example 9 with FlinkPod

use of org.apache.flink.kubernetes.kubeclient.FlinkPod in project flink by apache.

the class KubernetesJobManagerFactoryTest method testKerberosKeytabSecret.

@Test
public void testKerberosKeytabSecret() throws IOException {
    kubernetesJobManagerSpecification = KubernetesJobManagerFactory.buildKubernetesJobManagerSpecification(flinkPod, kubernetesJobManagerParameters);
    final Secret resultSecret = (Secret) this.kubernetesJobManagerSpecification.getAccompanyingResources().stream().filter(x -> x instanceof Secret && x.getMetadata().getName().equals(KerberosMountDecorator.getKerberosKeytabSecretName(CLUSTER_ID))).collect(Collectors.toList()).get(0);
    final Map<String, String> resultDatas = resultSecret.getData();
    assertEquals(1, resultDatas.size());
    assertEquals(Base64.getEncoder().encodeToString("some keytab".getBytes()), resultDatas.get(KEYTAB_FILE));
}
Also used : Secret(io.fabric8.kubernetes.api.model.Secret) Quantity(io.fabric8.kubernetes.api.model.Quantity) SecurityOptions(org.apache.flink.configuration.SecurityOptions) KubernetesTestUtils(org.apache.flink.kubernetes.KubernetesTestUtils) KubernetesJobManagerTestBase(org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerTestBase) CONFIG_FILE_LOG4J_NAME(org.apache.flink.kubernetes.utils.Constants.CONFIG_FILE_LOG4J_NAME) Map(java.util.Map) ExternalServiceDecorator(org.apache.flink.kubernetes.kubeclient.decorators.ExternalServiceDecorator) DeploymentSpec(io.fabric8.kubernetes.api.model.apps.DeploymentSpec) KubernetesUtils(org.apache.flink.kubernetes.utils.KubernetesUtils) KubernetesConfigOptions(org.apache.flink.kubernetes.configuration.KubernetesConfigOptions) FlinkPod(org.apache.flink.kubernetes.kubeclient.FlinkPod) HasMetadata(io.fabric8.kubernetes.api.model.HasMetadata) Collectors(java.util.stream.Collectors) Base64(java.util.Base64) List(java.util.List) Assert.assertFalse(org.junit.Assert.assertFalse) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Secret(io.fabric8.kubernetes.api.model.Secret) Matchers.is(org.hamcrest.Matchers.is) Constants(org.apache.flink.kubernetes.utils.Constants) Container(io.fabric8.kubernetes.api.model.Container) KubernetesConfigOptionsInternal(org.apache.flink.kubernetes.configuration.KubernetesConfigOptionsInternal) HashMap(java.util.HashMap) OwnerReference(io.fabric8.kubernetes.api.model.OwnerReference) KubernetesHaServicesFactory(org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory) PodSpec(io.fabric8.kubernetes.api.model.PodSpec) DeploymentOptions(org.apache.flink.configuration.DeploymentOptions) FlinkConfMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.FlinkConfMountDecorator) Service(io.fabric8.kubernetes.api.model.Service) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) KubernetesDeploymentTarget(org.apache.flink.kubernetes.configuration.KubernetesDeploymentTarget) FLINK_CONF_FILENAME(org.apache.flink.configuration.GlobalConfiguration.FLINK_CONF_FILENAME) InternalServiceDecorator(org.apache.flink.kubernetes.kubeclient.decorators.InternalServiceDecorator) Assert.assertNotNull(org.junit.Assert.assertNotNull) KubernetesJobManagerSpecification(org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerSpecification) Matchers(org.hamcrest.Matchers) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) KubernetesSessionClusterEntrypoint(org.apache.flink.kubernetes.entrypoint.KubernetesSessionClusterEntrypoint) HadoopConfMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.HadoopConfMountDecorator) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KerberosMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.KerberosMountDecorator) Assert.assertNull(org.junit.Assert.assertNull) HeadlessClusterIPService(org.apache.flink.kubernetes.kubeclient.services.HeadlessClusterIPService) CONFIG_FILE_LOGBACK_NAME(org.apache.flink.kubernetes.utils.Constants.CONFIG_FILE_LOGBACK_NAME) Deployment(io.fabric8.kubernetes.api.model.apps.Deployment) Collections(java.util.Collections) HighAvailabilityOptions(org.apache.flink.configuration.HighAvailabilityOptions) Assert.assertEquals(org.junit.Assert.assertEquals) Test(org.junit.Test)

Example 10 with FlinkPod

use of org.apache.flink.kubernetes.kubeclient.FlinkPod in project flink by apache.

the class KubernetesJobManagerFactoryTest method testKerberosConfConfigMap.

@Test
public void testKerberosConfConfigMap() throws IOException {
    kubernetesJobManagerSpecification = KubernetesJobManagerFactory.buildKubernetesJobManagerSpecification(flinkPod, kubernetesJobManagerParameters);
    final ConfigMap resultConfigMap = (ConfigMap) this.kubernetesJobManagerSpecification.getAccompanyingResources().stream().filter(x -> x instanceof ConfigMap && x.getMetadata().getName().equals(KerberosMountDecorator.getKerberosKrb5confConfigMapName(CLUSTER_ID))).collect(Collectors.toList()).get(0);
    assertEquals(Constants.API_VERSION, resultConfigMap.getApiVersion());
    assertEquals(KerberosMountDecorator.getKerberosKrb5confConfigMapName(CLUSTER_ID), resultConfigMap.getMetadata().getName());
    final Map<String, String> resultDatas = resultConfigMap.getData();
    assertEquals(1, resultDatas.size());
    assertEquals("some conf", resultDatas.get(KRB5_CONF_FILE));
}
Also used : Quantity(io.fabric8.kubernetes.api.model.Quantity) SecurityOptions(org.apache.flink.configuration.SecurityOptions) KubernetesTestUtils(org.apache.flink.kubernetes.KubernetesTestUtils) KubernetesJobManagerTestBase(org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerTestBase) CONFIG_FILE_LOG4J_NAME(org.apache.flink.kubernetes.utils.Constants.CONFIG_FILE_LOG4J_NAME) Map(java.util.Map) ExternalServiceDecorator(org.apache.flink.kubernetes.kubeclient.decorators.ExternalServiceDecorator) DeploymentSpec(io.fabric8.kubernetes.api.model.apps.DeploymentSpec) KubernetesUtils(org.apache.flink.kubernetes.utils.KubernetesUtils) KubernetesConfigOptions(org.apache.flink.kubernetes.configuration.KubernetesConfigOptions) FlinkPod(org.apache.flink.kubernetes.kubeclient.FlinkPod) HasMetadata(io.fabric8.kubernetes.api.model.HasMetadata) Collectors(java.util.stream.Collectors) Base64(java.util.Base64) List(java.util.List) Assert.assertFalse(org.junit.Assert.assertFalse) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Secret(io.fabric8.kubernetes.api.model.Secret) Matchers.is(org.hamcrest.Matchers.is) Constants(org.apache.flink.kubernetes.utils.Constants) Container(io.fabric8.kubernetes.api.model.Container) KubernetesConfigOptionsInternal(org.apache.flink.kubernetes.configuration.KubernetesConfigOptionsInternal) HashMap(java.util.HashMap) OwnerReference(io.fabric8.kubernetes.api.model.OwnerReference) KubernetesHaServicesFactory(org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory) PodSpec(io.fabric8.kubernetes.api.model.PodSpec) DeploymentOptions(org.apache.flink.configuration.DeploymentOptions) FlinkConfMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.FlinkConfMountDecorator) Service(io.fabric8.kubernetes.api.model.Service) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) KubernetesDeploymentTarget(org.apache.flink.kubernetes.configuration.KubernetesDeploymentTarget) FLINK_CONF_FILENAME(org.apache.flink.configuration.GlobalConfiguration.FLINK_CONF_FILENAME) InternalServiceDecorator(org.apache.flink.kubernetes.kubeclient.decorators.InternalServiceDecorator) Assert.assertNotNull(org.junit.Assert.assertNotNull) KubernetesJobManagerSpecification(org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerSpecification) Matchers(org.hamcrest.Matchers) Assert.assertTrue(org.junit.Assert.assertTrue) Test(org.junit.Test) IOException(java.io.IOException) KubernetesSessionClusterEntrypoint(org.apache.flink.kubernetes.entrypoint.KubernetesSessionClusterEntrypoint) HadoopConfMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.HadoopConfMountDecorator) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) KerberosMountDecorator(org.apache.flink.kubernetes.kubeclient.decorators.KerberosMountDecorator) Assert.assertNull(org.junit.Assert.assertNull) HeadlessClusterIPService(org.apache.flink.kubernetes.kubeclient.services.HeadlessClusterIPService) CONFIG_FILE_LOGBACK_NAME(org.apache.flink.kubernetes.utils.Constants.CONFIG_FILE_LOGBACK_NAME) Deployment(io.fabric8.kubernetes.api.model.apps.Deployment) Collections(java.util.Collections) HighAvailabilityOptions(org.apache.flink.configuration.HighAvailabilityOptions) Assert.assertEquals(org.junit.Assert.assertEquals) ConfigMap(io.fabric8.kubernetes.api.model.ConfigMap) Test(org.junit.Test)

Aggregations

FlinkPod (org.apache.flink.kubernetes.kubeclient.FlinkPod)54 Test (org.junit.Test)35 Container (io.fabric8.kubernetes.api.model.Container)15 List (java.util.List)14 Constants (org.apache.flink.kubernetes.utils.Constants)13 HashMap (java.util.HashMap)12 Collectors (java.util.stream.Collectors)12 KubernetesConfigOptions (org.apache.flink.kubernetes.configuration.KubernetesConfigOptions)12 HasMetadata (io.fabric8.kubernetes.api.model.HasMetadata)11 ConfigMap (io.fabric8.kubernetes.api.model.ConfigMap)10 Volume (io.fabric8.kubernetes.api.model.Volume)10 IOException (java.io.IOException)10 Collections (java.util.Collections)10 Map (java.util.Map)10 KubernetesJobManagerSpecification (org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerSpecification)9 KubernetesJobManagerTestBase (org.apache.flink.kubernetes.kubeclient.KubernetesJobManagerTestBase)9 FlinkConfMountDecorator (org.apache.flink.kubernetes.kubeclient.decorators.FlinkConfMountDecorator)9 HadoopConfMountDecorator (org.apache.flink.kubernetes.kubeclient.decorators.HadoopConfMountDecorator)9 KerberosMountDecorator (org.apache.flink.kubernetes.kubeclient.decorators.KerberosMountDecorator)9 Assert.assertEquals (org.junit.Assert.assertEquals)9