Search in sources :

Example 1 with ResourceRequirements

use of io.fabric8.kubernetes.api.model.ResourceRequirements in project flink by apache.

the class InitTaskManagerDecorator method decorateMainContainer.

private Container decorateMainContainer(Container container) {
    final ContainerBuilder mainContainerBuilder = new ContainerBuilder(container);
    // Overwrite fields
    final ResourceRequirements requirementsInPodTemplate = container.getResources() == null ? new ResourceRequirements() : container.getResources();
    final ResourceRequirements resourceRequirements = KubernetesUtils.getResourceRequirements(requirementsInPodTemplate, kubernetesTaskManagerParameters.getTaskManagerMemoryMB(), kubernetesTaskManagerParameters.getTaskManagerMemoryLimitFactor(), kubernetesTaskManagerParameters.getTaskManagerCPU(), kubernetesTaskManagerParameters.getTaskManagerCPULimitFactor(), kubernetesTaskManagerParameters.getTaskManagerExternalResources(), kubernetesTaskManagerParameters.getTaskManagerExternalResourceConfigKeys());
    final String image = KubernetesUtils.resolveUserDefinedValue(flinkConfig, KubernetesConfigOptions.CONTAINER_IMAGE, kubernetesTaskManagerParameters.getImage(), container.getImage(), "main container image");
    final String imagePullPolicy = KubernetesUtils.resolveUserDefinedValue(flinkConfig, KubernetesConfigOptions.CONTAINER_IMAGE_PULL_POLICY, kubernetesTaskManagerParameters.getImagePullPolicy().name(), container.getImagePullPolicy(), "main container image pull policy");
    mainContainerBuilder.withName(Constants.MAIN_CONTAINER_NAME).withImage(image).withImagePullPolicy(imagePullPolicy).withResources(resourceRequirements);
    // Merge fields
    mainContainerBuilder.addAllToPorts(getContainerPorts()).addAllToEnv(getCustomizedEnvs());
    getFlinkLogDirEnv().ifPresent(mainContainerBuilder::addToEnv);
    return mainContainerBuilder.build();
}
Also used : ContainerBuilder(io.fabric8.kubernetes.api.model.ContainerBuilder) ResourceRequirements(io.fabric8.kubernetes.api.model.ResourceRequirements)

Example 2 with ResourceRequirements

use of io.fabric8.kubernetes.api.model.ResourceRequirements in project flink by apache.

the class InitJobManagerDecorator method decorateMainContainer.

private Container decorateMainContainer(Container container) {
    final ContainerBuilder mainContainerBuilder = new ContainerBuilder(container);
    // Overwrite fields
    final String image = KubernetesUtils.resolveUserDefinedValue(flinkConfig, KubernetesConfigOptions.CONTAINER_IMAGE, kubernetesJobManagerParameters.getImage(), container.getImage(), "main container image");
    final String imagePullPolicy = KubernetesUtils.resolveUserDefinedValue(flinkConfig, KubernetesConfigOptions.CONTAINER_IMAGE_PULL_POLICY, kubernetesJobManagerParameters.getImagePullPolicy().name(), container.getImagePullPolicy(), "main container image pull policy");
    final ResourceRequirements requirementsInPodTemplate = container.getResources() == null ? new ResourceRequirements() : container.getResources();
    final ResourceRequirements requirements = KubernetesUtils.getResourceRequirements(requirementsInPodTemplate, kubernetesJobManagerParameters.getJobManagerMemoryMB(), kubernetesJobManagerParameters.getJobManagerMemoryLimitFactor(), kubernetesJobManagerParameters.getJobManagerCPU(), kubernetesJobManagerParameters.getJobManagerCPULimitFactor(), Collections.emptyMap(), Collections.emptyMap());
    mainContainerBuilder.withName(Constants.MAIN_CONTAINER_NAME).withImage(image).withImagePullPolicy(imagePullPolicy).withResources(requirements);
    // Merge fields
    mainContainerBuilder.addAllToPorts(getContainerPorts()).addAllToEnv(getCustomizedEnvs()).addNewEnv().withName(ENV_FLINK_POD_IP_ADDRESS).withValueFrom(new EnvVarSourceBuilder().withNewFieldRef(API_VERSION, POD_IP_FIELD_PATH).build()).endEnv();
    getFlinkLogDirEnv().ifPresent(mainContainerBuilder::addToEnv);
    return mainContainerBuilder.build();
}
Also used : ContainerBuilder(io.fabric8.kubernetes.api.model.ContainerBuilder) EnvVarSourceBuilder(io.fabric8.kubernetes.api.model.EnvVarSourceBuilder) ResourceRequirements(io.fabric8.kubernetes.api.model.ResourceRequirements)

Example 3 with ResourceRequirements

use of io.fabric8.kubernetes.api.model.ResourceRequirements in project camel by apache.

the class KubernetesPersistentVolumesClaimsProducerTest method createListAndDeletePersistentVolumeClaim.

@Test
public void createListAndDeletePersistentVolumeClaim() throws Exception {
    if (ObjectHelper.isEmpty(authToken)) {
        return;
    }
    Exchange ex = template.request("direct:create", new Processor() {

        @Override
        public void process(Exchange exchange) throws Exception {
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_NAMESPACE_NAME, "default");
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_PERSISTENT_VOLUME_CLAIM_NAME, "test");
            Map<String, String> labels = new HashMap<String, String>();
            labels.put("this", "rocks");
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_PERSISTENT_VOLUMES_CLAIMS_LABELS, labels);
            PersistentVolumeClaimSpec pvcSpec = new PersistentVolumeClaimSpec();
            ResourceRequirements rr = new ResourceRequirements();
            Map<String, Quantity> mp = new HashMap<String, Quantity>();
            mp.put("storage", new Quantity("100"));
            rr.setLimits(mp);
            Map<String, Quantity> req = new HashMap<String, Quantity>();
            req.put("storage", new Quantity("100"));
            rr.setRequests(req);
            pvcSpec.setResources(rr);
            pvcSpec.setVolumeName("vol001");
            List<String> access = new ArrayList<String>();
            access.add("ReadWriteOnce");
            pvcSpec.setAccessModes(access);
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_PERSISTENT_VOLUME_CLAIM_SPEC, pvcSpec);
        }
    });
    PersistentVolumeClaim pvc = ex.getOut().getBody(PersistentVolumeClaim.class);
    assertEquals(pvc.getMetadata().getName(), "test");
    ex = template.request("direct:listByLabels", new Processor() {

        @Override
        public void process(Exchange exchange) throws Exception {
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_NAMESPACE_NAME, "default");
            Map<String, String> labels = new HashMap<String, String>();
            labels.put("this", "rocks");
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_PERSISTENT_VOLUMES_CLAIMS_LABELS, labels);
        }
    });
    List<PersistentVolumeClaim> result = ex.getOut().getBody(List.class);
    boolean pvcExists = false;
    Iterator<PersistentVolumeClaim> it = result.iterator();
    while (it.hasNext()) {
        PersistentVolumeClaim pvcLocal = it.next();
        if ("test".equalsIgnoreCase(pvcLocal.getMetadata().getName())) {
            pvcExists = true;
        }
    }
    assertTrue(pvcExists);
    ex = template.request("direct:delete", new Processor() {

        @Override
        public void process(Exchange exchange) throws Exception {
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_NAMESPACE_NAME, "default");
            exchange.getIn().setHeader(KubernetesConstants.KUBERNETES_PERSISTENT_VOLUME_CLAIM_NAME, "test");
        }
    });
    boolean pvcDeleted = ex.getOut().getBody(Boolean.class);
    assertTrue(pvcDeleted);
}
Also used : Processor(org.apache.camel.Processor) HashMap(java.util.HashMap) Quantity(io.fabric8.kubernetes.api.model.Quantity) ResourceRequirements(io.fabric8.kubernetes.api.model.ResourceRequirements) Exchange(org.apache.camel.Exchange) PersistentVolumeClaimSpec(io.fabric8.kubernetes.api.model.PersistentVolumeClaimSpec) ArrayList(java.util.ArrayList) List(java.util.List) PersistentVolumeClaim(io.fabric8.kubernetes.api.model.PersistentVolumeClaim) HashMap(java.util.HashMap) Map(java.util.Map) Test(org.junit.Test)

Example 4 with ResourceRequirements

use of io.fabric8.kubernetes.api.model.ResourceRequirements in project halyard by spinnaker.

the class ResourceBuilder method buildResourceRequirements.

static ResourceRequirements buildResourceRequirements(String serviceName, DeploymentEnvironment deploymentEnvironment) {
    Map<String, Map> customSizing = deploymentEnvironment.getCustomSizing().get(serviceName);
    if (customSizing == null) {
        return null;
    }
    ResourceRequirementsBuilder resourceRequirementsBuilder = new ResourceRequirementsBuilder();
    if (customSizing.get("requests") != null) {
        resourceRequirementsBuilder.addToRequests("memory", new QuantityBuilder().withAmount(CustomSizing.stringOrNull(customSizing.get("requests").get("memory"))).build());
        resourceRequirementsBuilder.addToRequests("cpu", new QuantityBuilder().withAmount(CustomSizing.stringOrNull(customSizing.get("requests").get("cpu"))).build());
    }
    if (customSizing.get("limits") != null) {
        resourceRequirementsBuilder.addToLimits("memory", new QuantityBuilder().withAmount(CustomSizing.stringOrNull(customSizing.get("limits").get("memory"))).build());
        resourceRequirementsBuilder.addToLimits("cpu", new QuantityBuilder().withAmount(CustomSizing.stringOrNull(customSizing.get("limits").get("cpu"))).build());
    }
    return resourceRequirementsBuilder.build();
}
Also used : QuantityBuilder(io.fabric8.kubernetes.api.model.QuantityBuilder) ResourceRequirementsBuilder(io.fabric8.kubernetes.api.model.ResourceRequirementsBuilder) Map(java.util.Map)

Example 5 with ResourceRequirements

use of io.fabric8.kubernetes.api.model.ResourceRequirements in project flink by apache.

the class KubernetesUtils method getResourceRequirements.

/**
 * Get resource requirements from memory and cpu.
 *
 * @param resourceRequirements resource requirements in pod template
 * @param mem Memory in mb.
 * @param memoryLimitFactor limit factor for the memory, used to set the limit resources.
 * @param cpu cpu.
 * @param cpuLimitFactor limit factor for the cpu, used to set the limit resources.
 * @param externalResources external resources
 * @param externalResourceConfigKeys config keys of external resources
 * @return KubernetesResource requirements.
 */
public static ResourceRequirements getResourceRequirements(ResourceRequirements resourceRequirements, int mem, double memoryLimitFactor, double cpu, double cpuLimitFactor, Map<String, ExternalResource> externalResources, Map<String, String> externalResourceConfigKeys) {
    final Quantity cpuQuantity = new Quantity(String.valueOf(cpu));
    final Quantity cpuLimitQuantity = new Quantity(String.valueOf(cpu * cpuLimitFactor));
    final Quantity memQuantity = new Quantity(mem + Constants.RESOURCE_UNIT_MB);
    final Quantity memQuantityLimit = new Quantity(((int) (mem * memoryLimitFactor)) + Constants.RESOURCE_UNIT_MB);
    ResourceRequirementsBuilder resourceRequirementsBuilder = new ResourceRequirementsBuilder(resourceRequirements).addToRequests(Constants.RESOURCE_NAME_MEMORY, memQuantity).addToRequests(Constants.RESOURCE_NAME_CPU, cpuQuantity).addToLimits(Constants.RESOURCE_NAME_MEMORY, memQuantityLimit).addToLimits(Constants.RESOURCE_NAME_CPU, cpuLimitQuantity);
    // Add the external resources to resource requirement.
    for (Map.Entry<String, ExternalResource> externalResource : externalResources.entrySet()) {
        final String configKey = externalResourceConfigKeys.get(externalResource.getKey());
        if (!StringUtils.isNullOrWhitespaceOnly(configKey)) {
            final Quantity resourceQuantity = new Quantity(String.valueOf(externalResource.getValue().getValue().longValue()));
            resourceRequirementsBuilder.addToRequests(configKey, resourceQuantity).addToLimits(configKey, resourceQuantity);
            LOG.info("Request external resource {} with config key {}.", resourceQuantity.getAmount(), configKey);
        }
    }
    return resourceRequirementsBuilder.build();
}
Also used : ResourceRequirementsBuilder(io.fabric8.kubernetes.api.model.ResourceRequirementsBuilder) Quantity(io.fabric8.kubernetes.api.model.Quantity) KubernetesConfigMap(org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMap) Map(java.util.Map) HashMap(java.util.HashMap) ExternalResource(org.apache.flink.api.common.resources.ExternalResource)

Aggregations

ResourceRequirements (io.fabric8.kubernetes.api.model.ResourceRequirements)8 Quantity (io.fabric8.kubernetes.api.model.Quantity)7 Test (org.junit.Test)5 Map (java.util.Map)3 ContainerBuilder (io.fabric8.kubernetes.api.model.ContainerBuilder)2 ResourceRequirementsBuilder (io.fabric8.kubernetes.api.model.ResourceRequirementsBuilder)2 HashMap (java.util.HashMap)2 Then (io.cucumber.java.en.Then)1 EnvVarSourceBuilder (io.fabric8.kubernetes.api.model.EnvVarSourceBuilder)1 PersistentVolumeClaim (io.fabric8.kubernetes.api.model.PersistentVolumeClaim)1 PersistentVolumeClaimSpec (io.fabric8.kubernetes.api.model.PersistentVolumeClaimSpec)1 QuantityBuilder (io.fabric8.kubernetes.api.model.QuantityBuilder)1 ArrayList (java.util.ArrayList)1 List (java.util.List)1 Exchange (org.apache.camel.Exchange)1 Processor (org.apache.camel.Processor)1 ExternalResource (org.apache.flink.api.common.resources.ExternalResource)1 KubernetesConfigMap (org.apache.flink.kubernetes.kubeclient.resources.KubernetesConfigMap)1