Search in sources :

Example 6 with ClouderaManagerProduct

use of com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct in project cloudbreak by hortonworks.

the class PreWarmParcelParser method parseProductFromParcel.

public Optional<ClouderaManagerProduct> parseProductFromParcel(List<String> parcel, List<String> csdList) {
    Optional<String> url = parcel.stream().filter(parcelPart -> parcelPart.startsWith("http://") || parcelPart.startsWith("https://")).findFirst();
    Optional<String> nameAndVersion = parcel.stream().filter(parcelPart -> parcelPart.endsWith(".parcel")).findFirst();
    if (url.isEmpty() || nameAndVersion.isEmpty()) {
        LOGGER.warn("Parcel URL or (name and version) could not be found in the image metadata. " + "Parcel url: '{}', Parcel name and version: '{}'.", url.orElse("null"), nameAndVersion.orElse("null"));
        return Optional.empty();
    } else {
        ClouderaManagerProduct product = new ClouderaManagerProduct();
        String name = substringBefore(nameAndVersion.get(), "-");
        product.setName(name);
        product.setDisplayName(nameAndVersion.get());
        String version = substringAfter(substringBeforeLast(nameAndVersion.get(), "-"), "-");
        product.setVersion(version);
        LOGGER.debug("The parsed product name for parcel: '{}', version: '{}'. URL: '{}'", name, version, url.get());
        product.setParcel(url.get());
        product.setCsd(collectCsdParcels(csdList, name));
        return Optional.of(product);
    }
}
Also used : Inject(javax.inject.Inject) List(java.util.List) Component(org.springframework.stereotype.Component) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct) Logger(org.slf4j.Logger) StringUtils.substringBeforeLast(org.apache.commons.lang3.StringUtils.substringBeforeLast) LoggerFactory(org.slf4j.LoggerFactory) Optional(java.util.Optional) StringUtils.substringAfter(org.apache.commons.lang3.StringUtils.substringAfter) StringUtils.substringBefore(org.apache.commons.lang3.StringUtils.substringBefore) Collectors(java.util.stream.Collectors) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)

Example 7 with ClouderaManagerProduct

use of com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct in project cloudbreak by hortonworks.

the class StackService method setRuntime.

private void setRuntime(Stack stack, Set<Component> components) {
    ClouderaManagerProduct runtime = ComponentConfigProviderService.getComponent(components, ClouderaManagerProduct.class, CDH_PRODUCT_DETAILS);
    if (Objects.nonNull(runtime)) {
        String stackVersion = substringBefore(runtime.getVersion(), "-");
        LOGGER.debug("Setting runtime version {} for stack", stackVersion);
        stack.setStackVersion(stackVersion);
        stackRepository.save(stack);
    } else {
        // should not happen ever
        LOGGER.warn("Product component is not present amongst components, runtime could not be set!");
    }
}
Also used : ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)

Example 8 with ClouderaManagerProduct

use of com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct in project cloudbreak by hortonworks.

the class ClouderaManagerModificationService method removeUnusedParcelVersions.

private void removeUnusedParcelVersions(ParcelResourceApi parcelResourceApi, Set<ClouderaManagerProduct> products) throws ApiException {
    ParcelsResourceApi parcelsResourceApi = clouderaManagerApiFactory.getParcelsResourceApi(apiClient);
    for (ClouderaManagerProduct product : products) {
        LOGGER.info("Removing unused {} parcels.", product.getName());
        clouderaManagerParcelDecommissionService.removeUnusedParcelVersions(apiClient, parcelsResourceApi, parcelResourceApi, stack, product);
    }
}
Also used : ParcelsResourceApi(com.cloudera.api.swagger.ParcelsResourceApi) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)

Example 9 with ClouderaManagerProduct

use of com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct in project cloudbreak by hortonworks.

the class ClouderaManagerParcelManagementService method downloadParcels.

public void downloadParcels(Set<ClouderaManagerProduct> products, ParcelResourceApi parcelResourceApi, Stack stack, ApiClient apiClient) throws ApiException, CloudbreakException {
    for (ClouderaManagerProduct product : products) {
        LOGGER.info("Downloading {} parcel.", product.getName());
        ApiCommand apiCommand = parcelResourceApi.startDownloadCommand(stack.getName(), product.getName(), product.getVersion());
        ExtendedPollingResult pollingResult = clouderaManagerPollingServiceProvider.startPollingCdpRuntimeParcelDownload(stack, apiClient, apiCommand.getId(), new ParcelResource(stack.getName(), product.getName(), product.getVersion()));
        handlePollingResult(pollingResult.getPollingResult(), "Cluster was terminated while waiting for CDP Runtime Parcel to be downloaded", "Timeout during the updated CDP Runtime Parcel download.");
    }
}
Also used : ApiCommand(com.cloudera.api.swagger.model.ApiCommand) ParcelResource(com.sequenceiq.cloudbreak.cm.model.ParcelResource) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct) ExtendedPollingResult(com.sequenceiq.cloudbreak.polling.ExtendedPollingResult)

Example 10 with ClouderaManagerProduct

use of com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct in project cloudbreak by hortonworks.

the class CsdParcelDecoratorTest method testDecoratePillarWithCsdParcelsShouldAddTheParcelsToPillarWhenTheStackTypeIsWorkload.

@Test
public void testDecoratePillarWithCsdParcelsShouldAddTheParcelsToPillarWhenTheStackTypeIsWorkload() {
    Stack stack = createStack(StackType.WORKLOAD);
    Set<ClusterComponent> componentsByBlueprint = Collections.emptySet();
    Set<ClouderaManagerProduct> products = createProducts();
    Map<String, SaltPillarProperties> servicePillar = new HashMap<>();
    when(parcelService.getParcelComponentsByBlueprint(stack)).thenReturn(componentsByBlueprint);
    when(clouderaManagerProductsProvider.getProducts(componentsByBlueprint)).thenReturn(products);
    underTest.decoratePillarWithCsdParcels(stack, servicePillar);
    SaltPillarProperties pillarProperties = servicePillar.get("csd-downloader");
    assertEquals("/cloudera-manager/csd.sls", pillarProperties.getPath());
    Map<String, List<String>> csdUrls = (Map<String, List<String>>) pillarProperties.getProperties().get("cloudera-manager");
    assertTrue(csdUrls.get("csd-urls").containsAll(CSD_LIST));
    verify(parcelService).getParcelComponentsByBlueprint(stack);
    verify(clouderaManagerProductsProvider).getProducts(componentsByBlueprint);
}
Also used : ClusterComponent(com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent) HashMap(java.util.HashMap) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct) List(java.util.List) HashMap(java.util.HashMap) Map(java.util.Map) SaltPillarProperties(com.sequenceiq.cloudbreak.orchestrator.model.SaltPillarProperties) Stack(com.sequenceiq.cloudbreak.domain.stack.Stack) Test(org.junit.Test)

Aggregations

ClouderaManagerProduct (com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)91 Test (org.junit.Test)34 Test (org.junit.jupiter.api.Test)23 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)23 Blueprint (com.sequenceiq.cloudbreak.domain.Blueprint)20 ClouderaManagerRepo (com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerRepo)14 TemplatePreparationObject (com.sequenceiq.cloudbreak.template.TemplatePreparationObject)10 Image (com.sequenceiq.cloudbreak.cloud.model.catalog.Image)9 ClusterComponent (com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent)9 ParcelInfo (com.sequenceiq.cloudbreak.service.upgrade.sync.common.ParcelInfo)9 HashSet (java.util.HashSet)8 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)8 Stack (com.sequenceiq.cloudbreak.domain.stack.Stack)7 BlueprintView (com.sequenceiq.cloudbreak.template.views.BlueprintView)7 VirtualGroupRequest (com.sequenceiq.cloudbreak.auth.altus.VirtualGroupRequest)6 IdBroker (com.sequenceiq.cloudbreak.domain.stack.cluster.IdBroker)6 ExtendedPollingResult (com.sequenceiq.cloudbreak.polling.ExtendedPollingResult)6 GeneralClusterConfigs (com.sequenceiq.cloudbreak.template.model.GeneralClusterConfigs)6 BlueprintTextProcessor (com.sequenceiq.cloudbreak.template.processor.BlueprintTextProcessor)6 Gateway (com.sequenceiq.cloudbreak.domain.stack.cluster.gateway.Gateway)5