Search in sources :

Example 66 with StackV4Response

use of com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response in project cloudbreak by hortonworks.

the class ProvisionerServiceTest method waitCloudbreakClusterDeletionButFailed.

@Test
void waitCloudbreakClusterDeletionButFailed() {
    long clusterId = CLUSTER_ID.incrementAndGet();
    SdxCluster sdxCluster = generateValidSdxCluster(clusterId);
    when(sdxService.getById(clusterId)).thenReturn(sdxCluster);
    StackV4Response firstStackV4Response = new StackV4Response();
    firstStackV4Response.setStatus(Status.AVAILABLE);
    StackV4Response secondStackV4Response = new StackV4Response();
    secondStackV4Response.setStatus(Status.DELETE_FAILED);
    when(stackV4Endpoint.get(anyLong(), eq(sdxCluster.getClusterName()), anySet(), anyString())).thenReturn(firstStackV4Response).thenReturn(secondStackV4Response);
    when(regionAwareInternalCrnGenerator.getInternalCrnForServiceAsString()).thenReturn("crn:cdp:datahub:us-west-1:altus:user:__internal__actor__");
    when(regionAwareInternalCrnGeneratorFactory.iam()).thenReturn(regionAwareInternalCrnGenerator);
    PollingConfig pollingConfig = new PollingConfig(10, TimeUnit.MILLISECONDS, 500, TimeUnit.MILLISECONDS);
    Assertions.assertThrows(UserBreakException.class, () -> underTest.waitCloudbreakClusterDeletion(clusterId, pollingConfig));
}
Also used : StackV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) Test(org.junit.jupiter.api.Test)

Example 67 with StackV4Response

use of com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response in project cloudbreak by hortonworks.

the class ProvisionerServiceTest method startStackDeletionButClientError.

@Test
void startStackDeletionButClientError() {
    long clusterId = CLUSTER_ID.incrementAndGet();
    SdxCluster sdxCluster = generateValidSdxCluster(clusterId);
    when(sdxService.getById(clusterId)).thenReturn(sdxCluster);
    StackV4Response stackV4Response = new StackV4Response();
    stackV4Response.setStatus(Status.CREATE_FAILED);
    WebApplicationException webApplicationException = new WebApplicationException();
    doThrow(webApplicationException).when(stackV4Endpoint).deleteInternal(anyLong(), eq(sdxCluster.getClusterName()), eq(Boolean.FALSE), nullable(String.class));
    when(webApplicationExceptionMessageExtractor.getErrorMessage(webApplicationException)).thenReturn("web-error");
    when(regionAwareInternalCrnGenerator.getInternalCrnForServiceAsString()).thenReturn("crn:cdp:datahub:us-west-1:altus:user:__internal__actor__");
    when(regionAwareInternalCrnGeneratorFactory.iam()).thenReturn(regionAwareInternalCrnGenerator);
    RuntimeException actual = Assertions.assertThrows(RuntimeException.class, () -> underTest.startStackDeletion(clusterId, false));
    Assertions.assertEquals("Cannot delete cluster, error happened during the operation: web-error", actual.getMessage());
    verify(stackV4Endpoint).deleteInternal(eq(0L), eq(null), eq(false), nullable(String.class));
}
Also used : StackV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response) WebApplicationException(javax.ws.rs.WebApplicationException) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) ArgumentMatchers.anyString(org.mockito.ArgumentMatchers.anyString) Test(org.junit.jupiter.api.Test)

Example 68 with StackV4Response

use of com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response in project cloudbreak by hortonworks.

the class StackIdViewToStackResponseConverter method convert.

public StackV4Response convert(StackIdView source) {
    StackV4Response stackJson = new StackV4Response();
    stackJson.setId(source.getId());
    stackJson.setName(source.getName());
    stackJson.setCrn(source.getCrn());
    return stackJson;
}
Also used : StackV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response)

Example 69 with StackV4Response

use of com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response in project cloudbreak by hortonworks.

the class SdxService method getCdpVersion.

private Optional<String> getCdpVersion(StackV4Response stack) {
    String stackName = stack.getName();
    ClusterV4Response cluster = stack.getCluster();
    if (cluster != null) {
        ClouderaManagerV4Response cm = cluster.getCm();
        if (cm != null) {
            LOGGER.info("Repository details are available for cluster: {}: {}", stackName, cm);
            List<ClouderaManagerProductV4Response> products = cm.getProducts();
            if (products != null && !products.isEmpty()) {
                Optional<ClouderaManagerProductV4Response> cdpOpt = products.stream().filter(p -> "CDH".equals(p.getName())).findFirst();
                if (cdpOpt.isPresent()) {
                    return getRuntimeVersionFromCdpVersion(cdpOpt.get().getVersion());
                }
            }
        }
    }
    return Optional.empty();
}
Also used : SecurityRuleV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.util.requests.SecurityRuleV4Request) AttachRecipeV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.recipe.AttachRecipeV4Response) RangerRazEnabledV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.RangerRazEnabledV4Response) TransactionRuntimeExecutionException(com.sequenceiq.cloudbreak.common.service.TransactionService.TransactionRuntimeExecutionException) StringUtils(org.apache.commons.lang3.StringUtils) AwsInstanceTemplateV4Parameters(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.parameter.template.AwsInstanceTemplateV4Parameters) AwsInstanceTemplateV4SpotParameters(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.parameter.template.AwsInstanceTemplateV4SpotParameters) ThreadBasedUserCrnProvider(com.sequenceiq.cloudbreak.auth.ThreadBasedUserCrnProvider) Matcher(java.util.regex.Matcher) Pair(org.apache.commons.lang3.tuple.Pair) ValidationResult(com.sequenceiq.cloudbreak.validation.ValidationResult) ValidationResultBuilder(com.sequenceiq.cloudbreak.validation.ValidationResult.ValidationResultBuilder) Map(java.util.Map) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) InstanceGroupV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.instancegroup.InstanceGroupV4Request) EnumSet(java.util.EnumSet) AzureStackV4Parameters(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.parameter.stack.AzureStackV4Parameters) SdxRecipe(com.sequenceiq.sdx.api.model.SdxRecipe) UpdateRecipesV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.recipe.UpdateRecipesV4Response) Set(java.util.Set) Crn(com.sequenceiq.cloudbreak.auth.crn.Crn) VersionComparator(com.sequenceiq.cloudbreak.util.VersionComparator) SdxClusterRequest(com.sequenceiq.sdx.api.model.SdxClusterRequest) Stream(java.util.stream.Stream) CertExpirationState(com.sequenceiq.common.api.type.CertExpirationState) SdxAzureBase(com.sequenceiq.sdx.api.model.SdxAzureBase) SdxClusterResizeRequest(com.sequenceiq.sdx.api.model.SdxClusterResizeRequest) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) OwnerAssignmentService(com.sequenceiq.authorization.service.OwnerAssignmentService) CollectionUtils(org.apache.commons.collections4.CollectionUtils) StringUtils.isNotEmpty(org.apache.commons.lang3.StringUtils.isNotEmpty) ArrayList(java.util.ArrayList) CUSTOM(com.sequenceiq.sdx.api.model.SdxClusterShape.CUSTOM) LinkedHashMap(java.util.LinkedHashMap) Value(org.springframework.beans.factory.annotation.Value) AWS(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform.AWS) CrnResourceDescriptor(com.sequenceiq.cloudbreak.auth.crn.CrnResourceDescriptor) SecurityGroupV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.instancegroup.securitygroup.SecurityGroupV4Request) StackResponseEntries(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.StackResponseEntries) EnumUtils(org.apache.commons.lang3.EnumUtils) Service(org.springframework.stereotype.Service) CustomDomainSettingsV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.customdomain.CustomDomainSettingsV4Request) InstanceTemplateV4Base(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.InstanceTemplateV4Base) LinkedHashSet(java.util.LinkedHashSet) BadRequestException(com.sequenceiq.cloudbreak.common.exception.BadRequestException) InstanceGroupV4Base(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.base.InstanceGroupV4Base) ImageSettingsV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.image.ImageSettingsV4Request) SdxCustomClusterRequest(com.sequenceiq.sdx.api.model.SdxCustomClusterRequest) IOException(java.io.IOException) InstanceGroupType(com.sequenceiq.common.api.type.InstanceGroupType) RecipeV4Endpoint(com.sequenceiq.cloudbreak.api.endpoint.v4.recipes.RecipeV4Endpoint) DetachRecipeV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.recipe.DetachRecipeV4Request) FileSystemType(com.sequenceiq.common.model.FileSystemType) ClouderaManagerProductV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.clouderamanager.ClouderaManagerProductV4Response) EntitlementService(com.sequenceiq.cloudbreak.auth.altus.EntitlementService) CompactViewV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.common.CompactViewV4Response) LoggerFactory(org.slf4j.LoggerFactory) FlowCancelService(com.sequenceiq.flow.service.FlowCancelService) JsonUtil(com.sequenceiq.cloudbreak.common.json.JsonUtil) TransactionService(com.sequenceiq.cloudbreak.common.service.TransactionService) CloudbreakInternalCrnClient(com.sequenceiq.cloudbreak.client.CloudbreakInternalCrnClient) AuthorizationResourceType(com.sequenceiq.authorization.resource.AuthorizationResourceType) SdxCloudStorageRequest(com.sequenceiq.sdx.api.model.SdxCloudStorageRequest) HierarchyAuthResourcePropertyProvider(com.sequenceiq.authorization.service.HierarchyAuthResourcePropertyProvider) NotFoundException(com.sequenceiq.cloudbreak.common.exception.NotFoundException) StackV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response) ResourceWithId(com.sequenceiq.authorization.service.list.ResourceWithId) PayloadContext(com.sequenceiq.cloudbreak.common.event.PayloadContext) StackV4Endpoint(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.StackV4Endpoint) RecipeViewV4Responses(com.sequenceiq.cloudbreak.api.endpoint.v4.recipes.responses.RecipeViewV4Responses) Versioned(com.sequenceiq.cloudbreak.common.type.Versioned) Collection(java.util.Collection) CDPConfigService(com.sequenceiq.datalake.configuration.CDPConfigService) Collectors(java.util.stream.Collectors) SdxAwsSpotParameters(com.sequenceiq.sdx.api.model.SdxAwsSpotParameters) RegionAwareInternalCrnGeneratorFactory(com.sequenceiq.cloudbreak.auth.crn.RegionAwareInternalCrnGeneratorFactory) Json(com.sequenceiq.cloudbreak.common.json.Json) SdxStatusService(com.sequenceiq.datalake.service.sdx.status.SdxStatusService) List(java.util.List) AttachRecipeV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.recipe.AttachRecipeV4Request) TransactionExecutionException(com.sequenceiq.cloudbreak.common.service.TransactionService.TransactionExecutionException) CloudStorageRequest(com.sequenceiq.common.api.cloudstorage.CloudStorageRequest) Optional(java.util.Optional) Pattern(java.util.regex.Pattern) SdxInstanceGroupRequest(com.sequenceiq.sdx.api.model.SdxInstanceGroupRequest) Clock(com.sequenceiq.cloudbreak.common.service.Clock) ClouderaManagerV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.clouderamanager.ClouderaManagerV4Response) DatalakeStatusEnum(com.sequenceiq.datalake.entity.DatalakeStatusEnum) SdxClusterShape(com.sequenceiq.sdx.api.model.SdxClusterShape) SdxReactorFlowManager(com.sequenceiq.datalake.flow.SdxReactorFlowManager) ImageCatalogService(com.sequenceiq.datalake.service.imagecatalog.ImageCatalogService) MDCBuilder(com.sequenceiq.cloudbreak.logger.MDCBuilder) HashMap(java.util.HashMap) SdxClusterRepository(com.sequenceiq.datalake.repository.SdxClusterRepository) CloudPlatform(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform) HashSet(java.util.HashSet) Inject(javax.inject.Inject) FlowIdentifier(com.sequenceiq.flow.api.model.FlowIdentifier) Nonnull(javax.annotation.Nonnull) NotFoundException.notFound(com.sequenceiq.cloudbreak.common.exception.NotFoundException.notFound) SdxStatusEntity(com.sequenceiq.datalake.entity.SdxStatusEntity) Logger(org.slf4j.Logger) Benchmark.measure(com.sequenceiq.cloudbreak.util.Benchmark.measure) PayloadContextProvider(com.sequenceiq.flow.core.PayloadContextProvider) RegionAwareCrnGenerator(com.sequenceiq.cloudbreak.auth.crn.RegionAwareCrnGenerator) EnvironmentClientService(com.sequenceiq.datalake.service.EnvironmentClientService) SdxAwsBase(com.sequenceiq.sdx.api.model.SdxAwsBase) ImageV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.imagecatalog.responses.ImageV4Response) UpdateRecipesV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.recipe.UpdateRecipesV4Request) StackViewV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackViewV4Response) NameOrCrn(com.sequenceiq.cloudbreak.api.endpoint.v4.dto.NameOrCrn) DetachRecipeV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.recipe.DetachRecipeV4Response) CrnParseException(com.sequenceiq.cloudbreak.auth.crn.CrnParseException) StackV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.StackV4Request) ResourceIdProvider(com.sequenceiq.flow.core.ResourceIdProvider) AZURE(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform.AZURE) ClusterV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.ClusterV4Response) Comparator(java.util.Comparator) Collections(java.util.Collections) InstanceTemplateV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.instancegroup.template.InstanceTemplateV4Request) ClusterV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.ClusterV4Response) ClouderaManagerProductV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.clouderamanager.ClouderaManagerProductV4Response) ClouderaManagerV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.clouderamanager.ClouderaManagerV4Response)

Example 70 with StackV4Response

use of com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response in project cloudbreak by hortonworks.

the class SdxService method resizeSdx.

public Pair<SdxCluster, FlowIdentifier> resizeSdx(final String userCrn, final String clusterName, final SdxClusterResizeRequest sdxClusterResizeRequest) {
    LOGGER.info("Re-sizing SDX cluster with name {}", clusterName);
    String accountIdFromCrn = getAccountIdFromCrn(userCrn);
    String environmentName = sdxClusterResizeRequest.getEnvironment();
    SdxClusterShape shape = sdxClusterResizeRequest.getClusterShape();
    final SdxCluster sdxCluster = sdxClusterRepository.findByAccountIdAndClusterNameAndDeletedIsNullAndDetachedIsFalse(accountIdFromCrn, clusterName).orElseThrow(() -> notFound("SDX cluster", clusterName).get());
    MDCBuilder.buildMdcContext(sdxCluster);
    validateSdxResizeRequest(sdxCluster, accountIdFromCrn, shape);
    StackV4Response stackV4Response = getDetail(clusterName, Set.of(StackResponseEntries.HARDWARE_INFO.getEntryName(), StackResponseEntries.EVENTS.getEntryName()), accountIdFromCrn);
    DetailedEnvironmentResponse environment = validateAndGetEnvironment(environmentName);
    SdxCluster newSdxCluster = validateAndCreateNewSdxCluster(userCrn, clusterName, sdxCluster.getRuntime(), shape, sdxCluster.isRangerRazEnabled(), sdxCluster.isEnableMultiAz(), environment);
    newSdxCluster.setTags(sdxCluster.getTags());
    newSdxCluster.setCrn(sdxCluster.getCrn());
    CloudPlatform cloudPlatform = CloudPlatform.valueOf(environment.getCloudPlatform());
    if (!StringUtils.isBlank(sdxCluster.getCloudStorageBaseLocation())) {
        newSdxCluster.setCloudStorageBaseLocation(sdxCluster.getCloudStorageBaseLocation());
        newSdxCluster.setCloudStorageFileSystemType(sdxCluster.getCloudStorageFileSystemType());
    } else if (!CloudPlatform.YARN.equalsIgnoreCase(cloudPlatform.name()) && !CloudPlatform.GCP.equalsIgnoreCase(cloudPlatform.name()) && !CloudPlatform.MOCK.equalsIgnoreCase(cloudPlatform.name())) {
        throw new BadRequestException("Cloud storage parameter is required.");
    }
    newSdxCluster.setDatabaseAvailabilityType(sdxCluster.getDatabaseAvailabilityType());
    newSdxCluster.setDatabaseEngineVersion(sdxCluster.getDatabaseEngineVersion());
    StackV4Request stackRequest = getStackRequest(shape, sdxCluster.isRangerRazEnabled(), null, cloudPlatform, sdxCluster.getRuntime(), null);
    if (shape == SdxClusterShape.MEDIUM_DUTY_HA) {
        // This is added to make sure the host name used by Light and Medium duty are not the same.
        CustomDomainSettingsV4Request customDomainSettingsV4Request = new CustomDomainSettingsV4Request();
        customDomainSettingsV4Request.setHostname(sdxCluster.getClusterName() + SDX_RESIZE_NAME_SUFFIX);
        stackRequest.setCustomDomain(customDomainSettingsV4Request);
    }
    prepareCloudStorageForStack(stackRequest, stackV4Response, newSdxCluster, environment);
    prepareDefaultSecurityConfigs(null, stackRequest, cloudPlatform);
    try {
        if (!StringUtils.isBlank(sdxCluster.getStackRequestToCloudbreak())) {
            StackV4Request stackV4RequestOrig = JsonUtil.readValue(sdxCluster.getStackRequestToCloudbreak(), StackV4Request.class);
            stackRequest.setImage(stackV4RequestOrig.getImage());
        }
    } catch (IOException ioException) {
        LOGGER.error("Failed to re-use the image catalog. Will use default catalog", ioException);
    }
    stackRequest.setResourceCrn(newSdxCluster.getCrn());
    newSdxCluster.setStackRequest(stackRequest);
    FlowIdentifier flowIdentifier = sdxReactorFlowManager.triggerSdxResize(sdxCluster.getId(), newSdxCluster);
    return Pair.of(sdxCluster, flowIdentifier);
}
Also used : CloudPlatform(com.sequenceiq.cloudbreak.common.mappable.CloudPlatform) StackV4Response(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response) StackV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.StackV4Request) CustomDomainSettingsV4Request(com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.request.customdomain.CustomDomainSettingsV4Request) SdxCluster(com.sequenceiq.datalake.entity.SdxCluster) DetailedEnvironmentResponse(com.sequenceiq.environment.api.v1.environment.model.response.DetailedEnvironmentResponse) BadRequestException(com.sequenceiq.cloudbreak.common.exception.BadRequestException) IOException(java.io.IOException) FlowIdentifier(com.sequenceiq.flow.api.model.FlowIdentifier) SdxClusterShape(com.sequenceiq.sdx.api.model.SdxClusterShape)

Aggregations

StackV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.StackV4Response)101 Test (org.junit.jupiter.api.Test)26 SdxCluster (com.sequenceiq.datalake.entity.SdxCluster)22 Stack (com.sequenceiq.cloudbreak.domain.stack.Stack)19 ClusterV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.cluster.ClusterV4Response)14 FlowIdentifier (com.sequenceiq.flow.api.model.FlowIdentifier)13 Test (org.junit.Test)12 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)11 BadRequestException (com.sequenceiq.cloudbreak.common.exception.BadRequestException)8 TelemetryResponse (com.sequenceiq.common.api.telemetry.response.TelemetryResponse)8 MockedTestContext (com.sequenceiq.it.cloudbreak.context.MockedTestContext)7 TestContext (com.sequenceiq.it.cloudbreak.context.TestContext)7 DistroXTestDto (com.sequenceiq.it.cloudbreak.dto.distrox.DistroXTestDto)7 ArrayList (java.util.ArrayList)7 Test (org.testng.annotations.Test)7 InstanceGroupV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.instancegroup.InstanceGroupV4Response)6 TagsV4Response (com.sequenceiq.cloudbreak.api.endpoint.v4.stacks.response.tags.TagsV4Response)6 TransactionExecutionException (com.sequenceiq.cloudbreak.common.service.TransactionService.TransactionExecutionException)6 TransactionRuntimeExecutionException (com.sequenceiq.cloudbreak.common.service.TransactionService.TransactionRuntimeExecutionException)6 BaseDiagnosticsCollectionRequest (com.sequenceiq.common.api.diagnostics.BaseDiagnosticsCollectionRequest)6