Search in sources :

Example 21 with ParcelOperationStatus

use of com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus in project cloudbreak by hortonworks.

the class ClusterComponentUpdaterTest method testRemoveUnusedCdhProductsFromClusterComponentsWhenThereIsNoComponentToDeleteAndAnExtraComponentIsPresent.

@Test
public void testRemoveUnusedCdhProductsFromClusterComponentsWhenThereIsNoComponentToDeleteAndAnExtraComponentIsPresent() {
    Set<ClusterComponent> clusterComponentsByBlueprint = createComponents(Set.of(CDH, FLINK));
    Set<ClusterComponent> clusterComponentsFromDb = createComponents(Set.of(CDH, FLINK));
    clusterComponentsFromDb.add(createImageComponent());
    when(clusterComponentConfigProvider.getComponentsByClusterId(CLUSTER_ID)).thenReturn(clusterComponentsFromDb);
    underTest.removeUnusedCdhProductsFromClusterComponents(CLUSTER_ID, clusterComponentsByBlueprint, new ParcelOperationStatus(Map.of(), Map.of()));
    verify(clusterComponentConfigProvider).getComponentsByClusterId(CLUSTER_ID);
    verifyNoMoreInteractions(clusterComponentConfigProvider);
}
Also used : ClusterComponent(com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent) ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) Test(org.junit.Test)

Example 22 with ParcelOperationStatus

use of com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus in project cloudbreak by hortonworks.

the class ClusterComponentUpdaterTest method testRemoveUnusedCdhProductsFromClusterComponentsWhenThereIsNoComponentToDelete.

@Test
public void testRemoveUnusedCdhProductsFromClusterComponentsWhenThereIsNoComponentToDelete() {
    Set<ClusterComponent> clusterComponentsByBlueprint = createComponents(Set.of(CDH, FLINK));
    Set<ClusterComponent> clusterComponentsFromDb = createComponents(Set.of(CDH, FLINK));
    when(clusterComponentConfigProvider.getComponentsByClusterId(CLUSTER_ID)).thenReturn(clusterComponentsFromDb);
    underTest.removeUnusedCdhProductsFromClusterComponents(CLUSTER_ID, clusterComponentsByBlueprint, new ParcelOperationStatus(Map.of(), Map.of()));
    verify(clusterComponentConfigProvider).getComponentsByClusterId(CLUSTER_ID);
    verifyNoMoreInteractions(clusterComponentConfigProvider);
}
Also used : ClusterComponent(com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent) ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) Test(org.junit.Test)

Example 23 with ParcelOperationStatus

use of com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus in project cloudbreak by hortonworks.

the class ClouderaManagerModificationService method removeUnusedParcels.

@Override
public ParcelOperationStatus removeUnusedParcels(Set<ClusterComponent> usedParcelComponents, Set<String> parcelNamesFromImage) {
    ParcelsResourceApi parcelsResourceApi = clouderaManagerApiFactory.getParcelsResourceApi(apiClient);
    ParcelResourceApi parcelResourceApi = clouderaManagerApiFactory.getParcelResourceApi(apiClient);
    Set<String> usedParcelComponentNames = usedParcelComponents.stream().map(component -> component.getAttributes().getSilent(ClouderaManagerProduct.class).getName()).collect(Collectors.toSet());
    ParcelOperationStatus deactivateStatus = clouderaManagerParcelDecommissionService.deactivateUnusedParcels(parcelsResourceApi, parcelResourceApi, stack.getName(), usedParcelComponentNames, parcelNamesFromImage);
    ParcelOperationStatus undistributeStatus = clouderaManagerParcelDecommissionService.undistributeUnusedParcels(apiClient, parcelsResourceApi, parcelResourceApi, stack, usedParcelComponentNames, parcelNamesFromImage);
    ParcelOperationStatus removalStatus = clouderaManagerParcelDecommissionService.removeUnusedParcels(apiClient, parcelsResourceApi, parcelResourceApi, stack, usedParcelComponentNames, parcelNamesFromImage);
    ParcelOperationStatus result = removalStatus.merge(deactivateStatus).merge(undistributeStatus);
    LOGGER.info("Result of the parcel removal: {}", result);
    return result;
}
Also used : CloudbreakException(com.sequenceiq.cloudbreak.service.CloudbreakException) ApiBatchRequestElement(com.cloudera.api.swagger.model.ApiBatchRequestElement) ClusterClientInitException(com.sequenceiq.cloudbreak.cluster.service.ClusterClientInitException) LoggerFactory(org.slf4j.LoggerFactory) ApiService(com.cloudera.api.swagger.model.ApiService) ApiException(com.cloudera.api.swagger.client.ApiException) ClouderaManagerProductsProvider(com.sequenceiq.cloudbreak.cluster.service.ClouderaManagerProductsProvider) ParcelStatus(com.sequenceiq.cloudbreak.cm.model.ParcelStatus) ClustersResourceApi(com.cloudera.api.swagger.ClustersResourceApi) BigDecimal(java.math.BigDecimal) ApiCommandList(com.cloudera.api.swagger.model.ApiCommandList) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct) CloudbreakEventService(com.sequenceiq.cloudbreak.structuredevent.event.CloudbreakEventService) BatchResourceApi(com.cloudera.api.swagger.BatchResourceApi) Locale(java.util.Locale) Map(java.util.Map) ApiHostRefList(com.cloudera.api.swagger.model.ApiHostRefList) ClouderaManagerResourceApi(com.cloudera.api.swagger.ClouderaManagerResourceApi) HostTemplatesResourceApi(com.cloudera.api.swagger.HostTemplatesResourceApi) CLOUDERAMANAGER_VERSION_7_6_0(com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.CLOUDERAMANAGER_VERSION_7_6_0) Telemetry(com.sequenceiq.common.api.telemetry.model.Telemetry) ParcelResourceApi(com.cloudera.api.swagger.ParcelResourceApi) ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) Versioned(com.sequenceiq.cloudbreak.common.type.Versioned) Predicate(java.util.function.Predicate) Collection(java.util.Collection) HostsResourceApi(com.cloudera.api.swagger.HostsResourceApi) PollingResultErrorHandler(com.sequenceiq.cloudbreak.cm.polling.PollingResultErrorHandler) Set(java.util.Set) ApiBatchResponse(com.cloudera.api.swagger.model.ApiBatchResponse) ExtendedPollingResult(com.sequenceiq.cloudbreak.polling.ExtendedPollingResult) HttpClientConfig(com.sequenceiq.cloudbreak.client.HttpClientConfig) Collectors(java.util.stream.Collectors) ClouderaManagerRepo(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerRepo) Objects(java.util.Objects) List(java.util.List) ApiEntityTag(com.cloudera.api.swagger.model.ApiEntityTag) ApiRestartClusterArgs(com.cloudera.api.swagger.model.ApiRestartClusterArgs) PostConstruct(javax.annotation.PostConstruct) Optional(java.util.Optional) CancellationException(com.sequenceiq.cloudbreak.cloud.scheduler.CancellationException) PollingResult(com.sequenceiq.cloudbreak.polling.PollingResult) InstanceGroup(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup) ClusterModificationService(com.sequenceiq.cloudbreak.cluster.api.ClusterModificationService) ApiCallback(com.cloudera.api.swagger.client.ApiCallback) Stack(com.sequenceiq.cloudbreak.domain.stack.Stack) Cluster(com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster) ApiCommand(com.cloudera.api.swagger.model.ApiCommand) ApiClient(com.cloudera.api.swagger.client.ApiClient) ClouderaManagerClientInitException(com.sequenceiq.cloudbreak.cm.client.ClouderaManagerClientInitException) ApiConfigStalenessStatus(com.cloudera.api.swagger.model.ApiConfigStalenessStatus) CLUSTER_CM_CLUSTER_SERVICES_STARTING(com.sequenceiq.cloudbreak.event.ResourceEvent.CLUSTER_CM_CLUSTER_SERVICES_STARTING) Function(java.util.function.Function) Scope(org.springframework.context.annotation.Scope) ServicesResourceApi(com.cloudera.api.swagger.ServicesResourceApi) ClouderaManagerPollingServiceProvider(com.sequenceiq.cloudbreak.cm.polling.ClouderaManagerPollingServiceProvider) Inject(javax.inject.Inject) StackRepoDetails(com.sequenceiq.cloudbreak.cloud.model.component.StackRepoDetails) ClouderaManagerApiClientProvider(com.sequenceiq.cloudbreak.cm.client.ClouderaManagerApiClientProvider) UPDATE_IN_PROGRESS(com.sequenceiq.cloudbreak.api.endpoint.v4.common.Status.UPDATE_IN_PROGRESS) Service(org.springframework.stereotype.Service) ApiHost(com.cloudera.api.swagger.model.ApiHost) CMRepositoryVersionUtil.isVersionNewerOrEqualThanLimited(com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.isVersionNewerOrEqualThanLimited) CLUSTER_CM_CLUSTER_SERVICES_RESTARTING(com.sequenceiq.cloudbreak.event.ResourceEvent.CLUSTER_CM_CLUSTER_SERVICES_RESTARTING) ApiBatchRequest(com.cloudera.api.swagger.model.ApiBatchRequest) CLOUDERAMANAGER_VERSION_7_5_1(com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.CLOUDERAMANAGER_VERSION_7_5_1) ApiServiceState(com.cloudera.api.swagger.model.ApiServiceState) ClusterComponentConfigProvider(com.sequenceiq.cloudbreak.cluster.service.ClusterComponentConfigProvider) CLUSTER_CM_UPDATED_REMOTE_DATA_CONTEXT(com.sequenceiq.cloudbreak.event.ResourceEvent.CLUSTER_CM_UPDATED_REMOTE_DATA_CONTEXT) Logger(org.slf4j.Logger) CLOUDERAMANAGER_VERSION_7_1_0(com.sequenceiq.cloudbreak.cmtemplate.CMRepositoryVersionUtil.CLOUDERAMANAGER_VERSION_7_1_0) CLUSTER_CM_CLUSTER_SERVICES_STARTED(com.sequenceiq.cloudbreak.event.ResourceEvent.CLUSTER_CM_CLUSTER_SERVICES_STARTED) ResourceEvent(com.sequenceiq.cloudbreak.event.ResourceEvent) URLUtils(com.sequenceiq.cloudbreak.util.URLUtils) ApplicationContext(org.springframework.context.ApplicationContext) Consumer(java.util.function.Consumer) ClusterComponent(com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent) HostGroup(com.sequenceiq.cloudbreak.domain.stack.cluster.host.HostGroup) InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) HTTPMethod(com.cloudera.api.swagger.model.HTTPMethod) CLUSTER_CM_UPDATING_REMOTE_DATA_CONTEXT(com.sequenceiq.cloudbreak.event.ResourceEvent.CLUSTER_CM_UPDATING_REMOTE_DATA_CONTEXT) ApiHostRef(com.cloudera.api.swagger.model.ApiHostRef) StackType(com.sequenceiq.cloudbreak.api.endpoint.v4.common.StackType) MgmtServiceResourceApi(com.cloudera.api.swagger.MgmtServiceResourceApi) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Collections(java.util.Collections) ParcelsResourceApi(com.cloudera.api.swagger.ParcelsResourceApi) ClouderaManagerApiFactory(com.sequenceiq.cloudbreak.cm.client.retry.ClouderaManagerApiFactory) ParcelsResourceApi(com.cloudera.api.swagger.ParcelsResourceApi) ParcelResourceApi(com.cloudera.api.swagger.ParcelResourceApi) ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) ClouderaManagerProduct(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)

Example 24 with ParcelOperationStatus

use of com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus in project cloudbreak by hortonworks.

the class ClouderaManagerParcelDecommissionService method runSingleParcelCommand.

private ParcelOperationStatus runSingleParcelCommand(String commandName, String stackName, Entry<String, String> parcelForCommand, ParcelCommand parcelCommand) {
    Parcel parcel = new Parcel(parcelForCommand.getKey(), parcelForCommand.getValue());
    try {
        LOGGER.debug("[{}] Start for {} parcel", commandName, parcel);
        parcelCommand.apply(stackName, parcel.getName(), parcel.getVersion());
        LOGGER.info("[{}] Successful for parcel: {}", commandName, parcel);
        return new ParcelOperationStatus().withSuccessful(parcel.getName(), parcel.getVersion());
    } catch (ApiException e) {
        LOGGER.info(String.format("[%s] Failed for parcel: %s", commandName, parcel), e);
        return new ParcelOperationStatus().withFailed(parcel.getName(), parcel.getVersion());
    }
}
Also used : ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) ApiParcel(com.cloudera.api.swagger.model.ApiParcel) ApiException(com.cloudera.api.swagger.client.ApiException)

Example 25 with ParcelOperationStatus

use of com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus in project cloudbreak by hortonworks.

the class ClusterUpscaleServiceTest method testInstallServicesOnNewHostWithRestart.

@Test
public void testInstallServicesOnNewHostWithRestart() throws CloudbreakException {
    Stack stack = new Stack();
    stack.setId(1L);
    Cluster cluster = new Cluster();
    cluster.setId(2L);
    stack.setCluster(cluster);
    when(stackService.getByIdWithClusterInTransaction(eq(1L))).thenReturn(stack);
    ClusterApi clusterApi = mock(ClusterApi.class);
    when(clusterApiConnectors.getConnector(any(Stack.class))).thenReturn(clusterApi);
    HostGroup hostGroup = new HostGroup();
    InstanceGroup instanceGroup = new InstanceGroup();
    InstanceMetaData im1 = new InstanceMetaData();
    im1.setInstanceStatus(InstanceStatus.SERVICES_HEALTHY);
    InstanceMetaData im2 = new InstanceMetaData();
    im2.setInstanceStatus(InstanceStatus.SERVICES_HEALTHY);
    InstanceMetaData im3 = new InstanceMetaData();
    im3.setInstanceStatus(InstanceStatus.SERVICES_HEALTHY);
    instanceGroup.setInstanceMetaData(Set.of(im1, im2, im3));
    hostGroup.setInstanceGroup(instanceGroup);
    when(hostGroupService.getByClusterWithRecipes(any())).thenReturn(Set.of(hostGroup));
    when(parcelService.removeUnusedParcelComponents(stack)).thenReturn(new ParcelOperationStatus(Collections.emptyMap(), Collections.emptyMap()));
    underTest.installServicesOnNewHosts(1L, Set.of("master"), true, true);
    verify(clusterApi, times(1)).upscaleCluster(any());
    verify(clusterApi, times(1)).restartAll(false);
    verify(parcelService).removeUnusedParcelComponents(stack);
}
Also used : InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) ClusterApi(com.sequenceiq.cloudbreak.cluster.api.ClusterApi) ParcelOperationStatus(com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus) Cluster(com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster) HostGroup(com.sequenceiq.cloudbreak.domain.stack.cluster.host.HostGroup) Stack(com.sequenceiq.cloudbreak.domain.stack.Stack) InstanceGroup(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup) Test(org.junit.jupiter.api.Test)

Aggregations

ParcelOperationStatus (com.sequenceiq.cloudbreak.cluster.model.ParcelOperationStatus)25 Test (org.junit.jupiter.api.Test)16 Stack (com.sequenceiq.cloudbreak.domain.stack.Stack)15 ClusterComponent (com.sequenceiq.cloudbreak.domain.stack.cluster.ClusterComponent)8 ApiException (com.cloudera.api.swagger.client.ApiException)7 HostGroup (com.sequenceiq.cloudbreak.domain.stack.cluster.host.HostGroup)7 ExtendedPollingResult (com.sequenceiq.cloudbreak.polling.ExtendedPollingResult)7 ApiParcelList (com.cloudera.api.swagger.model.ApiParcelList)6 ClouderaManagerProduct (com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerProduct)5 ParcelResourceApi (com.cloudera.api.swagger.ParcelResourceApi)3 ParcelsResourceApi (com.cloudera.api.swagger.ParcelsResourceApi)3 ApiClient (com.cloudera.api.swagger.client.ApiClient)3 ApiParcel (com.cloudera.api.swagger.model.ApiParcel)3 Cluster (com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster)3 InstanceGroup (com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup)3 InstanceMetaData (com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData)3 CloudbreakException (com.sequenceiq.cloudbreak.service.CloudbreakException)3 Test (org.junit.Test)3 HashMultimap (com.google.common.collect.HashMultimap)2 Multimap (com.google.common.collect.Multimap)2