Search in sources :

Example 11 with Cluster

use of com.sequenceiq.cloudbreak.domain.Cluster in project cloudbreak by hortonworks.

the class ClusterHostServiceRunner method saveSharedRangerService.

private void saveSharedRangerService(Stack stack, Map<String, SaltPillarProperties> servicePillar) {
    Long datalakeId = stack.getDatalakeId();
    if (datalakeId != null) {
        Stack dataLakeStack = stackRepository.findOne(datalakeId);
        Cluster dataLakeCluster = dataLakeStack.getCluster();
        Set<String> groupNames = blueprintProcessorFactory.get(dataLakeCluster.getBlueprint().getBlueprintText()).getHostGroupsWithComponent("RANGER_ADMIN");
        List<HostGroup> groups = dataLakeCluster.getHostGroups().stream().filter(hg -> groupNames.contains(hg.getName())).collect(Collectors.toList());
        Set<String> hostNames = new HashSet<>();
        groups.forEach(hg -> hostNames.addAll(hg.getHostMetadata().stream().map(HostMetadata::getHostName).collect(Collectors.toList())));
        Map<String, Object> rangerMap = new HashMap<>();
        rangerMap.put("servers", hostNames);
        rangerMap.put("port", "6080");
        servicePillar.put("datalake-services", new SaltPillarProperties("/datalake/init.sls", singletonMap("datalake-services", singletonMap("ranger", rangerMap))));
    }
}
Also used : CloudbreakException(com.sequenceiq.cloudbreak.service.CloudbreakException) KerberosConfig(com.sequenceiq.cloudbreak.domain.KerberosConfig) AmbariRepo(com.sequenceiq.cloudbreak.cloud.model.AmbariRepo) InstanceMetaDataRepository(com.sequenceiq.cloudbreak.repository.InstanceMetaDataRepository) KerberosDetailService(com.sequenceiq.cloudbreak.blueprint.kerberos.KerberosDetailService) CloudbreakOrchestratorCancelledException(com.sequenceiq.cloudbreak.orchestrator.exception.CloudbreakOrchestratorCancelledException) StringUtils(org.apache.commons.lang3.StringUtils) InstanceGroup(com.sequenceiq.cloudbreak.domain.InstanceGroup) RdsConfigService(com.sequenceiq.cloudbreak.service.rdsconfig.RdsConfigService) Map(java.util.Map) AmbariSecurityConfigProvider(com.sequenceiq.cloudbreak.service.cluster.ambari.AmbariSecurityConfigProvider) HostOrchestrator(com.sequenceiq.cloudbreak.orchestrator.host.HostOrchestrator) RdsView(com.sequenceiq.cloudbreak.blueprint.template.views.RdsView) Transactional(javax.transaction.Transactional) SaltPillarProperties(com.sequenceiq.cloudbreak.orchestrator.model.SaltPillarProperties) Set(java.util.Set) CloudbreakOrchestratorException(com.sequenceiq.cloudbreak.orchestrator.exception.CloudbreakOrchestratorException) Collectors(java.util.stream.Collectors) GatewayConfigService(com.sequenceiq.cloudbreak.service.GatewayConfigService) ProxyConfigProvider(com.sequenceiq.cloudbreak.service.proxy.ProxyConfigProvider) List(java.util.List) CloudbreakOrchestratorFailedException(com.sequenceiq.cloudbreak.orchestrator.exception.CloudbreakOrchestratorFailedException) Node(com.sequenceiq.cloudbreak.orchestrator.model.Node) GatewayConfig(com.sequenceiq.cloudbreak.orchestrator.model.GatewayConfig) Optional(java.util.Optional) CancellationException(com.sequenceiq.cloudbreak.cloud.scheduler.CancellationException) SmartSenseCredentialConfigService(com.sequenceiq.cloudbreak.service.SmartSenseCredentialConfigService) StackRepository(com.sequenceiq.cloudbreak.repository.StackRepository) RdsType(com.sequenceiq.cloudbreak.api.model.rds.RdsType) HashMap(java.util.HashMap) HostGroupRepository(com.sequenceiq.cloudbreak.repository.HostGroupRepository) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Inject(javax.inject.Inject) HostMetadata(com.sequenceiq.cloudbreak.domain.HostMetadata) StackRepoDetails(com.sequenceiq.cloudbreak.cloud.model.component.StackRepoDetails) Collections.singletonMap(java.util.Collections.singletonMap) RDSConfig(com.sequenceiq.cloudbreak.domain.RDSConfig) Json(com.sequenceiq.cloudbreak.domain.json.Json) ClusterComponentConfigProvider(com.sequenceiq.cloudbreak.service.ClusterComponentConfigProvider) Stack(com.sequenceiq.cloudbreak.domain.Stack) HostGroup(com.sequenceiq.cloudbreak.domain.HostGroup) ClusterDeletionBasedExitCriteriaModel.clusterDeletionBasedModel(com.sequenceiq.cloudbreak.core.bootstrap.service.ClusterDeletionBasedExitCriteriaModel.clusterDeletionBasedModel) InstanceMetaData(com.sequenceiq.cloudbreak.domain.InstanceMetaData) ExposedServices(com.sequenceiq.cloudbreak.domain.ExposedServices) IOException(java.io.IOException) PostgresConfigService(com.sequenceiq.cloudbreak.core.bootstrap.service.container.postgres.PostgresConfigService) Cluster(com.sequenceiq.cloudbreak.domain.Cluster) ExecutorType(com.sequenceiq.cloudbreak.api.model.ExecutorType) Component(org.springframework.stereotype.Component) ExposedService(com.sequenceiq.cloudbreak.api.model.ExposedService) BlueprintProcessorFactory(com.sequenceiq.cloudbreak.blueprint.BlueprintProcessorFactory) AmbariDatabaseMapper(com.sequenceiq.cloudbreak.converter.mapper.AmbariDatabaseMapper) ComponentLocatorService(com.sequenceiq.cloudbreak.service.blueprint.ComponentLocatorService) Comparator(java.util.Comparator) Collections(java.util.Collections) LdapConfig(com.sequenceiq.cloudbreak.domain.LdapConfig) SaltConfig(com.sequenceiq.cloudbreak.orchestrator.model.SaltConfig) HashMap(java.util.HashMap) Cluster(com.sequenceiq.cloudbreak.domain.Cluster) HostGroup(com.sequenceiq.cloudbreak.domain.HostGroup) SaltPillarProperties(com.sequenceiq.cloudbreak.orchestrator.model.SaltPillarProperties) Stack(com.sequenceiq.cloudbreak.domain.Stack) HashSet(java.util.HashSet)

Example 12 with Cluster

use of com.sequenceiq.cloudbreak.domain.Cluster in project cloudbreak by hortonworks.

the class ClusterSyncHandler method accept.

@Override
public void accept(Event<ClusterSyncRequest> event) {
    ClusterSyncRequest request = event.getData();
    ClusterSyncResult result;
    try {
        Stack stack = stackService.getByIdWithLists(request.getStackId());
        String proxyIp = stackUtil.extractAmbariIp(stack);
        String contextPath = stack.getCluster().getGateway().getPath();
        proxyRegistrator.register(stack.getName(), contextPath, proxyIp);
        Cluster cluster = clusterService.retrieveClusterByStackId(request.getStackId());
        ambariClusterStatusUpdater.updateClusterStatus(stack, cluster);
        result = new ClusterSyncResult(request);
    } catch (Exception e) {
        result = new ClusterSyncResult(e.getMessage(), e, request);
    }
    eventBus.notify(result.selector(), new Event<>(event.getHeaders(), result));
}
Also used : ClusterSyncRequest(com.sequenceiq.cloudbreak.reactor.api.event.resource.ClusterSyncRequest) Cluster(com.sequenceiq.cloudbreak.domain.Cluster) ClusterSyncResult(com.sequenceiq.cloudbreak.reactor.api.event.resource.ClusterSyncResult) Stack(com.sequenceiq.cloudbreak.domain.Stack)

Example 13 with Cluster

use of com.sequenceiq.cloudbreak.domain.Cluster in project cloudbreak by hortonworks.

the class DependecyDeletionService method deleteDependencies.

public void deleteDependencies(Stack stack) {
    deleteNetwork(stack.getNetwork());
    deleteCredential(stack.getCredential());
    for (InstanceGroup instanceGroup : stack.getInstanceGroups()) {
        deleteSecurityGroup(instanceGroup.getSecurityGroup());
        deleteTemplate(instanceGroup.getTemplate());
    }
    if (stack.getCluster() != null) {
        Cluster cluster = stack.getCluster();
        deleteBlueprint(cluster.getBlueprint());
        Set<HostGroup> hostGroupsInCluster = hostGroupRepository.findHostGroupsInCluster(cluster.getId());
        for (HostGroup hostGroup : hostGroupsInCluster) {
            hostGroup.getRecipes().forEach(this::deleteRecipe);
        }
    }
}
Also used : Cluster(com.sequenceiq.cloudbreak.domain.Cluster) HostGroup(com.sequenceiq.cloudbreak.domain.HostGroup) InstanceGroup(com.sequenceiq.cloudbreak.domain.InstanceGroup)

Example 14 with Cluster

use of com.sequenceiq.cloudbreak.domain.Cluster in project cloudbreak by hortonworks.

the class ClusterCreationSetupService method prepare.

public Cluster prepare(ClusterRequest request, Stack stack, Blueprint blueprint, IdentityUser user) throws Exception {
    String stackName = stack.getName();
    long start = System.currentTimeMillis();
    Cluster cluster = conversionService.convert(request, Cluster.class);
    cluster.setStack(stack);
    LOGGER.info("Cluster conversion took {} ms for stack {}", System.currentTimeMillis() - start, stackName);
    start = System.currentTimeMillis();
    cluster = clusterDecorator.decorate(cluster, request, blueprint, user, stack);
    LOGGER.info("Cluster object decorated in {} ms for stack {}", System.currentTimeMillis() - start, stackName);
    start = System.currentTimeMillis();
    List<ClusterComponent> components = new ArrayList<>();
    Set<Component> allComponent = componentConfigProvider.getAllComponentsByStackIdAndType(stack.getId(), Sets.newHashSet(ComponentType.AMBARI_REPO_DETAILS, ComponentType.HDP_REPO_DETAILS, ComponentType.IMAGE));
    Optional<Component> stackAmbariRepoConfig = allComponent.stream().filter(c -> c.getComponentType().equals(ComponentType.AMBARI_REPO_DETAILS) && c.getName().equalsIgnoreCase(ComponentType.AMBARI_REPO_DETAILS.name())).findAny();
    Optional<Component> stackHdpRepoConfig = allComponent.stream().filter(c -> c.getComponentType().equals(ComponentType.HDP_REPO_DETAILS) && c.getName().equalsIgnoreCase(ComponentType.HDP_REPO_DETAILS.name())).findAny();
    Optional<Component> stackImageComponent = allComponent.stream().filter(c -> c.getComponentType().equals(ComponentType.IMAGE) && c.getName().equalsIgnoreCase(ComponentType.IMAGE.name())).findAny();
    ClusterComponent ambariRepoConfig = determineAmbariRepoConfig(stackAmbariRepoConfig, request.getAmbariRepoDetailsJson(), stackImageComponent, cluster);
    components.add(ambariRepoConfig);
    ClusterComponent hdpRepoConfig = determineHDPRepoConfig(blueprint, stack.getId(), stackHdpRepoConfig, request, cluster, user, stackImageComponent);
    components.add(hdpRepoConfig);
    checkVDFFile(ambariRepoConfig, hdpRepoConfig, stackName);
    LOGGER.info("Cluster components saved in {} ms for stack {}", System.currentTimeMillis() - start, stackName);
    start = System.currentTimeMillis();
    Cluster savedCluster = clusterService.create(user, stack, cluster, components);
    LOGGER.info("Cluster object creation took {} ms for stack {}", System.currentTimeMillis() - start, stackName);
    return savedCluster;
}
Also used : ComponentType(com.sequenceiq.cloudbreak.common.type.ComponentType) AmbariRepo(com.sequenceiq.cloudbreak.cloud.model.AmbariRepo) Component(com.sequenceiq.cloudbreak.domain.Component) IdentityUser(com.sequenceiq.cloudbreak.common.model.user.IdentityUser) LoggerFactory(org.slf4j.LoggerFactory) FileSystemValidator(com.sequenceiq.cloudbreak.controller.validation.filesystem.FileSystemValidator) ClusterService(com.sequenceiq.cloudbreak.service.cluster.ClusterService) StringUtils(org.apache.commons.lang3.StringUtils) CUSTOM_VDF_REPO_KEY(com.sequenceiq.cloudbreak.cloud.model.component.StackRepoDetails.CUSTOM_VDF_REPO_KEY) JsonNode(com.fasterxml.jackson.databind.JsonNode) Blueprint(com.sequenceiq.cloudbreak.domain.Blueprint) DefaultHDPInfo(com.sequenceiq.cloudbreak.cloud.model.component.DefaultHDPInfo) VDF_REPO_KEY_PREFIX(com.sequenceiq.cloudbreak.cloud.model.component.StackRepoDetails.VDF_REPO_KEY_PREFIX) Set(java.util.Set) AMBARI_VERSION_2_6_0_0(com.sequenceiq.cloudbreak.service.cluster.ambari.AmbariRepositoryVersionService.AMBARI_VERSION_2_6_0_0) Sets(com.google.common.collect.Sets) List(java.util.List) AmbariRepoDetailsJson(com.sequenceiq.cloudbreak.api.model.AmbariRepoDetailsJson) DefaultHDFInfo(com.sequenceiq.cloudbreak.cloud.model.component.DefaultHDFInfo) Entry(java.util.Map.Entry) ComponentConfigProvider(com.sequenceiq.cloudbreak.service.ComponentConfigProvider) Optional(java.util.Optional) AmbariStackDetailsJson(com.sequenceiq.cloudbreak.api.model.AmbariStackDetailsJson) MDCBuilder(com.sequenceiq.cloudbreak.logger.MDCBuilder) SerializationUtils(org.apache.commons.lang3.SerializationUtils) Image(com.sequenceiq.cloudbreak.cloud.model.Image) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) StackRepoDetails(com.sequenceiq.cloudbreak.cloud.model.component.StackRepoDetails) ClusterComponent(com.sequenceiq.cloudbreak.domain.ClusterComponent) Named(javax.inject.Named) Json(com.sequenceiq.cloudbreak.domain.json.Json) StackInfo(com.sequenceiq.cloudbreak.cloud.model.component.StackInfo) CredentialToCloudCredentialConverter(com.sequenceiq.cloudbreak.converter.spi.CredentialToCloudCredentialConverter) Stack(com.sequenceiq.cloudbreak.domain.Stack) ConversionService(org.springframework.core.convert.ConversionService) ClusterDecorator(com.sequenceiq.cloudbreak.service.decorator.ClusterDecorator) AmbariRepositoryVersionService(com.sequenceiq.cloudbreak.service.cluster.ambari.AmbariRepositoryVersionService) Logger(org.slf4j.Logger) ClusterRequest(com.sequenceiq.cloudbreak.api.model.ClusterRequest) DefaultHDFEntries(com.sequenceiq.cloudbreak.cloud.model.component.DefaultHDFEntries) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) CloudCredential(com.sequenceiq.cloudbreak.cloud.model.CloudCredential) Cluster(com.sequenceiq.cloudbreak.domain.Cluster) CloudbreakImageNotFoundException(com.sequenceiq.cloudbreak.core.CloudbreakImageNotFoundException) DefaultHDPEntries(com.sequenceiq.cloudbreak.cloud.model.component.DefaultHDPEntries) BlueprintService(com.sequenceiq.cloudbreak.service.blueprint.BlueprintService) BlueprintUtils(com.sequenceiq.cloudbreak.blueprint.utils.BlueprintUtils) DefaultAmbariRepoService(com.sequenceiq.cloudbreak.service.DefaultAmbariRepoService) JsonUtil(com.sequenceiq.cloudbreak.util.JsonUtil) ClusterComponent(com.sequenceiq.cloudbreak.domain.ClusterComponent) ArrayList(java.util.ArrayList) Cluster(com.sequenceiq.cloudbreak.domain.Cluster) Component(com.sequenceiq.cloudbreak.domain.Component) ClusterComponent(com.sequenceiq.cloudbreak.domain.ClusterComponent)

Example 15 with Cluster

use of com.sequenceiq.cloudbreak.domain.Cluster in project cloudbreak by hortonworks.

the class AmbariClusterService method updateStatus.

@Override
@Transactional(TxType.NEVER)
public void updateStatus(Long stackId, StatusRequest statusRequest) {
    Stack stack = stackService.getByIdWithLists(stackId);
    Cluster cluster = stack.getCluster();
    if (cluster == null) {
        throw new BadRequestException(String.format("There is no cluster installed on stack '%s'.", stack.getName()));
    }
    switch(statusRequest) {
        case SYNC:
            sync(stack);
            break;
        case STOPPED:
            stop(stack, cluster);
            break;
        case STARTED:
            start(stack, cluster);
            break;
        default:
            throw new BadRequestException("Cannot update the status of cluster because status request not valid");
    }
}
Also used : Cluster(com.sequenceiq.cloudbreak.domain.Cluster) BadRequestException(com.sequenceiq.cloudbreak.controller.BadRequestException) Stack(com.sequenceiq.cloudbreak.domain.Stack) Transactional(javax.transaction.Transactional)

Aggregations

Cluster (com.sequenceiq.cloudbreak.domain.Cluster)144 Stack (com.sequenceiq.cloudbreak.domain.Stack)68 Test (org.junit.Test)64 AmbariClient (com.sequenceiq.ambari.client.AmbariClient)31 HostGroup (com.sequenceiq.cloudbreak.domain.HostGroup)26 Blueprint (com.sequenceiq.cloudbreak.domain.Blueprint)22 CloudbreakException (com.sequenceiq.cloudbreak.service.CloudbreakException)22 HashMap (java.util.HashMap)22 HashSet (java.util.HashSet)15 List (java.util.List)15 ArrayList (java.util.ArrayList)13 HttpClientConfig (com.sequenceiq.cloudbreak.client.HttpClientConfig)12 BadRequestException (com.sequenceiq.cloudbreak.controller.BadRequestException)12 BlueprintPreparationObject (com.sequenceiq.cloudbreak.blueprint.BlueprintPreparationObject)11 Matchers.anyString (org.mockito.Matchers.anyString)11 CancellationException (com.sequenceiq.cloudbreak.cloud.scheduler.CancellationException)10 InstanceMetaData (com.sequenceiq.cloudbreak.domain.InstanceMetaData)10 InstanceGroup (com.sequenceiq.cloudbreak.domain.InstanceGroup)9 Json (com.sequenceiq.cloudbreak.domain.json.Json)9 PollingResult (com.sequenceiq.cloudbreak.service.PollingResult)9