Search in sources :

Example 66 with InstanceMetaData

use of com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData in project cloudbreak by hortonworks.

the class ClouderaManagerSetupService method prepareTemplate.

@Override
public String prepareTemplate(Map<HostGroup, List<InstanceMetaData>> instanceMetaDataByHostGroup, TemplatePreparationObject templatePreparationObject, String sdxContext, String sdxCrn, KerberosConfig kerberosConfig) {
    Long clusterId = stack.getCluster().getId();
    try {
        Set<InstanceMetaData> instances = instanceMetaDataByHostGroup.values().stream().flatMap(Collection::stream).collect(Collectors.toSet());
        waitForHosts(instances);
        String sdxContextName = Optional.ofNullable(sdxContext).map(this::setupRemoteDataContext).orElse(null);
        ClouderaManagerRepo clouderaManagerRepoDetails = clusterComponentProvider.getClouderaManagerRepoDetails(clusterId);
        ApiClusterTemplate apiClusterTemplate = getCmTemplate(templatePreparationObject, sdxContextName, instanceMetaDataByHostGroup, clouderaManagerRepoDetails, clusterId);
        return getExtendedBlueprintText(apiClusterTemplate);
    } catch (CancellationException cancellationException) {
        throw cancellationException;
    } catch (Exception e) {
        throw mapException(e);
    }
}
Also used : InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) ClouderaManagerRepo(com.sequenceiq.cloudbreak.cloud.model.ClouderaManagerRepo) CancellationException(com.sequenceiq.cloudbreak.cloud.scheduler.CancellationException) CloudbreakException(com.sequenceiq.cloudbreak.service.CloudbreakException) ClusterClientInitException(com.sequenceiq.cloudbreak.cluster.service.ClusterClientInitException) ApiException(com.cloudera.api.swagger.client.ApiException) CancellationException(com.sequenceiq.cloudbreak.cloud.scheduler.CancellationException) ClouderaManagerClientInitException(com.sequenceiq.cloudbreak.cm.client.ClouderaManagerClientInitException) CloudStorageConfigurationFailedException(com.sequenceiq.cloudbreak.cm.exception.CloudStorageConfigurationFailedException) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) CloudbreakServiceException(com.sequenceiq.cloudbreak.common.exception.CloudbreakServiceException) ApiClusterTemplate(com.cloudera.api.swagger.model.ApiClusterTemplate)

Example 67 with InstanceMetaData

use of com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData in project cloudbreak by hortonworks.

the class ClouderaHostGroupAssociationBuilder method buildHostGroupAssociation.

private List<Map<String, String>> buildHostGroupAssociation(List<InstanceMetaData> instanceMetadataList) {
    List<Map<String, String>> hostInfoForHostGroup = new ArrayList<>();
    for (InstanceMetaData metaData : instanceMetadataList) {
        Map<String, String> hostInfo = new HashMap<>();
        hostInfo.put(ClusterHostAttributes.FQDN, metaData.getDiscoveryFQDN());
        NullUtil.putIfPresent(hostInfo, ClusterHostAttributes.RACK_ID, metaData.getRackId());
        hostInfoForHostGroup.add(hostInfo);
    }
    return hostInfoForHostGroup;
}
Also used : InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) HashMap(java.util.HashMap) ArrayList(java.util.ArrayList) Map(java.util.Map) HashMap(java.util.HashMap)

Example 68 with InstanceMetaData

use of com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData in project cloudbreak by hortonworks.

the class StackCreatorService method prepareInstanceMetaDataSubnetAndAvailabilityZoneAndRackId.

private void prepareInstanceMetaDataSubnetAndAvailabilityZoneAndRackId(String stackSubnetId, String stackAz, InstanceGroup instanceGroup) {
    for (InstanceMetaData instanceMetaData : instanceGroup.getAllInstanceMetaData()) {
        if (Strings.isNullOrEmpty(instanceMetaData.getSubnetId()) && Strings.isNullOrEmpty(instanceMetaData.getAvailabilityZone())) {
            instanceMetaData.setSubnetId(stackSubnetId);
            instanceMetaData.setAvailabilityZone(stackAz);
        }
        instanceMetaData.setRackId(multiAzCalculatorService.determineRackId(instanceMetaData.getSubnetId(), instanceMetaData.getAvailabilityZone()));
    }
}
Also used : InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData)

Example 69 with InstanceMetaData

use of com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData in project cloudbreak by hortonworks.

the class StackCreatorService method fillInstanceMetadata.

@VisibleForTesting
void fillInstanceMetadata(DetailedEnvironmentResponse environment, Stack stack) {
    long privateIdNumber = 0;
    Map<String, String> subnetAzPairs = multiAzCalculatorService.prepareSubnetAzMap(environment);
    String stackSubnetId = getStackSubnetIdIfExists(stack);
    String stackAz = stackSubnetId == null ? null : subnetAzPairs.get(stackSubnetId);
    for (InstanceGroup instanceGroup : sortInstanceGroups(stack)) {
        for (InstanceMetaData instanceMetaData : instanceGroup.getAllInstanceMetaData()) {
            instanceMetaData.setPrivateId(privateIdNumber++);
            instanceMetaData.setInstanceStatus(InstanceStatus.REQUESTED);
        }
        multiAzCalculatorService.calculateByRoundRobin(subnetAzPairs, instanceGroup);
        prepareInstanceMetaDataSubnetAndAvailabilityZoneAndRackId(stackSubnetId, stackAz, instanceGroup);
    }
}
Also used : InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) InstanceGroup(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup) VisibleForTesting(com.google.common.annotations.VisibleForTesting)

Example 70 with InstanceMetaData

use of com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData in project cloudbreak by hortonworks.

the class VmStatusCheckerConclusionStep method checkCMForInstanceStatuses.

private Conclusion checkCMForInstanceStatuses(ClusterApi connector, Set<InstanceMetaData> runningInstances, Long clusterId) {
    ExtendedHostStatuses extendedHostStatuses = connector.clusterStatusService().getExtendedHostStatuses(runtimeVersionService.getRuntimeVersion(clusterId));
    Map<HostName, Set<HealthCheck>> hostStatuses = extendedHostStatuses.getHostsHealth();
    Map<String, String> unhealthyHosts = hostStatuses.keySet().stream().filter(hostName -> !extendedHostStatuses.isHostHealthy(hostName)).collect(Collectors.toMap(StringType::value, extendedHostStatuses::statusReasonForHost));
    Set<String> noReportHosts = runningInstances.stream().map(InstanceMetaData::getDiscoveryFQDN).filter(Objects::nonNull).filter(discoveryFQDN -> !hostStatuses.containsKey(hostName(discoveryFQDN))).collect(toSet());
    if (!unhealthyHosts.isEmpty() || !noReportHosts.isEmpty()) {
        String conclusion = String.format("Unhealthy and/or unknown VMs found based on CM status. Unhealthy VMs: %s, unknown VMs: %s. " + "Please check the instances on your cloud provider for further details.", unhealthyHosts, noReportHosts);
        String details = String.format("Unhealthy and/or unknown VMs found based on CM status. Unhealthy VMs: %s, unknown VMs: %s", unhealthyHosts, noReportHosts);
        LOGGER.warn(details);
        return failed(conclusion, details);
    } else {
        return succeeded();
    }
}
Also used : ExtendedHostStatuses(com.sequenceiq.cloudbreak.cluster.status.ExtendedHostStatuses) Stack(com.sequenceiq.cloudbreak.domain.stack.Stack) HealthCheck(com.sequenceiq.cloudbreak.common.type.HealthCheck) ExtendedHostStatuses(com.sequenceiq.cloudbreak.cluster.status.ExtendedHostStatuses) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) ClusterApi(com.sequenceiq.cloudbreak.cluster.api.ClusterApi) Inject(javax.inject.Inject) InstanceMetaDataToCloudInstanceConverter(com.sequenceiq.cloudbreak.converter.spi.InstanceMetaDataToCloudInstanceConverter) Map(java.util.Map) InstanceMetaDataService(com.sequenceiq.cloudbreak.service.stack.InstanceMetaDataService) StackInstanceStatusChecker(com.sequenceiq.cloudbreak.service.stack.StackInstanceStatusChecker) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) Collectors.toSet(java.util.stream.Collectors.toSet) Logger(org.slf4j.Logger) RuntimeVersionService(com.sequenceiq.cloudbreak.service.stack.RuntimeVersionService) Set(java.util.Set) StringType(com.sequenceiq.cloudbreak.cloud.model.generic.StringType) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) HostName.hostName(com.sequenceiq.cloudbreak.cloud.model.HostName.hostName) List(java.util.List) Component(org.springframework.stereotype.Component) InstanceMetaData(com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData) InstanceSyncState(com.sequenceiq.cloudbreak.service.stack.flow.InstanceSyncState) HostName(com.sequenceiq.cloudbreak.cloud.model.HostName) ClusterApiConnectors(com.sequenceiq.cloudbreak.service.cluster.ClusterApiConnectors) StackService(com.sequenceiq.cloudbreak.service.stack.StackService) Collectors.toSet(java.util.stream.Collectors.toSet) Set(java.util.Set) Objects(java.util.Objects) HostName(com.sequenceiq.cloudbreak.cloud.model.HostName)

Aggregations

InstanceMetaData (com.sequenceiq.cloudbreak.domain.stack.instance.InstanceMetaData)415 Stack (com.sequenceiq.cloudbreak.domain.stack.Stack)165 InstanceGroup (com.sequenceiq.cloudbreak.domain.stack.instance.InstanceGroup)152 Test (org.junit.jupiter.api.Test)143 Map (java.util.Map)92 HashSet (java.util.HashSet)90 Set (java.util.Set)86 List (java.util.List)84 HostGroup (com.sequenceiq.cloudbreak.domain.stack.cluster.host.HostGroup)77 Cluster (com.sequenceiq.cloudbreak.domain.stack.cluster.Cluster)73 Collectors (java.util.stream.Collectors)71 ArrayList (java.util.ArrayList)62 Test (org.junit.Test)60 ArgumentMatchers.anyString (org.mockito.ArgumentMatchers.anyString)57 Optional (java.util.Optional)52 StackService (com.sequenceiq.cloudbreak.service.stack.StackService)48 Inject (javax.inject.Inject)47 Logger (org.slf4j.Logger)47 LoggerFactory (org.slf4j.LoggerFactory)47 ParameterizedTest (org.junit.jupiter.params.ParameterizedTest)45