Search in sources :

Example 1 with AmazonAutoScalingClient

use of com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsLaunchConfigurationImageUpdateService method updateImage.

public void updateImage(AuthenticatedContext authenticatedContext, CloudStack stack, CloudResource cfResource) {
    AwsCredentialView credentialView = new AwsCredentialView(authenticatedContext.getCloudCredential());
    String regionName = authenticatedContext.getCloudContext().getLocation().getRegion().getRegionName();
    AmazonCloudFormationClient cloudFormationClient = awsClient.createCloudFormationClient(credentialView, regionName);
    AmazonAutoScalingClient autoScalingClient = awsClient.createAutoScalingClient(credentialView, regionName);
    Map<AutoScalingGroup, String> scalingGroups = autoScalingGroupHandler.getAutoScalingGroups(cloudFormationClient, autoScalingClient, cfResource);
    List<LaunchConfiguration> oldLaunchConfigurations = launchConfigurationHandler.getLaunchConfigurations(autoScalingClient, scalingGroups.keySet());
    for (LaunchConfiguration oldLaunchConfiguration : oldLaunchConfigurations) {
        changeImageInAutoscalingGroup(authenticatedContext, stack, autoScalingClient, scalingGroups, oldLaunchConfiguration);
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) LaunchConfiguration(com.amazonaws.services.autoscaling.model.LaunchConfiguration) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)

Example 2 with AmazonAutoScalingClient

use of com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsLaunchTemplateUpdateService method updateFields.

public void updateFields(AuthenticatedContext authenticatedContext, String stackName, Map<LaunchTemplateField, String> updatableFields, boolean dryRun) {
    AwsCredentialView credentialView = new AwsCredentialView(authenticatedContext.getCloudCredential());
    String regionName = authenticatedContext.getCloudContext().getLocation().getRegion().getRegionName();
    AmazonCloudFormationClient cloudFormationClient = awsClient.createCloudFormationClient(credentialView, regionName);
    AmazonAutoScalingClient autoScalingClient = awsClient.createAutoScalingClient(credentialView, regionName);
    AmazonEc2Client ec2Client = awsClient.createEc2Client(credentialView, regionName);
    Map<AutoScalingGroup, String> autoScalingGroups = autoScalingGroupHandler.getAutoScalingGroups(cloudFormationClient, autoScalingClient, stackName);
    autoScalingGroups = filterGroupsForDryRun(autoScalingGroups, dryRun);
    LOGGER.debug("Modifying the {} fields for the [{}] autoscaling groups' launchtemplates [dryrun: {}]", updatableFields, autoScalingGroups.values(), dryRun);
    for (Map.Entry<AutoScalingGroup, String> asgEntry : autoScalingGroups.entrySet()) {
        AutoScalingGroup autoScalingGroup = asgEntry.getKey();
        LOGGER.debug("Creating new launchtemplate version for [{}] autoscale group...", autoScalingGroup.getAutoScalingGroupName());
        LaunchTemplateSpecification launchTemplateSpecification = getLaunchTemplateSpecification(autoScalingGroup);
        CreateLaunchTemplateVersionResult createLaunchTemplateVersionResult = getCreateLaunchTemplateVersionRequest(ec2Client, updatableFields, launchTemplateSpecification);
        modifyLaunchTemplate(ec2Client, launchTemplateSpecification, createLaunchTemplateVersionResult, !dryRun);
        if (dryRun) {
            LOGGER.debug("Autoscale group update will be skipped because of dryrun, which just test the permissions for modifiying the launch template.");
        } else {
            updateAutoScalingGroup(updatableFields, autoScalingClient, autoScalingGroup, launchTemplateSpecification, createLaunchTemplateVersionResult);
        }
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) LaunchTemplateSpecification(com.amazonaws.services.autoscaling.model.LaunchTemplateSpecification) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) CreateLaunchTemplateVersionResult(com.amazonaws.services.ec2.model.CreateLaunchTemplateVersionResult) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient) Map(java.util.Map)

Example 3 with AmazonAutoScalingClient

use of com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsMetadataCollector method collectCloudVmMetaDataStatuses.

private List<CloudVmMetaDataStatus> collectCloudVmMetaDataStatuses(AuthenticatedContext ac, List<CloudInstance> vms, List<CloudResource> resources, List<String> knownInstanceIdList) {
    LOGGER.debug("Collect Cloud VM metadata statuses");
    List<CloudVmMetaDataStatus> collectedCloudVmMetaDataStatuses = new ArrayList<>();
    String region = ac.getCloudContext().getLocation().getRegion().value();
    AmazonCloudFormationClient amazonCFClient = awsClient.createCloudFormationClient(new AwsCredentialView(ac.getCloudCredential()), region);
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), region);
    AmazonEc2Client amazonEC2Client = new AuthenticatedContextView(ac).getAmazonEC2Client();
    Multimap<String, CloudInstance> instanceGroupMap = getInstanceGroupMap(vms);
    Set<String> subnetIds = new HashSet<>();
    Multimap<String, Instance> instancesOnAWSForGroup = ArrayListMultimap.create();
    for (String group : instanceGroupMap.keySet()) {
        List<Instance> instancesForGroup = collectInstancesForGroup(ac, amazonASClient, amazonEC2Client, amazonCFClient, group);
        LOGGER.info("Collected instances for group: {}", instancesForGroup.stream().map(Instance::getInstanceId).collect(Collectors.joining(",")));
        instancesOnAWSForGroup.putAll(group, instancesForGroup);
        subnetIds.addAll(getSubnetIdsForInstances(instancesForGroup));
    }
    LOGGER.info("Collected subnet IDs: {}", subnetIds);
    Map<String, String> subnetIdToAvailabilityZoneMap = buildSubnetIdToAvailabilityZoneMap(subnetIds, amazonEC2Client);
    LOGGER.info("Subnet id to availability zone map: {}", subnetIdToAvailabilityZoneMap);
    Multimap<String, Instance> unknownInstancesForGroup = getUnknownInstancesForGroup(knownInstanceIdList, instancesOnAWSForGroup);
    for (CloudInstance vm : vms) {
        if (vm.getInstanceId() == null) {
            addFromUnknownMap(vm, resources, unknownInstancesForGroup, collectedCloudVmMetaDataStatuses, subnetIdToAvailabilityZoneMap);
        } else {
            addKnownInstance(vm, instancesOnAWSForGroup, collectedCloudVmMetaDataStatuses, subnetIdToAvailabilityZoneMap);
        }
    }
    LOGGER.info("Collected cloud VM metadata and statuses: {}", collectedCloudVmMetaDataStatuses);
    return collectedCloudVmMetaDataStatuses;
}
Also used : CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Instance(com.amazonaws.services.ec2.model.Instance) ArrayList(java.util.ArrayList) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) CloudVmMetaDataStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient) HashSet(java.util.HashSet)

Example 4 with AmazonAutoScalingClient

use of com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsLaunchService method launch.

public List<CloudResourceStatus> launch(AuthenticatedContext ac, CloudStack stack, PersistenceNotifier resourceNotifier, AdjustmentTypeWithThreshold adjustmentTypeWithThreshold) throws Exception {
    LOGGER.info("Launch AWS cluster with adjustment and threshold: {}", adjustmentTypeWithThreshold);
    createKeyPair(ac, stack);
    String cFStackName = cfStackUtil.getCfStackName(ac);
    AwsCredentialView credentialView = new AwsCredentialView(ac.getCloudCredential());
    String regionName = ac.getCloudContext().getLocation().getRegion().value();
    AmazonCloudFormationClient cfClient = awsClient.createCloudFormationClient(credentialView, regionName);
    AmazonEc2Client amazonEC2Client = awsClient.createEc2Client(credentialView, regionName);
    Network network = stack.getNetwork();
    AwsNetworkView awsNetworkView = new AwsNetworkView(network);
    boolean mapPublicIpOnLaunch = awsNetworkService.isMapPublicOnLaunch(awsNetworkView, amazonEC2Client);
    DescribeStacksRequest describeStacksRequest = new DescribeStacksRequest().withStackName(cFStackName);
    ModelContext modelContext = null;
    try {
        cfClient.describeStacks(describeStacksRequest);
        LOGGER.debug("Stack already exists: {}", cFStackName);
    } catch (AmazonServiceException ignored) {
        boolean existingVPC = awsNetworkView.isExistingVPC();
        boolean existingSubnet = awsNetworkView.isExistingSubnet();
        CloudResource cloudFormationStack = new Builder().type(ResourceType.CLOUDFORMATION_STACK).availabilityZone(ac.getCloudContext().getLocation().getAvailabilityZone().value()).name(cFStackName).build();
        resourceNotifier.notifyAllocation(cloudFormationStack, ac.getCloudContext());
        String cidr = network.getSubnet().getCidr();
        String subnet = isNoCIDRProvided(existingVPC, existingSubnet, cidr) ? awsNetworkService.findNonOverLappingCIDR(ac, stack) : cidr;
        modelContext = awsModelService.buildDefaultModelContext(ac, stack, resourceNotifier);
        String cfTemplate = cloudFormationTemplateBuilder.build(modelContext);
        LOGGER.debug("CloudFormationTemplate: {}", cfTemplate);
        cfClient.createStack(awsStackRequestHelper.createCreateStackRequest(ac, stack, cFStackName, subnet, cfTemplate));
    }
    LOGGER.debug("CloudFormation stack creation request sent with stack name: '{}' for stack: '{}'", cFStackName, ac.getCloudContext().getId());
    Waiter<DescribeStacksRequest> creationWaiter = cfClient.waiters().stackCreateComplete();
    StackCancellationCheck stackCancellationCheck = new StackCancellationCheck(ac.getCloudContext().getId());
    run(creationWaiter, describeStacksRequest, stackCancellationCheck, String.format("CloudFormation stack %s creation failed.", cFStackName), () -> awsCloudFormationErrorMessageProvider.getErrorReason(ac, cFStackName, ResourceStatus.CREATE_FAILED));
    List<CloudResource> networkResources = saveGeneratedSubnet(ac, stack, cFStackName, cfClient, resourceNotifier);
    suspendAutoscalingGoupsWhenNewInstancesAreReady(ac, stack);
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(credentialView, regionName);
    List<CloudResource> instances = cfStackUtil.getInstanceCloudResources(ac, cfClient, amazonASClient, stack.getGroups());
    if (mapPublicIpOnLaunch) {
        associatePublicIpsToGatewayInstances(stack, cFStackName, cfClient, amazonEC2Client, instances);
    }
    awsComputeResourceService.buildComputeResourcesForLaunch(ac, stack, adjustmentTypeWithThreshold, instances, networkResources);
    awsTaggingService.tagRootVolumes(ac, amazonEC2Client, instances, stack.getTags());
    awsCloudWatchService.addCloudWatchAlarmsForSystemFailures(instances, regionName, credentialView);
    return awsResourceConnector.check(ac, instances);
}
Also used : DescribeStacksRequest(com.amazonaws.services.cloudformation.model.DescribeStacksRequest) StackCancellationCheck(com.sequenceiq.cloudbreak.cloud.aws.scheduler.StackCancellationCheck) CloudFormationTemplateBuilder(com.sequenceiq.cloudbreak.cloud.aws.CloudFormationTemplateBuilder) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) ModelContext(com.sequenceiq.cloudbreak.cloud.aws.common.resource.ModelContext) AwsNetworkView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsNetworkView) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) Network(com.sequenceiq.cloudbreak.cloud.model.Network) AmazonServiceException(com.amazonaws.AmazonServiceException) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)

Example 5 with AmazonAutoScalingClient

use of com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsAutoScalingService method suspendAutoScaling.

public void suspendAutoScaling(AuthenticatedContext ac, CloudStack stack) {
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    for (Group group : stack.getGroups()) {
        String asGroupName = cfStackUtil.getAutoscalingGroupName(ac, group.getName(), ac.getCloudContext().getLocation().getRegion().value());
        LOGGER.info("Suspend autoscaling group '{}'", asGroupName);
        amazonASClient.suspendProcesses(new SuspendProcessesRequest().withAutoScalingGroupName(asGroupName).withScalingProcesses(SUSPENDED_PROCESSES));
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) Group(com.sequenceiq.cloudbreak.cloud.model.Group) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) SuspendProcessesRequest(com.amazonaws.services.autoscaling.model.SuspendProcessesRequest)

Aggregations

AmazonAutoScalingClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient)28 AutoScalingGroup (com.amazonaws.services.autoscaling.model.AutoScalingGroup)24 AmazonEc2Client (com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client)20 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)20 DescribeAutoScalingGroupsRequest (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest)19 AwsCredentialView (com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView)19 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)19 AmazonCloudFormationClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)18 CloudStack (com.sequenceiq.cloudbreak.cloud.model.CloudStack)18 Group (com.sequenceiq.cloudbreak.cloud.model.Group)15 ArrayList (java.util.ArrayList)15 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)14 DescribeAutoScalingGroupsResult (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsResult)13 InstanceAuthentication (com.sequenceiq.cloudbreak.cloud.model.InstanceAuthentication)12 List (java.util.List)12 Waiter (com.amazonaws.waiters.Waiter)11 CloudContext (com.sequenceiq.cloudbreak.cloud.context.CloudContext)10 CloudCredential (com.sequenceiq.cloudbreak.cloud.model.CloudCredential)10 Test (org.junit.jupiter.api.Test)10 AmazonServiceException (com.amazonaws.AmazonServiceException)9