Search in sources :

Example 1 with AmazonAutoScalingClient

use of com.amazonaws.services.autoscaling.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsClient method createAutoScalingClient.

public AmazonAutoScalingClient createAutoScalingClient(AwsCredentialView awsCredential, String regionName) {
    AmazonAutoScalingClient client = isRoleAssumeRequired(awsCredential) ? new AmazonAutoScalingClient(credentialClient.retrieveCachedSessionCredentials(awsCredential)) : new AmazonAutoScalingClient(createAwsCredentials(awsCredential));
    client.setRegion(RegionUtils.getRegion(regionName));
    return client;
}
Also used : AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient)

Example 2 with AmazonAutoScalingClient

use of com.amazonaws.services.autoscaling.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsResourceConnector method suspendAutoScaling.

private void suspendAutoScaling(AuthenticatedContext ac, CloudStack stack) {
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    for (Group group : stack.getGroups()) {
        String asGroupName = cfStackUtil.getAutoscalingGroupName(ac, group.getName(), ac.getCloudContext().getLocation().getRegion().value());
        amazonASClient.suspendProcesses(new SuspendProcessesRequest().withAutoScalingGroupName(asGroupName).withScalingProcesses(SUSPENDED_PROCESSES));
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) Group(com.sequenceiq.cloudbreak.cloud.model.Group) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) SuspendProcessesRequest(com.amazonaws.services.autoscaling.model.SuspendProcessesRequest)

Example 3 with AmazonAutoScalingClient

use of com.amazonaws.services.autoscaling.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsResourceConnector method resumeAutoScalingPolicies.

private void resumeAutoScalingPolicies(AuthenticatedContext ac, CloudStack stack) {
    for (Group instanceGroup : stack.getGroups()) {
        try {
            String asGroupName = cfStackUtil.getAutoscalingGroupName(ac, instanceGroup.getName(), ac.getCloudContext().getLocation().getRegion().value());
            if (asGroupName != null) {
                AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
                List<AutoScalingGroup> asGroups = amazonASClient.describeAutoScalingGroups(new DescribeAutoScalingGroupsRequest().withAutoScalingGroupNames(asGroupName)).getAutoScalingGroups();
                if (!asGroups.isEmpty()) {
                    if (!asGroups.get(0).getSuspendedProcesses().isEmpty()) {
                        amazonASClient.updateAutoScalingGroup(new UpdateAutoScalingGroupRequest().withAutoScalingGroupName(asGroupName).withMinSize(0).withDesiredCapacity(0));
                        amazonASClient.resumeProcesses(new ResumeProcessesRequest().withAutoScalingGroupName(asGroupName));
                    }
                }
            } else {
                LOGGER.info("Autoscaling Group's physical id is null (the resource doesn't exist), it is not needed to resume scaling policies.");
            }
        } catch (AmazonServiceException e) {
            if (e.getErrorMessage().matches("Resource.*does not exist for stack.*") || e.getErrorMessage().matches("Stack '.*' does not exist.*")) {
                LOGGER.info(e.getMessage());
            } else {
                throw e;
            }
        }
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) Group(com.sequenceiq.cloudbreak.cloud.model.Group) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) AmazonServiceException(com.amazonaws.AmazonServiceException) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest) ResumeProcessesRequest(com.amazonaws.services.autoscaling.model.ResumeProcessesRequest)

Example 4 with AmazonAutoScalingClient

use of com.amazonaws.services.autoscaling.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class AwsResourceConnector method upscale.

@Override
public List<CloudResourceStatus> upscale(AuthenticatedContext ac, CloudStack stack, List<CloudResource> resources) {
    resumeAutoScaling(ac, stack);
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    AmazonCloudFormationClient cloudFormationClient = awsClient.createCloudFormationClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    AmazonEC2Client amazonEC2Client = awsClient.createAccess(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    List<Group> scaledGroups = getScaledGroups(stack);
    for (Group group : scaledGroups) {
        String asGroupName = cfStackUtil.getAutoscalingGroupName(ac, cloudFormationClient, group.getName());
        amazonASClient.updateAutoScalingGroup(new UpdateAutoScalingGroupRequest().withAutoScalingGroupName(asGroupName).withMaxSize(group.getInstancesSize()).withDesiredCapacity(group.getInstancesSize()));
        LOGGER.info("Updated Auto Scaling group's desiredCapacity: [stack: '{}', to: '{}']", ac.getCloudContext().getId(), resources.size());
    }
    scheduleStatusChecks(stack, ac, cloudFormationClient);
    suspendAutoScaling(ac, stack);
    boolean mapPublicIpOnLaunch = isMapPublicOnLaunch(new AwsNetworkView(stack.getNetwork()), amazonEC2Client);
    List<Group> gateways = getGatewayGroups(scaledGroups);
    if (mapPublicIpOnLaunch && !gateways.isEmpty()) {
        String cFStackName = getCloudFormationStackResource(resources).getName();
        Map<String, String> eipAllocationIds = getElasticIpAllocationIds(cFStackName, cloudFormationClient);
        for (Group gateway : gateways) {
            List<String> eips = getEipsForGatewayGroup(eipAllocationIds, gateway);
            List<String> freeEips = getFreeIps(eips, amazonEC2Client);
            List<String> instanceIds = getInstancesForGroup(ac, amazonASClient, cloudFormationClient, gateway);
            List<String> newInstances = instanceIds.stream().filter(iid -> gateway.getInstances().stream().noneMatch(inst -> iid.equals(inst.getInstanceId()))).collect(Collectors.toList());
            associateElasticIpsToInstances(amazonEC2Client, freeEips, newInstances);
        }
    }
    return singletonList(new CloudResourceStatus(getCloudFormationStackResource(resources), ResourceStatus.UPDATED));
}
Also used : AmazonEC2Client(com.amazonaws.services.ec2.AmazonEC2Client) CreateStackRequest(com.amazonaws.services.cloudformation.model.CreateStackRequest) DescribeSubnetsRequest(com.amazonaws.services.ec2.model.DescribeSubnetsRequest) ReleaseAddressRequest(com.amazonaws.services.ec2.model.ReleaseAddressRequest) ROLLBACK_IN_PROGRESS(com.amazonaws.services.cloudformation.model.StackStatus.ROLLBACK_IN_PROGRESS) SuspendProcessesRequest(com.amazonaws.services.autoscaling.model.SuspendProcessesRequest) StringUtils(org.apache.commons.lang3.StringUtils) Collections.singletonList(java.util.Collections.singletonList) DescribeSubnetsResult(com.amazonaws.services.ec2.model.DescribeSubnetsResult) AmazonCloudFormationClient(com.amazonaws.services.cloudformation.AmazonCloudFormationClient) Arrays.asList(java.util.Arrays.asList) Map(java.util.Map) ROLLBACK_FAILED(com.amazonaws.services.cloudformation.model.StackStatus.ROLLBACK_FAILED) AdjustmentType(com.sequenceiq.cloudbreak.api.model.AdjustmentType) SubnetInfo(org.apache.commons.net.util.SubnetUtils.SubnetInfo) AmazonServiceException(com.amazonaws.AmazonServiceException) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) Retry(com.sequenceiq.cloudbreak.service.Retry) SyncPollingScheduler(com.sequenceiq.cloudbreak.cloud.scheduler.SyncPollingScheduler) AwsInstanceProfileView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsInstanceProfileView) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView) DELETE_FAILED(com.amazonaws.services.cloudformation.model.StackStatus.DELETE_FAILED) AwsPollTaskFactory(com.sequenceiq.cloudbreak.cloud.aws.task.AwsPollTaskFactory) DescribeStacksRequest(com.amazonaws.services.cloudformation.model.DescribeStacksRequest) StackStatus(com.amazonaws.services.cloudformation.model.StackStatus) DescribeKeyPairsRequest(com.amazonaws.services.ec2.model.DescribeKeyPairsRequest) InstanceGroupType(com.sequenceiq.cloudbreak.api.model.InstanceGroupType) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Value(org.springframework.beans.factory.annotation.Value) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) Lists(com.google.common.collect.Lists) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) Service(org.springframework.stereotype.Service) Image(com.amazonaws.services.ec2.model.Image) Filter(com.amazonaws.services.ec2.model.Filter) Network(com.sequenceiq.cloudbreak.cloud.model.Network) DeleteStackRequest(com.amazonaws.services.cloudformation.model.DeleteStackRequest) AmazonEC2Client(com.amazonaws.services.ec2.AmazonEC2Client) DELETE_COMPLETE(com.amazonaws.services.cloudformation.model.StackStatus.DELETE_COMPLETE) CREATE_FAILED(com.amazonaws.services.cloudformation.model.StackStatus.CREATE_FAILED) DetachInstancesRequest(com.amazonaws.services.autoscaling.model.DetachInstancesRequest) Parameter(com.amazonaws.services.cloudformation.model.Parameter) ROLLBACK_COMPLETE(com.amazonaws.services.cloudformation.model.StackStatus.ROLLBACK_COMPLETE) IOException(java.io.IOException) DescribeAddressesRequest(com.amazonaws.services.ec2.model.DescribeAddressesRequest) TerminateInstancesRequest(com.amazonaws.services.ec2.model.TerminateInstancesRequest) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) InstanceStatus(com.sequenceiq.cloudbreak.cloud.model.InstanceStatus) ImportKeyPairRequest(com.amazonaws.services.ec2.model.ImportKeyPairRequest) Configuration(freemarker.template.Configuration) InetAddresses(com.google.common.net.InetAddresses) SubnetUtils(org.apache.commons.net.util.SubnetUtils) DescribeVpcsRequest(com.amazonaws.services.ec2.model.DescribeVpcsRequest) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) OnFailure(com.amazonaws.services.cloudformation.model.OnFailure) TlsInfo(com.sequenceiq.cloudbreak.cloud.model.TlsInfo) AmazonCloudFormation(com.amazonaws.services.cloudformation.AmazonCloudFormation) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) DescribeImagesRequest(com.amazonaws.services.ec2.model.DescribeImagesRequest) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) CREATE_COMPLETE(com.amazonaws.services.cloudformation.model.StackStatus.CREATE_COMPLETE) Collection(java.util.Collection) Output(com.amazonaws.services.cloudformation.model.Output) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) Collectors(java.util.stream.Collectors) AwsNetworkView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsNetworkView) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) ModelContext(com.sequenceiq.cloudbreak.cloud.aws.CloudFormationTemplateBuilder.ModelContext) ActionWentFailException(com.sequenceiq.cloudbreak.service.Retry.ActionWentFailException) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Entry(java.util.Map.Entry) Optional(java.util.Optional) AmazonAutoScaling(com.amazonaws.services.autoscaling.AmazonAutoScaling) HashMap(java.util.HashMap) Inject(javax.inject.Inject) DeleteKeyPairRequest(com.amazonaws.services.ec2.model.DeleteKeyPairRequest) DisassociateAddressRequest(com.amazonaws.services.ec2.model.DisassociateAddressRequest) Charset(java.nio.charset.Charset) AmazonEC2(com.amazonaws.services.ec2.AmazonEC2) Qualifier(org.springframework.beans.factory.annotation.Qualifier) ResumeProcessesRequest(com.amazonaws.services.autoscaling.model.ResumeProcessesRequest) PollTask(com.sequenceiq.cloudbreak.cloud.task.PollTask) DescribeImagesResult(com.amazonaws.services.ec2.model.DescribeImagesResult) Subnet(com.amazonaws.services.ec2.model.Subnet) ResourceConnector(com.sequenceiq.cloudbreak.cloud.ResourceConnector) Logger(org.slf4j.Logger) ResourceType(com.sequenceiq.cloudbreak.common.type.ResourceType) Address(com.amazonaws.services.ec2.model.Address) Vpc(com.amazonaws.services.ec2.model.Vpc) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) AssociateAddressRequest(com.amazonaws.services.ec2.model.AssociateAddressRequest) DescribeAddressesResult(com.amazonaws.services.ec2.model.DescribeAddressesResult) Collections(java.util.Collections) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) Group(com.sequenceiq.cloudbreak.cloud.model.Group) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) AwsNetworkView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsNetworkView) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) UpdateAutoScalingGroupRequest(com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest) AmazonCloudFormationClient(com.amazonaws.services.cloudformation.AmazonCloudFormationClient)

Example 5 with AmazonAutoScalingClient

use of com.amazonaws.services.autoscaling.AmazonAutoScalingClient in project cloudbreak by hortonworks.

the class ASGroupStatusCheckerTaskTest method successTest.

@Test
public void successTest() throws Exception {
    int requiredInstances = 160;
    AuthenticatedContext authenticatedContext = mock(AuthenticatedContext.class);
    CloudContext cloudContext = mock(CloudContext.class);
    String regionName = "eu-west-1";
    when(cloudContext.getLocation()).thenReturn(Location.location(Region.region(regionName)));
    when(authenticatedContext.getCloudContext()).thenReturn(cloudContext);
    CloudCredential cloudCredential = mock(CloudCredential.class);
    when(authenticatedContext.getCloudCredential()).thenReturn(cloudCredential);
    String asGroupName = "as-group";
    AwsClient awsClient = mock(AwsClient.class);
    AmazonEC2Client amazonEC2Client = mock(AmazonEC2Client.class);
    when(awsClient.createAccess(any(AwsCredentialView.class), eq(regionName))).thenReturn(amazonEC2Client);
    DescribeInstanceStatusResult firstDescribeInstanceStatusResult = new DescribeInstanceStatusResult();
    List<InstanceStatus> firstInstanceStatuses = returnInstanceStatus(0, 100);
    firstDescribeInstanceStatusResult.setInstanceStatuses(firstInstanceStatuses);
    DescribeInstanceStatusResult secondDescribeInstanceStatusResult = new DescribeInstanceStatusResult();
    List<InstanceStatus> secondInstanceStatuses = returnInstanceStatus(100, 160);
    secondDescribeInstanceStatusResult.setInstanceStatuses(secondInstanceStatuses);
    when(amazonEC2Client.describeInstanceStatus(any(DescribeInstanceStatusRequest.class))).thenReturn(firstDescribeInstanceStatusResult).thenReturn(secondDescribeInstanceStatusResult);
    CloudFormationStackUtil cloudFormationStackUtil = mock(CloudFormationStackUtil.class);
    List<String> instancIds = new ArrayList<>();
    for (int i = 0; i < requiredInstances; i++) {
        instancIds.add(Integer.toString(i));
    }
    AmazonAutoScalingClient autoScalingClient = mock(AmazonAutoScalingClient.class);
    when(awsClient.createAutoScalingClient(any(AwsCredentialView.class), anyString())).thenReturn(autoScalingClient);
    when(autoScalingClient.describeScalingActivities(any(DescribeScalingActivitiesRequest.class))).thenReturn(new DescribeScalingActivitiesResult());
    when(cloudFormationStackUtil.getInstanceIds(any(AmazonAutoScalingClient.class), eq(asGroupName))).thenReturn(instancIds);
    ASGroupStatusCheckerTask asGroupStatusCheckerTask = new ASGroupStatusCheckerTask(authenticatedContext, asGroupName, requiredInstances, awsClient, cloudFormationStackUtil);
    Boolean taskResult = asGroupStatusCheckerTask.call();
    ArgumentCaptor<DescribeInstanceStatusRequest> instanceStatusRequestArgumentCaptor = ArgumentCaptor.forClass(DescribeInstanceStatusRequest.class);
    verify(amazonEC2Client, times(2)).describeInstanceStatus(instanceStatusRequestArgumentCaptor.capture());
    List<DescribeInstanceStatusRequest> allValues = instanceStatusRequestArgumentCaptor.getAllValues();
    assertEquals(100, allValues.get(0).getInstanceIds().size());
    assertEquals(60, allValues.get(1).getInstanceIds().size());
    assertTrue(taskResult);
}
Also used : AmazonEC2Client(com.amazonaws.services.ec2.AmazonEC2Client) CloudCredential(com.sequenceiq.cloudbreak.cloud.model.CloudCredential) DescribeInstanceStatusRequest(com.amazonaws.services.ec2.model.DescribeInstanceStatusRequest) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) ArrayList(java.util.ArrayList) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Matchers.anyString(org.mockito.Matchers.anyString) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView) AmazonAutoScalingClient(com.amazonaws.services.autoscaling.AmazonAutoScalingClient) DescribeInstanceStatusResult(com.amazonaws.services.ec2.model.DescribeInstanceStatusResult) DescribeScalingActivitiesResult(com.amazonaws.services.autoscaling.model.DescribeScalingActivitiesResult) InstanceStatus(com.amazonaws.services.ec2.model.InstanceStatus) AwsClient(com.sequenceiq.cloudbreak.cloud.aws.AwsClient) DescribeScalingActivitiesRequest(com.amazonaws.services.autoscaling.model.DescribeScalingActivitiesRequest) CloudFormationStackUtil(com.sequenceiq.cloudbreak.cloud.aws.CloudFormationStackUtil) Test(org.junit.Test)

Aggregations

AmazonAutoScalingClient (com.amazonaws.services.autoscaling.AmazonAutoScalingClient)18 AwsCredentialView (com.sequenceiq.cloudbreak.cloud.aws.view.AwsCredentialView)9 AmazonEC2Client (com.amazonaws.services.ec2.AmazonEC2Client)6 AmazonServiceException (com.amazonaws.AmazonServiceException)5 AutoScalingGroup (com.amazonaws.services.autoscaling.model.AutoScalingGroup)5 ArrayList (java.util.ArrayList)5 Group (com.sequenceiq.cloudbreak.cloud.model.Group)4 DescribeAutoScalingGroupsRequest (com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest)3 ResumeProcessesRequest (com.amazonaws.services.autoscaling.model.ResumeProcessesRequest)3 AmazonCloudFormationClient (com.amazonaws.services.cloudformation.AmazonCloudFormationClient)3 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)3 ClientConfiguration (com.amazonaws.ClientConfiguration)2 BasicAWSCredentials (com.amazonaws.auth.BasicAWSCredentials)2 AmazonAutoScaling (com.amazonaws.services.autoscaling.AmazonAutoScaling)2 DescribeScalingActivitiesRequest (com.amazonaws.services.autoscaling.model.DescribeScalingActivitiesRequest)2 DescribeScalingActivitiesResult (com.amazonaws.services.autoscaling.model.DescribeScalingActivitiesResult)2 DetachInstancesRequest (com.amazonaws.services.autoscaling.model.DetachInstancesRequest)2 SuspendProcessesRequest (com.amazonaws.services.autoscaling.model.SuspendProcessesRequest)2 UpdateAutoScalingGroupRequest (com.amazonaws.services.autoscaling.model.UpdateAutoScalingGroupRequest)2 DescribeStacksRequest (com.amazonaws.services.cloudformation.model.DescribeStacksRequest)2