Search in sources :

Example 21 with AwsCredentialView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView in project cloudbreak by hortonworks.

the class AwsAutoScalingService method suspendAutoScaling.

public void suspendAutoScaling(AuthenticatedContext ac, CloudStack stack) {
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), ac.getCloudContext().getLocation().getRegion().value());
    for (Group group : stack.getGroups()) {
        String asGroupName = cfStackUtil.getAutoscalingGroupName(ac, group.getName(), ac.getCloudContext().getLocation().getRegion().value());
        LOGGER.info("Suspend autoscaling group '{}'", asGroupName);
        amazonASClient.suspendProcesses(new SuspendProcessesRequest().withAutoScalingGroupName(asGroupName).withScalingProcesses(SUSPENDED_PROCESSES));
    }
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AutoScalingGroup(com.amazonaws.services.autoscaling.model.AutoScalingGroup) Group(com.sequenceiq.cloudbreak.cloud.model.Group) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) SuspendProcessesRequest(com.amazonaws.services.autoscaling.model.SuspendProcessesRequest)

Example 22 with AwsCredentialView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView in project cloudbreak by hortonworks.

the class AwsCloudWatchService method deleteCloudWatchAlarmsForSystemFailures.

private void deleteCloudWatchAlarmsForSystemFailures(String regionName, AwsCredentialView credentialView, List<String> instanceIds) {
    // The list of alarms received may be longer than 100 items, but alarms can only be deleted in batches of max
    // size 100. To work around this, break the instanceIds list into chunks no greater than 100, and process each
    // chunk. See https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_DeleteAlarms.html
    final AtomicInteger counter = new AtomicInteger(0);
    Map<Integer, List<String>> grouping = instanceIds.stream().map(instanceId -> instanceId + alarmSuffix).collect(Collectors.groupingBy(s -> counter.getAndIncrement() / maxBatchsize));
    LOGGER.debug("Batched cloudwatch alarm delete requests: {}", grouping);
    grouping.values().stream().flatMap(alarmNames -> getExistingCloudWatchAlarms(regionName, credentialView, alarmNames)).filter(alarmNames -> !alarmNames.isEmpty()).forEach(alarmNames -> deleteCloudWatchAlarms(regionName, credentialView, alarmNames));
}
Also used : AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Arrays(java.util.Arrays) INSTANCE_TYPE(com.sequenceiq.cloudbreak.cloud.model.CloudResource.INSTANCE_TYPE) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) AmazonCloudWatchClient(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonCloudWatchClient) Inject(javax.inject.Inject) Value(org.springframework.beans.factory.annotation.Value) Dimension(com.amazonaws.services.cloudwatch.model.Dimension) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Service(org.springframework.stereotype.Service) Map(java.util.Map) DeleteAlarmsRequest(com.amazonaws.services.cloudwatch.model.DeleteAlarmsRequest) Logger(org.slf4j.Logger) DescribeAlarmsRequest(com.amazonaws.services.cloudwatch.model.DescribeAlarmsRequest) MetricAlarm(com.amazonaws.services.cloudwatch.model.MetricAlarm) AwsCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.AwsCloudFormationClient) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Collectors(java.util.stream.Collectors) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) PutMetricAlarmRequest(com.amazonaws.services.cloudwatch.model.PutMetricAlarmRequest) Stream(java.util.stream.Stream) AmazonCloudWatchException(com.amazonaws.services.cloudwatch.model.AmazonCloudWatchException) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) List(java.util.List)

Example 23 with AwsCredentialView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView in project cloudbreak by hortonworks.

the class AwsTerminateService method terminate.

public List<CloudResourceStatus> terminate(AuthenticatedContext ac, CloudStack stack, List<CloudResource> resources) {
    LOGGER.debug("Deleting stack: {}", ac.getCloudContext().getId());
    AwsCredentialView credentialView = new AwsCredentialView(ac.getCloudCredential());
    AuthenticatedContextView authenticatedContextView = new AuthenticatedContextView(ac);
    String regionName = authenticatedContextView.getRegion();
    AmazonEc2Client amazonEC2Client = authenticatedContextView.getAmazonEC2Client();
    AmazonCloudFormationClient amazonCloudFormationClient = awsClient.createCloudFormationClient(credentialView, regionName);
    LOGGER.debug("Calling deleteCloudWatchAlarmsForSystemFailures from AwsTerminateService");
    awsCloudWatchService.deleteAllCloudWatchAlarmsForSystemFailures(stack, regionName, credentialView);
    waitAndDeleteCloudformationStack(ac, stack, resources, amazonCloudFormationClient);
    awsComputeResourceService.deleteComputeResources(ac, stack, resources);
    deleteKeyPair(ac, stack, amazonEC2Client, credentialView, regionName);
    deleteLaunchConfiguration(resources, ac);
    LOGGER.debug("Deleting stack finished");
    return awsResourceConnector.check(ac, resources);
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)

Example 24 with AwsCredentialView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView in project cloudbreak by hortonworks.

the class AwsNativeCloudWatchServiceTest method testDeleteCloudWatchAlarmsForSystemFailuresBatchesDeletion.

@Test
void testDeleteCloudWatchAlarmsForSystemFailuresBatchesDeletion() {
    List<CloudInstance> cloudInstances = new LinkedList<>();
    List<String> alarmNames1 = new LinkedList<>();
    List<String> alarmNames2 = new LinkedList<>();
    List<String> alarmNames3 = new LinkedList<>();
    List<MetricAlarm> alarms1 = new LinkedList<>();
    List<MetricAlarm> alarms2 = new LinkedList<>();
    List<MetricAlarm> alarms3 = new LinkedList<>();
    for (int i = 1; i <= 100; i++) {
        String alarmName = "i-" + i + "-Status-Check-Failed-System";
        alarmNames1.add(alarmName);
        MetricAlarm alarm = mock(MetricAlarm.class);
        when(alarm.getAlarmName()).thenReturn(alarmName);
        alarms1.add(alarm);
    }
    for (int i = 101; i <= 200; i++) {
        String alarmName = "i-" + i + "-Status-Check-Failed-System";
        alarmNames2.add(alarmName);
        MetricAlarm alarm = mock(MetricAlarm.class);
        when(alarm.getAlarmName()).thenReturn(alarmName);
        alarms2.add(alarm);
    }
    for (int i = 201; i <= 210; i++) {
        String alarmName = "i-" + i + "-Status-Check-Failed-System";
        alarmNames3.add(alarmName);
        MetricAlarm alarm = mock(MetricAlarm.class);
        when(alarm.getAlarmName()).thenReturn(alarmName);
        alarms3.add(alarm);
    }
    for (int i = 1; i <= 210; i++) {
        String instanceId = "i-" + i;
        CloudInstance cloudInstance = mock(CloudInstance.class);
        when(cloudInstance.getInstanceId()).thenReturn(instanceId);
        cloudInstances.add(cloudInstance);
    }
    DescribeAlarmsResult describeAlarmsResult1 = mock(DescribeAlarmsResult.class);
    DescribeAlarmsResult describeAlarmsResult2 = mock(DescribeAlarmsResult.class);
    DescribeAlarmsResult describeAlarmsResult3 = mock(DescribeAlarmsResult.class);
    when(describeAlarmsResult1.getMetricAlarms()).thenReturn(alarms1);
    when(describeAlarmsResult2.getMetricAlarms()).thenReturn(alarms2);
    when(describeAlarmsResult3.getMetricAlarms()).thenReturn(alarms3);
    AwsCredentialView credentialView = mock(AwsCredentialView.class);
    CloudStack stack = mock(CloudStack.class);
    Group group = mock(Group.class);
    List<Group> groups = List.of(group);
    when(stack.getGroups()).thenReturn(groups);
    when(group.getInstances()).thenReturn(cloudInstances);
    when(commonAwsClient.createCloudWatchClient(credentialView, REGION)).thenReturn(mockAmazonCloudWatchClient);
    when(mockAmazonCloudWatchClient.describeAlarms(any())).thenReturn(describeAlarmsResult1, describeAlarmsResult2, describeAlarmsResult3);
    underTest.deleteAllCloudWatchAlarmsForSystemFailures(stack, REGION, credentialView);
    ArgumentCaptor<DescribeAlarmsRequest> captorDescribe = ArgumentCaptor.forClass(DescribeAlarmsRequest.class);
    ArgumentCaptor<DeleteAlarmsRequest> captorDelete = ArgumentCaptor.forClass(DeleteAlarmsRequest.class);
    verify(mockAmazonCloudWatchClient, times(3)).describeAlarms(captorDescribe.capture());
    verify(mockAmazonCloudWatchClient, times(3)).deleteAlarms(captorDelete.capture());
    assertEquals(List.of(alarmNames1, alarmNames2, alarmNames3), captorDescribe.getAllValues().stream().map(DescribeAlarmsRequest::getAlarmNames).collect(Collectors.toList()));
    assertEquals(List.of(alarmNames1, alarmNames2, alarmNames3), captorDelete.getAllValues().stream().map(DeleteAlarmsRequest::getAlarmNames).collect(Collectors.toList()));
}
Also used : Group(com.sequenceiq.cloudbreak.cloud.model.Group) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) DescribeAlarmsRequest(com.amazonaws.services.cloudwatch.model.DescribeAlarmsRequest) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) LinkedList(java.util.LinkedList) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) MetricAlarm(com.amazonaws.services.cloudwatch.model.MetricAlarm) DescribeAlarmsResult(com.amazonaws.services.cloudwatch.model.DescribeAlarmsResult) DeleteAlarmsRequest(com.amazonaws.services.cloudwatch.model.DeleteAlarmsRequest) Test(org.junit.jupiter.api.Test)

Example 25 with AwsCredentialView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView in project cloudbreak by hortonworks.

the class AwsNativeCloudWatchServiceTest method testDeleteCloudWatchAlarmsForSystemFailuresWhenOneAlarmHasAlreadyBeenDeleted.

@Test
void testDeleteCloudWatchAlarmsForSystemFailuresWhenOneAlarmHasAlreadyBeenDeleted() {
    String alarm1Name = "i-1-Status-Check-Failed-System";
    String alarm2Name = "i-2-Status-Check-Failed-System";
    String alarm3Name = "i-3-Status-Check-Failed-System";
    String instanceId1 = "i-1";
    String instanceId2 = "i-2";
    String instanceId3 = "i-3";
    MetricAlarm alarm1 = mock(MetricAlarm.class);
    MetricAlarm alarm2 = mock(MetricAlarm.class);
    DescribeAlarmsResult describeAlarmsResult = mock(DescribeAlarmsResult.class);
    AwsCredentialView credentialView = mock(AwsCredentialView.class);
    CloudStack stack = mock(CloudStack.class);
    Group group = mock(Group.class);
    CloudInstance instance1 = mock(CloudInstance.class);
    CloudInstance instance2 = mock(CloudInstance.class);
    CloudInstance instance3 = mock(CloudInstance.class);
    List<Group> groups = List.of(group);
    when(stack.getGroups()).thenReturn(groups);
    when(group.getInstances()).thenReturn(List.of(instance1, instance2, instance3));
    when(instance1.getInstanceId()).thenReturn(instanceId1);
    when(instance2.getInstanceId()).thenReturn(instanceId2);
    when(instance3.getInstanceId()).thenReturn(instanceId3);
    when(commonAwsClient.createCloudWatchClient(eq(credentialView), eq(REGION))).thenReturn(mockAmazonCloudWatchClient);
    when(mockAmazonCloudWatchClient.describeAlarms(any())).thenReturn(describeAlarmsResult);
    // alarm 3 was already deleted
    when(describeAlarmsResult.getMetricAlarms()).thenReturn(List.of(alarm1, alarm2));
    when(alarm1.getAlarmName()).thenReturn(alarm1Name);
    when(alarm2.getAlarmName()).thenReturn(alarm2Name);
    underTest.deleteAllCloudWatchAlarmsForSystemFailures(stack, REGION, credentialView);
    ArgumentCaptor<DescribeAlarmsRequest> captorDescribe = ArgumentCaptor.forClass(DescribeAlarmsRequest.class);
    ArgumentCaptor<DeleteAlarmsRequest> captorDelete = ArgumentCaptor.forClass(DeleteAlarmsRequest.class);
    verify(mockAmazonCloudWatchClient, times(1)).describeAlarms(captorDescribe.capture());
    verify(mockAmazonCloudWatchClient, times(1)).deleteAlarms(captorDelete.capture());
    assertEquals(List.of(alarm1Name, alarm2Name, alarm3Name), captorDescribe.getValue().getAlarmNames());
    // only delete alarms that were not already deleted
    assertEquals(List.of(alarm1Name, alarm2Name), captorDelete.getValue().getAlarmNames());
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) MetricAlarm(com.amazonaws.services.cloudwatch.model.MetricAlarm) Group(com.sequenceiq.cloudbreak.cloud.model.Group) DescribeAlarmsResult(com.amazonaws.services.cloudwatch.model.DescribeAlarmsResult) DeleteAlarmsRequest(com.amazonaws.services.cloudwatch.model.DeleteAlarmsRequest) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) DescribeAlarmsRequest(com.amazonaws.services.cloudwatch.model.DescribeAlarmsRequest) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) Test(org.junit.jupiter.api.Test)

Aggregations

AwsCredentialView (com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView)94 AmazonEc2Client (com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client)32 CloudConnectorException (com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException)32 List (java.util.List)25 AmazonServiceException (com.amazonaws.AmazonServiceException)22 AmazonCloudFormationClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)21 Logger (org.slf4j.Logger)21 Inject (javax.inject.Inject)20 ArrayList (java.util.ArrayList)19 Collectors (java.util.stream.Collectors)19 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)18 Group (com.sequenceiq.cloudbreak.cloud.model.Group)18 Set (java.util.Set)18 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)17 CloudStack (com.sequenceiq.cloudbreak.cloud.model.CloudStack)17 Map (java.util.Map)16 LoggerFactory (org.slf4j.LoggerFactory)16 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)15 Service (org.springframework.stereotype.Service)15 AmazonAutoScalingClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient)14