Search in sources :

Example 16 with DescribeInstancesRequest

use of software.amazon.awssdk.services.ec2.model.DescribeInstancesRequest in project cloudbreak by hortonworks.

the class AwsDownscaleService method waitTermination.

private void waitTermination(List<String> instanceIds, Waiter<DescribeInstancesRequest> instanceTerminatedWaiter, StackCancellationCheck stackCancellationCheck) {
    DescribeInstancesRequest describeInstancesRequest = new DescribeInstancesRequest().withInstanceIds(instanceIds);
    run(instanceTerminatedWaiter, describeInstancesRequest, stackCancellationCheck, String.format("There was an error when application are deleting instances %s", String.join(",", instanceIds)));
}
Also used : DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest)

Example 17 with DescribeInstancesRequest

use of software.amazon.awssdk.services.ec2.model.DescribeInstancesRequest in project cloudbreak by hortonworks.

the class AwsDownscaleService method getExistingInstances.

private List<String> getExistingInstances(List<String> instanceIdsToDelete, AmazonEc2Client amazonEC2Client) {
    DescribeInstancesResult instancesResult = amazonEC2Client.describeInstances(new DescribeInstancesRequest().withInstanceIds(instanceIdsToDelete));
    List<String> existingInstances = instancesResult.getReservations().stream().flatMap(reservation -> reservation.getInstances().stream()).map(com.amazonaws.services.ec2.model.Instance::getInstanceId).collect(Collectors.toList());
    List<String> missingInstances = new ArrayList<>(instanceIdsToDelete);
    missingInstances.removeAll(existingInstances);
    LOGGER.debug("Known instances on AWS: {} Missing instances: {}", existingInstances, missingInstances);
    return existingInstances;
}
Also used : DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) ArrayList(java.util.ArrayList) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest)

Example 18 with DescribeInstancesRequest

use of software.amazon.awssdk.services.ec2.model.DescribeInstancesRequest in project cloudbreak by hortonworks.

the class AwsAutoScalingService method waitForGroup.

private void waitForGroup(AmazonEc2Client amClient, AmazonAutoScalingClient asClient, String autoScalingGroupName, Integer requiredInstanceCount, Long stackId, List<String> knownInstances) throws AmazonAutoscalingFailed {
    Waiter<DescribeAutoScalingGroupsRequest> groupInServiceWaiter = asClient.waiters().groupInService();
    PollingStrategy backoff = getBackoffCancellablePollingStrategy(new StackCancellationCheck(stackId));
    try {
        groupInServiceWaiter.run(new WaiterParameters<>(new DescribeAutoScalingGroupsRequest().withAutoScalingGroupNames(autoScalingGroupName)).withPollingStrategy(backoff));
    } catch (Exception e) {
        throw new AmazonAutoscalingFailed(e.getMessage(), e);
    }
    Waiter<DescribeAutoScalingGroupsRequest> instancesInServiceWaiter = customAmazonWaiterProvider.getAutoscalingInstancesInServiceWaiter(asClient, requiredInstanceCount);
    try {
        instancesInServiceWaiter.run(new WaiterParameters<>(new DescribeAutoScalingGroupsRequest().withAutoScalingGroupNames(autoScalingGroupName)).withPollingStrategy(backoff));
    } catch (WaiterTimedOutException e) {
        String message = String.format("Polling timed out. Not all the %s instance(s) reached InService state.", requiredInstanceCount);
        throw new AmazonAutoscalingFailed(message, e);
    } catch (Exception e) {
        throw new AmazonAutoscalingFailed(e.getMessage(), e);
    }
    List<String> instanceIds = cloudFormationStackUtil.getInstanceIds(asClient, autoScalingGroupName);
    List<String> instanceIdsForWait = getInstanceIdsForWait(instanceIds, knownInstances);
    if (requiredInstanceCount != 0) {
        List<List<String>> partitionedInstanceIdsList = Lists.partition(instanceIdsForWait, MAX_INSTANCE_ID_SIZE);
        Waiter<DescribeInstancesRequest> instanceRunningStateWaiter = amClient.waiters().instanceRunning();
        for (List<String> partitionedInstanceIds : partitionedInstanceIdsList) {
            try {
                instanceRunningStateWaiter.run(new WaiterParameters<>(new DescribeInstancesRequest().withInstanceIds(partitionedInstanceIds)).withPollingStrategy(backoff));
            } catch (AmazonServiceException e) {
                LOGGER.error("Cannot describeInstances", e);
                e.setErrorMessage("Cannot describeInstances. " + e.getErrorMessage());
                throw e;
            } catch (Exception e) {
                throw new AmazonAutoscalingFailed("Error occurred in describeInstances: " + e.getMessage(), e);
            }
        }
    }
}
Also used : WaiterTimedOutException(com.amazonaws.waiters.WaiterTimedOutException) StackCancellationCheck(com.sequenceiq.cloudbreak.cloud.aws.scheduler.StackCancellationCheck) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) AmazonServiceException(com.amazonaws.AmazonServiceException) WaiterTimedOutException(com.amazonaws.waiters.WaiterTimedOutException) WaiterParameters(com.amazonaws.waiters.WaiterParameters) DescribeAutoScalingGroupsRequest(com.amazonaws.services.autoscaling.model.DescribeAutoScalingGroupsRequest) BackoffCancellablePollingStrategy.getBackoffCancellablePollingStrategy(com.sequenceiq.cloudbreak.cloud.aws.scheduler.BackoffCancellablePollingStrategy.getBackoffCancellablePollingStrategy) PollingStrategy(com.amazonaws.waiters.PollingStrategy) AmazonServiceException(com.amazonaws.AmazonServiceException) List(java.util.List)

Example 19 with DescribeInstancesRequest

use of software.amazon.awssdk.services.ec2.model.DescribeInstancesRequest in project cloudbreak by hortonworks.

the class AwsNativeMetadataCollector method describeInstances.

private List<CloudVmMetaDataStatus> describeInstances(List<CloudInstance> vms, List<String> instanceIdList, Map<String, CloudResource> resourcesByInstanceId, AmazonEc2Client ec2Client) {
    LOGGER.info("Collecting metadata for instance IDs: '{}'", String.join(", ", instanceIdList));
    List<CloudVmMetaDataStatus> result = new ArrayList<>();
    DescribeInstancesRequest instancesRequest = new DescribeInstancesRequest().withInstanceIds(instanceIdList);
    DescribeInstancesResult instancesResult = ec2Client.describeInstances(instancesRequest);
    List<Instance> instances = instancesResult.getReservations().stream().flatMap(reservation -> reservation.getInstances().stream()).collect(Collectors.toList());
    for (Instance instance : instances) {
        String instanceId = instance.getInstanceId();
        CloudResource connectedResource = findCloudResourceByInstanceId(resourcesByInstanceId, instanceId);
        CloudInstance matchedInstance = findCloudInstanceByPrivateId(vms, connectedResource);
        CloudInstance updatedInstance = new CloudInstance(connectedResource.getInstanceId(), matchedInstance.getTemplate(), matchedInstance.getAuthentication(), matchedInstance.getSubnetId(), matchedInstance.getAvailabilityZone());
        CloudInstanceMetaData cloudInstanceMetaData = new CloudInstanceMetaData(instance.getPrivateIpAddress(), instance.getPublicIpAddress(), awsLifeCycleMapper.getLifeCycle(instance));
        CloudVmInstanceStatus cloudVmInstanceStatus = new CloudVmInstanceStatus(updatedInstance, updatedInstance.getTemplate().getStatus());
        CloudVmMetaDataStatus newMetadataStatus = new CloudVmMetaDataStatus(cloudVmInstanceStatus, cloudInstanceMetaData);
        result.add(newMetadataStatus);
    }
    return result;
}
Also used : EntitlementService(com.sequenceiq.cloudbreak.auth.altus.EntitlementService) AmazonElasticLoadBalancingClient(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonElasticLoadBalancingClient) LoggerFactory(org.slf4j.LoggerFactory) AwsLifeCycleMapper(com.sequenceiq.cloudbreak.cloud.aws.common.util.AwsLifeCycleMapper) DescribeLoadBalancersRequest(com.amazonaws.services.elasticloadbalancingv2.model.DescribeLoadBalancersRequest) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Function(java.util.function.Function) StringUtils(org.apache.commons.lang3.StringUtils) DescribeLoadBalancersResult(com.amazonaws.services.elasticloadbalancingv2.model.DescribeLoadBalancersResult) ArrayList(java.util.ArrayList) LoadBalancerType(com.sequenceiq.common.api.type.LoadBalancerType) LinkedHashMap(java.util.LinkedHashMap) Inject(javax.inject.Inject) Value(org.springframework.beans.factory.annotation.Value) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) INSTANCE_NOT_FOUND_ERROR_CODE(com.sequenceiq.cloudbreak.cloud.aws.common.AwsInstanceConnector.INSTANCE_NOT_FOUND_ERROR_CODE) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) Service(org.springframework.stereotype.Service) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) Map(java.util.Map) Instance(com.amazonaws.services.ec2.model.Instance) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) NotFoundException(com.sequenceiq.cloudbreak.common.exception.NotFoundException) LinkedHashSet(java.util.LinkedHashSet) InstanceStoreMetadata(com.sequenceiq.cloudbreak.cloud.model.InstanceStoreMetadata) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) AmazonServiceException(com.amazonaws.AmazonServiceException) CloudInstanceMetaData(com.sequenceiq.cloudbreak.cloud.model.CloudInstanceMetaData) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Set(java.util.Set) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) MetadataCollector(com.sequenceiq.cloudbreak.cloud.MetadataCollector) Collectors(java.util.stream.Collectors) List(java.util.List) AwsPlatformResources(com.sequenceiq.cloudbreak.cloud.aws.common.AwsPlatformResources) InstanceStatus(com.sequenceiq.cloudbreak.cloud.model.InstanceStatus) Optional(java.util.Optional) CloudLoadBalancerMetadata(com.sequenceiq.cloudbreak.cloud.model.CloudLoadBalancerMetadata) CommonAwsClient(com.sequenceiq.cloudbreak.cloud.aws.common.CommonAwsClient) CloudVmMetaDataStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus) LoadBalancerTypeConverter(com.sequenceiq.cloudbreak.cloud.aws.common.loadbalancer.LoadBalancerTypeConverter) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Instance(com.amazonaws.services.ec2.model.Instance) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) ArrayList(java.util.ArrayList) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) CloudInstanceMetaData(com.sequenceiq.cloudbreak.cloud.model.CloudInstanceMetaData) CloudVmMetaDataStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource)

Example 20 with DescribeInstancesRequest

use of software.amazon.awssdk.services.ec2.model.DescribeInstancesRequest in project cloudbreak by hortonworks.

the class EC2ClientActions method stopHostGroupInstances.

public void stopHostGroupInstances(List<String> instanceIds) {
    AmazonEC2 ec2Client = buildEC2Client();
    StopInstancesResult stopInstancesResult = ec2Client.stopInstances(new StopInstancesRequest().withInstanceIds(instanceIds));
    for (String instanceId : instanceIds) {
        try {
            Log.log(LOGGER, format(" EC2 instance [%s] state is [%s] ", instanceId, Objects.requireNonNull(stopInstancesResult.getStoppingInstances().stream().filter(instance -> instance.getInstanceId().equals(instanceId)).findAny().orElse(null)).getCurrentState().getName()));
            ec2Client.waiters().instanceStopped().run(new WaiterParameters<DescribeInstancesRequest>(new DescribeInstancesRequest().withInstanceIds(instanceId)).withPollingStrategy(new PollingStrategy(new MaxAttemptsRetryStrategy(80), new FixedDelayStrategy(30))));
            DescribeInstancesResult describeInstanceResult = ec2Client.describeInstances(new DescribeInstancesRequest().withInstanceIds(instanceId));
            InstanceState actualInstanceState = describeInstanceResult.getReservations().get(0).getInstances().get(0).getState();
            if (STOPPED_STATE.equals(actualInstanceState.getName())) {
                Log.log(LOGGER, format(" EC2 Instance: %s state is: %s ", instanceId, STOPPED_STATE));
            } else {
                LOGGER.error("EC2 Instance: {} stop has not been successful. So the actual state is: {} ", instanceId, actualInstanceState.getName());
                throw new TestFailException(" EC2 Instance: " + instanceId + " stop has not been successful, because of the actual state is: " + actualInstanceState.getName());
            }
        } catch (WaiterUnrecoverableException e) {
            LOGGER.error("EC2 Instance {} stop has not been successful, because of WaiterUnrecoverableException: {}", instanceId, e);
        } catch (WaiterTimedOutException e) {
            LOGGER.error("EC2 Instance {} stop has not been successful, because of WaiterTimedOutException: {}", instanceId, e);
        } catch (EC2UnexpectedException e) {
            LOGGER.error("EC2 Instance {} stop has not been successful, because of EC2UnexpectedException: {}", instanceId, e);
        }
    }
}
Also used : EC2UnexpectedException(com.amazonaws.services.lambda.model.EC2UnexpectedException) EbsInstanceBlockDevice(com.amazonaws.services.ec2.model.EbsInstanceBlockDevice) StopInstancesResult(com.amazonaws.services.ec2.model.StopInstancesResult) WaiterParameters(com.amazonaws.waiters.WaiterParameters) LoggerFactory(org.slf4j.LoggerFactory) TerminateInstancesResult(com.amazonaws.services.ec2.model.TerminateInstancesResult) BooleanUtils(org.apache.commons.lang3.BooleanUtils) StopInstancesRequest(com.amazonaws.services.ec2.model.StopInstancesRequest) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) EC2Client(com.sequenceiq.it.cloudbreak.util.aws.amazonec2.client.EC2Client) InstanceBlockDeviceMapping(com.amazonaws.services.ec2.model.InstanceBlockDeviceMapping) TestFailException(com.sequenceiq.it.cloudbreak.exception.TestFailException) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) Map(java.util.Map) AmazonEC2(com.amazonaws.services.ec2.AmazonEC2) DescribeVolumesRequest(com.amazonaws.services.ec2.model.DescribeVolumesRequest) Volume(com.amazonaws.services.ec2.model.Volume) InstanceState(com.amazonaws.services.ec2.model.InstanceState) Instance(com.amazonaws.services.ec2.model.Instance) WaiterUnrecoverableException(com.amazonaws.waiters.WaiterUnrecoverableException) SdxUtil(com.sequenceiq.it.cloudbreak.util.SdxUtil) Reservation(com.amazonaws.services.ec2.model.Reservation) FixedDelayStrategy(com.amazonaws.waiters.FixedDelayStrategy) Logger(org.slf4j.Logger) Collection(java.util.Collection) DescribeVolumesResult(com.amazonaws.services.ec2.model.DescribeVolumesResult) Log(com.sequenceiq.it.cloudbreak.log.Log) Set(java.util.Set) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) WaiterTimedOutException(com.amazonaws.waiters.WaiterTimedOutException) TerminateInstancesRequest(com.amazonaws.services.ec2.model.TerminateInstancesRequest) Collectors(java.util.stream.Collectors) String.format(java.lang.String.format) Objects(java.util.Objects) MaxAttemptsRetryStrategy(com.amazonaws.waiters.MaxAttemptsRetryStrategy) List(java.util.List) Component(org.springframework.stereotype.Component) PollingStrategy(com.amazonaws.waiters.PollingStrategy) Tag(com.amazonaws.services.ec2.model.Tag) EC2UnexpectedException(com.amazonaws.services.lambda.model.EC2UnexpectedException) WaiterTimedOutException(com.amazonaws.waiters.WaiterTimedOutException) MaxAttemptsRetryStrategy(com.amazonaws.waiters.MaxAttemptsRetryStrategy) WaiterUnrecoverableException(com.amazonaws.waiters.WaiterUnrecoverableException) AmazonEC2(com.amazonaws.services.ec2.AmazonEC2) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) WaiterParameters(com.amazonaws.waiters.WaiterParameters) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) InstanceState(com.amazonaws.services.ec2.model.InstanceState) PollingStrategy(com.amazonaws.waiters.PollingStrategy) StopInstancesResult(com.amazonaws.services.ec2.model.StopInstancesResult) TestFailException(com.sequenceiq.it.cloudbreak.exception.TestFailException) StopInstancesRequest(com.amazonaws.services.ec2.model.StopInstancesRequest) FixedDelayStrategy(com.amazonaws.waiters.FixedDelayStrategy)

Aggregations

DescribeInstancesRequest (com.amazonaws.services.ec2.model.DescribeInstancesRequest)52 DescribeInstancesResult (com.amazonaws.services.ec2.model.DescribeInstancesResult)35 Instance (com.amazonaws.services.ec2.model.Instance)30 Reservation (com.amazonaws.services.ec2.model.Reservation)29 ArrayList (java.util.ArrayList)26 List (java.util.List)23 Filter (com.amazonaws.services.ec2.model.Filter)18 Map (java.util.Map)15 Collectors (java.util.stream.Collectors)15 AmazonEC2 (com.amazonaws.services.ec2.AmazonEC2)12 Tag (com.amazonaws.services.ec2.model.Tag)11 Collection (java.util.Collection)11 Set (java.util.Set)11 Inject (javax.inject.Inject)11 DescribeVolumesResult (com.amazonaws.services.ec2.model.DescribeVolumesResult)10 TerminateInstancesRequest (com.amazonaws.services.ec2.model.TerminateInstancesRequest)10 TerminateInstancesResult (com.amazonaws.services.ec2.model.TerminateInstancesResult)10 Logger (org.slf4j.Logger)10 LoggerFactory (org.slf4j.LoggerFactory)10 StopInstancesRequest (com.amazonaws.services.ec2.model.StopInstancesRequest)9