Search in sources :

Example 1 with AuthenticatedContextView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView in project cloudbreak by hortonworks.

the class AwsSetup method prerequisites.

@Override
public void prerequisites(AuthenticatedContext ac, CloudStack stack, PersistenceNotifier persistenceNotifier) {
    AwsNetworkView awsNetworkView = new AwsNetworkView(stack.getNetwork());
    AwsCredentialView credentialView = new AwsCredentialView(ac.getCloudCredential());
    String region = ac.getCloudContext().getLocation().getRegion().value();
    verifySpotInstances(stack);
    if (awsNetworkView.isExistingVPC()) {
        try {
            AmazonEc2Client amazonEC2Client = new AuthenticatedContextView(ac).getAmazonEC2Client();
            validateExistingIGW(awsNetworkView, amazonEC2Client);
            validateExistingSubnet(awsNetworkView, amazonEC2Client);
        } catch (AmazonServiceException e) {
            throw new CloudConnectorException(e.getErrorMessage());
        } catch (AmazonClientException e) {
            throw new CloudConnectorException(e.getMessage());
        }
    }
    validateRegionAndZone(ac.getCloudCredential(), ac.getCloudContext().getLocation());
    validateExistingKeyPair(stack.getInstanceAuthentication(), credentialView, region, ac);
    LOGGER.debug("setup has been executed");
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AwsNetworkView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsNetworkView) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) AmazonClientException(com.amazonaws.AmazonClientException) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) AmazonServiceException(com.amazonaws.AmazonServiceException) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client)

Example 2 with AuthenticatedContextView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView in project cloudbreak by hortonworks.

the class AwsMetadataCollector method collectCloudVmMetaDataStatuses.

private List<CloudVmMetaDataStatus> collectCloudVmMetaDataStatuses(AuthenticatedContext ac, List<CloudInstance> vms, List<CloudResource> resources, List<String> knownInstanceIdList) {
    LOGGER.debug("Collect Cloud VM metadata statuses");
    List<CloudVmMetaDataStatus> collectedCloudVmMetaDataStatuses = new ArrayList<>();
    String region = ac.getCloudContext().getLocation().getRegion().value();
    AmazonCloudFormationClient amazonCFClient = awsClient.createCloudFormationClient(new AwsCredentialView(ac.getCloudCredential()), region);
    AmazonAutoScalingClient amazonASClient = awsClient.createAutoScalingClient(new AwsCredentialView(ac.getCloudCredential()), region);
    AmazonEc2Client amazonEC2Client = new AuthenticatedContextView(ac).getAmazonEC2Client();
    Multimap<String, CloudInstance> instanceGroupMap = getInstanceGroupMap(vms);
    Set<String> subnetIds = new HashSet<>();
    Multimap<String, Instance> instancesOnAWSForGroup = ArrayListMultimap.create();
    for (String group : instanceGroupMap.keySet()) {
        List<Instance> instancesForGroup = collectInstancesForGroup(ac, amazonASClient, amazonEC2Client, amazonCFClient, group);
        LOGGER.info("Collected instances for group: {}", instancesForGroup.stream().map(Instance::getInstanceId).collect(Collectors.joining(",")));
        instancesOnAWSForGroup.putAll(group, instancesForGroup);
        subnetIds.addAll(getSubnetIdsForInstances(instancesForGroup));
    }
    LOGGER.info("Collected subnet IDs: {}", subnetIds);
    Map<String, String> subnetIdToAvailabilityZoneMap = buildSubnetIdToAvailabilityZoneMap(subnetIds, amazonEC2Client);
    LOGGER.info("Subnet id to availability zone map: {}", subnetIdToAvailabilityZoneMap);
    Multimap<String, Instance> unknownInstancesForGroup = getUnknownInstancesForGroup(knownInstanceIdList, instancesOnAWSForGroup);
    for (CloudInstance vm : vms) {
        if (vm.getInstanceId() == null) {
            addFromUnknownMap(vm, resources, unknownInstancesForGroup, collectedCloudVmMetaDataStatuses, subnetIdToAvailabilityZoneMap);
        } else {
            addKnownInstance(vm, instancesOnAWSForGroup, collectedCloudVmMetaDataStatuses, subnetIdToAvailabilityZoneMap);
        }
    }
    LOGGER.info("Collected cloud VM metadata and statuses: {}", collectedCloudVmMetaDataStatuses);
    return collectedCloudVmMetaDataStatuses;
}
Also used : CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Instance(com.amazonaws.services.ec2.model.Instance) ArrayList(java.util.ArrayList) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AmazonAutoScalingClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient) CloudVmMetaDataStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmMetaDataStatus) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient) HashSet(java.util.HashSet)

Example 3 with AuthenticatedContextView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView in project cloudbreak by hortonworks.

the class AwsStackRequestHelper method getRootDeviceName.

private String getRootDeviceName(AuthenticatedContext ac, CloudStack cloudStack) {
    AmazonEc2Client ec2Client = new AuthenticatedContextView(ac).getAmazonEC2Client();
    DescribeImagesResult images = ec2Client.describeImages(new DescribeImagesRequest().withImageIds(cloudStack.getImage().getImageName()));
    if (images.getImages().isEmpty()) {
        throw new CloudConnectorException(String.format("AMI is not available: '%s'.", cloudStack.getImage().getImageName()));
    }
    Image image = images.getImages().get(0);
    if (image == null) {
        throw new CloudConnectorException(String.format("Couldn't describe AMI '%s'.", cloudStack.getImage().getImageName()));
    }
    return image.getRootDeviceName();
}
Also used : DescribeImagesResult(com.amazonaws.services.ec2.model.DescribeImagesResult) CloudConnectorException(com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException) DescribeImagesRequest(com.amazonaws.services.ec2.model.DescribeImagesRequest) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) Image(com.amazonaws.services.ec2.model.Image)

Example 4 with AuthenticatedContextView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView in project cloudbreak by hortonworks.

the class AwsTerminateService method terminate.

public List<CloudResourceStatus> terminate(AuthenticatedContext ac, CloudStack stack, List<CloudResource> resources) {
    LOGGER.debug("Deleting stack: {}", ac.getCloudContext().getId());
    AwsCredentialView credentialView = new AwsCredentialView(ac.getCloudCredential());
    AuthenticatedContextView authenticatedContextView = new AuthenticatedContextView(ac);
    String regionName = authenticatedContextView.getRegion();
    AmazonEc2Client amazonEC2Client = authenticatedContextView.getAmazonEC2Client();
    AmazonCloudFormationClient amazonCloudFormationClient = awsClient.createCloudFormationClient(credentialView, regionName);
    LOGGER.debug("Calling deleteCloudWatchAlarmsForSystemFailures from AwsTerminateService");
    awsCloudWatchService.deleteAllCloudWatchAlarmsForSystemFailures(stack, regionName, credentialView);
    waitAndDeleteCloudformationStack(ac, stack, resources, amazonCloudFormationClient);
    awsComputeResourceService.deleteComputeResources(ac, stack, resources);
    deleteKeyPair(ac, stack, amazonEC2Client, credentialView, regionName);
    deleteLaunchConfiguration(resources, ac);
    LOGGER.debug("Deleting stack finished");
    return awsResourceConnector.check(ac, resources);
}
Also used : AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) AmazonCloudFormationClient(com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)

Example 5 with AuthenticatedContextView

use of com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView in project cloudbreak by hortonworks.

the class AwsInstanceConnector method checkWithoutRetry.

@Override
public List<CloudVmInstanceStatus> checkWithoutRetry(AuthenticatedContext ac, List<CloudInstance> vms) {
    LOGGER.debug("Check instances on aws side: {}", vms);
    List<CloudInstance> cloudInstancesWithInstanceId = vms.stream().filter(cloudInstance -> cloudInstance.getInstanceId() != null).collect(Collectors.toList());
    LOGGER.debug("Instances with instanceId: {}", cloudInstancesWithInstanceId);
    List<String> instanceIds = cloudInstancesWithInstanceId.stream().map(CloudInstance::getInstanceId).collect(Collectors.toList());
    String region = ac.getCloudContext().getLocation().getRegion().value();
    try {
        DescribeInstancesResult result = new AuthenticatedContextView(ac).getAmazonEC2Client().describeInstances(new DescribeInstancesRequest().withInstanceIds(instanceIds));
        LOGGER.debug("Result from AWS: {}", result);
        return fillCloudVmInstanceStatuses(ac, cloudInstancesWithInstanceId, region, result);
    } catch (AmazonEC2Exception e) {
        handleEC2Exception(vms, e);
    } catch (SdkClientException e) {
        LOGGER.warn("Failed to send request to AWS: ", e);
        throw e;
    }
    return Collections.emptyList();
}
Also used : InstanceConnector(com.sequenceiq.cloudbreak.cloud.InstanceConnector) AwsInstanceStatusMapper(com.sequenceiq.cloudbreak.cloud.aws.common.util.AwsInstanceStatusMapper) PollerUtil(com.sequenceiq.cloudbreak.cloud.aws.common.poller.PollerUtil) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) StopInstancesRequest(com.amazonaws.services.ec2.model.StopInstancesRequest) CollectionUtils(org.apache.commons.collections4.CollectionUtils) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet) Inject(javax.inject.Inject) Value(org.springframework.beans.factory.annotation.Value) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) Matcher(java.util.regex.Matcher) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) StartInstancesRequest(com.amazonaws.services.ec2.model.StartInstancesRequest) Service(org.springframework.stereotype.Service) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) BiConsumer(java.util.function.BiConsumer) Retryable(org.springframework.retry.annotation.Retryable) Instance(com.amazonaws.services.ec2.model.Instance) CloudVmInstanceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudVmInstanceStatus) EnumSet(java.util.EnumSet) GetConsoleOutputRequest(com.amazonaws.services.ec2.model.GetConsoleOutputRequest) Reservation(com.amazonaws.services.ec2.model.Reservation) Logger(org.slf4j.Logger) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Collection(java.util.Collection) Set(java.util.Set) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) Collectors(java.util.stream.Collectors) Backoff(org.springframework.retry.annotation.Backoff) Sets(com.google.common.collect.Sets) List(java.util.List) SdkClientException(com.amazonaws.SdkClientException) GetConsoleOutputResult(com.amazonaws.services.ec2.model.GetConsoleOutputResult) InstanceStatus(com.sequenceiq.cloudbreak.cloud.model.InstanceStatus) AmazonEC2Exception(com.amazonaws.services.ec2.model.AmazonEC2Exception) Optional(java.util.Optional) CloudOperationNotSupportedException(com.sequenceiq.cloudbreak.cloud.exception.CloudOperationNotSupportedException) Pattern(java.util.regex.Pattern) Collections(java.util.Collections) DescribeInstancesResult(com.amazonaws.services.ec2.model.DescribeInstancesResult) SdkClientException(com.amazonaws.SdkClientException) AuthenticatedContextView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) DescribeInstancesRequest(com.amazonaws.services.ec2.model.DescribeInstancesRequest) AmazonEC2Exception(com.amazonaws.services.ec2.model.AmazonEC2Exception)

Aggregations

AmazonEc2Client (com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client)11 AuthenticatedContextView (com.sequenceiq.cloudbreak.cloud.aws.common.view.AuthenticatedContextView)11 AwsCredentialView (com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView)5 CloudConnectorException (com.sequenceiq.cloudbreak.cloud.exception.CloudConnectorException)5 AmazonServiceException (com.amazonaws.AmazonServiceException)3 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)3 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)3 SdkClientException (com.amazonaws.SdkClientException)2 AmazonEC2Exception (com.amazonaws.services.ec2.model.AmazonEC2Exception)2 DescribeImagesRequest (com.amazonaws.services.ec2.model.DescribeImagesRequest)2 DescribeImagesResult (com.amazonaws.services.ec2.model.DescribeImagesResult)2 DescribeInstancesRequest (com.amazonaws.services.ec2.model.DescribeInstancesRequest)2 DescribeInstancesResult (com.amazonaws.services.ec2.model.DescribeInstancesResult)2 GetConsoleOutputRequest (com.amazonaws.services.ec2.model.GetConsoleOutputRequest)2 GetConsoleOutputResult (com.amazonaws.services.ec2.model.GetConsoleOutputResult)2 Instance (com.amazonaws.services.ec2.model.Instance)2 AmazonAutoScalingClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonAutoScalingClient)2 AmazonCloudFormationClient (com.sequenceiq.cloudbreak.cloud.aws.client.AmazonCloudFormationClient)2 AmazonElasticLoadBalancingClient (com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonElasticLoadBalancingClient)2 CloudOperationNotSupportedException (com.sequenceiq.cloudbreak.cloud.exception.CloudOperationNotSupportedException)2