Search in sources :

Example 6 with AwsContext

use of com.sequenceiq.cloudbreak.cloud.aws.common.context.AwsContext in project cloudbreak by hortonworks.

the class AwsEfsResourceBuilder method build.

@Override
public List<CloudResource> build(AwsContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, List<CloudResource> buildableResource, CloudStack cloudStack) throws Exception {
    LOGGER.debug("Create EFS on provider" + buildableResource.stream().map(CloudResource::getName).collect(Collectors.toList()));
    AmazonEfsClient client = getAmazonEfsClient(auth);
    Map<String, CloudEfsAttributes> efsSetMap = Collections.synchronizedMap(new HashMap<>());
    List<Future<?>> futures = new ArrayList<>();
    List<CloudResource> requestedResources = buildableResource.stream().filter(cloudResource -> CommonStatus.REQUESTED.equals(cloudResource.getStatus())).collect(Collectors.toList());
    LOGGER.debug("Start creating EFS for stack: '{}' group: '{}'", auth.getCloudContext().getName(), group.getName());
    futures.addAll(requestedResources.stream().map(requestedResource -> creatEfsRequest(requestedResource, cloudStack, efsSetMap)).map(request -> intermediateBuilderExecutor.submit(() -> {
        CreateFileSystemResult result = client.createFileSystem(request);
        CloudEfsAttributes resultAttributes = getResultEfsAttributes(request, result, efsSetMap);
        efsSetMap.put(resultAttributes.getName(), resultAttributes);
    })).collect(Collectors.toList()));
    LOGGER.debug("Waiting for EFS creation requests");
    for (Future<?> future : futures) {
        future.get();
    }
    LOGGER.debug("EFS creation requests sent");
    return buildableResource.stream().peek(resource -> {
        CloudEfsAttributes resultEfsAttributes = efsSetMap.getOrDefault(resource.getName(), null);
        if (resultEfsAttributes != null) {
            CloudEfsAttributes resourceEfsAttributes = resource.getParameter(CloudResource.ATTRIBUTES, CloudEfsAttributes.class);
            resourceEfsAttributes.setFileSystemId(resultEfsAttributes.getFileSystemId());
            resourceEfsAttributes.setFileState(resultEfsAttributes.getFileState());
        }
    }).map(copyResourceWithCreatedStatus()).collect(Collectors.toList());
}
Also used : AmazonEfsClient(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEfsClient) DescribeMountTargetsResult(com.amazonaws.services.elasticfilesystem.model.DescribeMountTargetsResult) AmazonElasticFileSystemException(com.amazonaws.services.elasticfilesystem.model.AmazonElasticFileSystemException) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) AmazonEfsClient(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEfsClient) Image(com.sequenceiq.cloudbreak.cloud.model.Image) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Future(java.util.concurrent.Future) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) AwsContext(com.sequenceiq.cloudbreak.cloud.aws.common.context.AwsContext) Map(java.util.Map) Qualifier(org.springframework.beans.factory.annotation.Qualifier) StreamSupport(java.util.stream.StreamSupport) DescribeMountTargetsRequest(com.amazonaws.services.elasticfilesystem.model.DescribeMountTargetsRequest) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CreateFileSystemRequest(com.amazonaws.services.elasticfilesystem.model.CreateFileSystemRequest) DescribeFileSystemsResult(com.amazonaws.services.elasticfilesystem.model.DescribeFileSystemsResult) FileSystemDescription(com.amazonaws.services.elasticfilesystem.model.FileSystemDescription) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Collection(java.util.Collection) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) DescribeFileSystemsRequest(com.amazonaws.services.elasticfilesystem.model.DescribeFileSystemsRequest) Tag(com.amazonaws.services.elasticfilesystem.model.Tag) Collectors(java.util.stream.Collectors) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) CreateFileSystemResult(com.amazonaws.services.elasticfilesystem.model.CreateFileSystemResult) List(java.util.List) Component(org.springframework.stereotype.Component) DeleteMountTargetRequest(com.amazonaws.services.elasticfilesystem.model.DeleteMountTargetRequest) LifeCycleState(com.sequenceiq.cloudbreak.cloud.model.filesystem.efs.LifeCycleState) MountTargetDescription(com.amazonaws.services.elasticfilesystem.model.MountTargetDescription) CloudEfsAttributes(com.sequenceiq.cloudbreak.cloud.model.CloudEfsAttributes) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) DeleteFileSystemRequest(com.amazonaws.services.elasticfilesystem.model.DeleteFileSystemRequest) CommonAwsClient(com.sequenceiq.cloudbreak.cloud.aws.common.CommonAwsClient) Collections(java.util.Collections) AwsTaggingService(com.sequenceiq.cloudbreak.cloud.aws.common.AwsTaggingService) StringUtils(org.springframework.util.StringUtils) CloudEfsAttributes(com.sequenceiq.cloudbreak.cloud.model.CloudEfsAttributes) ArrayList(java.util.ArrayList) CreateFileSystemResult(com.amazonaws.services.elasticfilesystem.model.CreateFileSystemResult) Future(java.util.concurrent.Future) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource)

Example 7 with AwsContext

use of com.sequenceiq.cloudbreak.cloud.aws.common.context.AwsContext in project cloudbreak by hortonworks.

the class AwsVolumeResourceBuilder method create.

@Override
public List<CloudResource> create(AwsContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, Image image) {
    LOGGER.debug("Create volume resources for {} in group: {}", instance.getInstanceId(), group.getName());
    InstanceTemplate template = group.getReferenceInstanceTemplate();
    if (CollectionUtils.isEmpty(template.getVolumes())) {
        LOGGER.debug("No volume requested in group: {}", group.getName());
        return List.of();
    }
    List<CloudResource> computeResources = context.getComputeResources(privateId);
    Optional<CloudResource> reattachableVolumeSet = computeResources.stream().filter(resource -> ResourceType.AWS_VOLUMESET.equals(resource.getType())).findFirst();
    if (reattachableVolumeSet.isEmpty()) {
        reattachableVolumeSet = fetchCloudResourceFromDBIfAvailable(privateId, auth, group, computeResources);
    } else {
        LOGGER.debug("Reattachable volumeset found: {}", reattachableVolumeSet.get());
    }
    String subnetId = getSubnetId(context, instance);
    Optional<String> availabilityZone = getAvailabilityZone(context, instance);
    return List.of(reattachableVolumeSet.orElseGet(createVolumeSet(privateId, auth, group, subnetId, availabilityZone)));
}
Also used : AwsResourceNameService(com.sequenceiq.cloudbreak.cloud.aws.common.service.AwsResourceNameService) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) DescribeSubnetsRequest(com.amazonaws.services.ec2.model.DescribeSubnetsRequest) CREATED(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus.CREATED) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) AwsDiskType(com.sequenceiq.common.model.AwsDiskType) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) TagSpecification(com.amazonaws.services.ec2.model.TagSpecification) StringUtils(org.apache.commons.lang3.StringUtils) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) DescribeSubnetsResult(com.amazonaws.services.ec2.model.DescribeSubnetsResult) Future(java.util.concurrent.Future) AwsCredentialView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) FAILED(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus.FAILED) CreateVolumeRequest(com.amazonaws.services.ec2.model.CreateVolumeRequest) Map(java.util.Map) EbsInstanceBlockDeviceSpecification(com.amazonaws.services.ec2.model.EbsInstanceBlockDeviceSpecification) Pair(org.springframework.data.util.Pair) NotFoundException(com.sequenceiq.cloudbreak.common.exception.NotFoundException) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) CloudVolumeUsageType(com.sequenceiq.cloudbreak.cloud.model.CloudVolumeUsageType) IN_PROGRESS(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus.IN_PROGRESS) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) DescribeVolumesResult(com.amazonaws.services.ec2.model.DescribeVolumesResult) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) BinaryOperator(java.util.function.BinaryOperator) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) CollectionUtils(org.springframework.util.CollectionUtils) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) AwsMethodExecutor(com.sequenceiq.cloudbreak.cloud.aws.common.util.AwsMethodExecutor) NotNull(org.jetbrains.annotations.NotNull) AwsInstanceView(com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsInstanceView) HashMap(java.util.HashMap) Image(com.sequenceiq.cloudbreak.cloud.model.Image) ResourceRetriever(com.sequenceiq.cloudbreak.cloud.service.ResourceRetriever) AtomicReference(java.util.concurrent.atomic.AtomicReference) Function(java.util.function.Function) Supplier(java.util.function.Supplier) InstanceBlockDeviceMappingSpecification(com.amazonaws.services.ec2.model.InstanceBlockDeviceMappingSpecification) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) CreateVolumeResult(com.amazonaws.services.ec2.model.CreateVolumeResult) AmazonEc2Client(com.sequenceiq.cloudbreak.cloud.aws.common.client.AmazonEc2Client) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) AwsContext(com.sequenceiq.cloudbreak.cloud.aws.common.context.AwsContext) Qualifier(org.springframework.beans.factory.annotation.Qualifier) DescribeVolumesRequest(com.amazonaws.services.ec2.model.DescribeVolumesRequest) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) Subnet(com.amazonaws.services.ec2.model.Subnet) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) ATTACHED(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus.ATTACHED) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) DELETED(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus.DELETED) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) ModifyInstanceAttributeResult(com.amazonaws.services.ec2.model.ModifyInstanceAttributeResult) Component(org.springframework.stereotype.Component) PreserveResourceException(com.sequenceiq.cloudbreak.cloud.template.compute.PreserveResourceException) ModifyInstanceAttributeRequest(com.amazonaws.services.ec2.model.ModifyInstanceAttributeRequest) DeleteVolumeRequest(com.amazonaws.services.ec2.model.DeleteVolumeRequest) AmazonEC2Exception(com.amazonaws.services.ec2.model.AmazonEC2Exception) CommonAwsClient(com.sequenceiq.cloudbreak.cloud.aws.common.CommonAwsClient) Collections(java.util.Collections) Volume(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes.Volume) AwsTaggingService(com.sequenceiq.cloudbreak.cloud.aws.common.AwsTaggingService) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate)

Aggregations

AwsContext (com.sequenceiq.cloudbreak.cloud.aws.common.context.AwsContext)7 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)7 CloudStack (com.sequenceiq.cloudbreak.cloud.model.CloudStack)7 CommonAwsClient (com.sequenceiq.cloudbreak.cloud.aws.common.CommonAwsClient)6 AwsCredentialView (com.sequenceiq.cloudbreak.cloud.aws.common.view.AwsCredentialView)6 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)6 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)6 CloudResourceStatus (com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus)6 Group (com.sequenceiq.cloudbreak.cloud.model.Group)6 Image (com.sequenceiq.cloudbreak.cloud.model.Image)6 CommonStatus (com.sequenceiq.common.api.type.CommonStatus)6 ResourceType (com.sequenceiq.common.api.type.ResourceType)6 List (java.util.List)6 Optional (java.util.Optional)6 Future (java.util.concurrent.Future)6 Function (java.util.function.Function)6 Collectors (java.util.stream.Collectors)6 Inject (javax.inject.Inject)6 Logger (org.slf4j.Logger)6 LoggerFactory (org.slf4j.LoggerFactory)6