Search in sources :

Example 16 with OperationContext

use of com.vmware.xenon.common.OperationContext in project photon-model by vmware.

the class AWSS3StorageEnumerationAdapterService method enumerateS3Buckets.

/**
 * Call the listBuckets() method to enumerate S3 buckets.
 * AWS SDK does not have an async method for listing buckets, so we use the synchronous method
 * in a fixed thread pool for S3 enumeration service.
 * If listBuckets() call fails due to unsupported region, we mark the S3 client invalid,
 * stop the enumeration flow and patch back to parent.
 */
private void enumerateS3Buckets(S3StorageEnumerationContext aws) {
    logInfo(() -> String.format("Running creation enumeration in refresh mode for %s", aws.request.original.resourceReference));
    OperationContext operationContext = OperationContext.getOperationContext();
    this.executorService.submit(() -> {
        try {
            List<Bucket> bucketList = aws.amazonS3Client.listBuckets();
            for (Bucket bucket : bucketList) {
                aws.remoteBucketsByBucketName.put(bucket.getName(), bucket);
            }
            OperationContext.restoreOperationContext(operationContext);
            if (aws.remoteBucketsByBucketName.isEmpty()) {
                aws.subStage = S3StorageEnumerationSubStage.DELETE_DISKS;
            }
            handleReceivedEnumerationData(aws);
        } catch (Exception e) {
            if (e instanceof AmazonS3Exception && ((AmazonS3Exception) e).getStatusCode() == Operation.STATUS_CODE_FORBIDDEN) {
                markClientInvalid(aws);
            } else {
                logSevere("Exception enumerating S3 buckets for [ex=%s]", e.getMessage());
                aws.error = e;
                aws.stage = S3StorageEnumerationStages.ERROR;
                handleEnumerationRequest(aws);
            }
        }
    });
}
Also used : OperationContext(com.vmware.xenon.common.OperationContext) Bucket(com.amazonaws.services.s3.model.Bucket) AmazonS3Exception(com.amazonaws.services.s3.model.AmazonS3Exception) AmazonS3Exception(com.amazonaws.services.s3.model.AmazonS3Exception)

Example 17 with OperationContext

use of com.vmware.xenon.common.OperationContext in project photon-model by vmware.

the class AzureInstanceService method deleteVHDBlobsInAzure.

private DeferredResult<Void> deleteVHDBlobsInAzure(AzureInstanceContext ctx) {
    if (isManagedDisk().test(ctx.bootDiskState)) {
        return DeferredResult.completed(null);
    }
    String defaultSAName = ctx.bootDiskState.storageDescription.name;
    String defaultSAKey = ctx.storageAccountKeysForDisks.get(ctx.bootDiskState.storageDescription.authCredentialsLink);
    List<DiskService.DiskStateExpanded> diskStatesToDelete = ctx.dataDiskStates.stream().filter(disk -> disk.persistent == false).collect(Collectors.toList());
    if (ctx.bootDiskState.persistent == false) {
        diskStatesToDelete.add(ctx.bootDiskState);
    }
    OperationContext opCtx = OperationContext.getOperationContext();
    // Create DR
    DeferredResult<Void> dr = new DeferredResult<>();
    CompletableFuture.runAsync(() -> {
        diskStatesToDelete.forEach(disk -> {
            try {
                CloudStorageAccount storageAccountOfVHDBlob;
                if (disk.storageDescription == null) {
                    storageAccountOfVHDBlob = CloudStorageAccount.parse(String.format(STORAGE_CONNECTION_STRING, defaultSAName, defaultSAKey));
                } else {
                    String key = ctx.storageAccountKeysForDisks.get(disk.storageDescription.authCredentialsLink);
                    storageAccountOfVHDBlob = CloudStorageAccount.parse(String.format(STORAGE_CONNECTION_STRING, disk.storageDescription.name, key));
                }
                CloudBlobClient client = storageAccountOfVHDBlob.createCloudBlobClient();
                CloudBlobContainer container = client.getContainerReference("vhds");
                String vhdBlobName = disk.id.substring(disk.id.lastIndexOf("/") + 1);
                CloudPageBlob blob = container.getPageBlobReference(vhdBlobName);
                blob.deleteIfExists();
            } catch (URISyntaxException | InvalidKeyException | StorageException e) {
                logSevere("Unable to delete VHD file for disk [%s] because of %s ", disk.name, e.toString());
            }
        });
    }, getHost().allocateExecutor(this)).whenComplete((obj, throwable) -> {
        OperationContext.restoreOperationContext(opCtx);
        dr.complete(obj);
    });
    return dr;
}
Also used : VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) NetworkManagementClientImpl(com.microsoft.azure.management.network.implementation.NetworkManagementClientImpl) PROVISIONING_STATE_SUCCEEDED(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.PROVISIONING_STATE_SUCCEEDED) ComputeManager(com.microsoft.azure.management.compute.implementation.ComputeManager) IpAssignment(com.vmware.photon.controller.model.resources.NetworkInterfaceDescriptionService.IpAssignment) VirtualNetworkInner(com.microsoft.azure.management.network.implementation.VirtualNetworkInner) LifecycleState(com.vmware.photon.controller.model.resources.ComputeService.LifecycleState) COMPUTE_NAMESPACE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.COMPUTE_NAMESPACE) DISK_CONTROLLER_NUMBER(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.DISK_CONTROLLER_NUMBER) StorageManagementClientImpl(com.microsoft.azure.management.storage.implementation.StorageManagementClientImpl) AzureDiagnosticSettings(com.vmware.photon.controller.model.adapters.azure.model.diagnostics.AzureDiagnosticSettings) INVALID_RESOURCE_GROUP(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.INVALID_RESOURCE_GROUP) Utils(com.vmware.xenon.common.Utils) Map(java.util.Map) StorageDescription(com.vmware.photon.controller.model.resources.StorageDescriptionService.StorageDescription) ProvisioningState(com.microsoft.azure.management.storage.ProvisioningState) OSDisk(com.microsoft.azure.management.compute.OSDisk) AZURE_STORAGE_ACCOUNT_KEY1(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_ACCOUNT_KEY1) ProviderInner(com.microsoft.azure.management.resources.implementation.ProviderInner) CachingTypes(com.microsoft.azure.management.compute.CachingTypes) Indexable(com.microsoft.azure.management.resources.fluentcore.model.Indexable) StorageAccountListKeysResultInner(com.microsoft.azure.management.storage.implementation.StorageAccountListKeysResultInner) NetworkInterfaceState(com.vmware.photon.controller.model.resources.NetworkInterfaceService.NetworkInterfaceState) StatelessService(com.vmware.xenon.common.StatelessService) AZURE_STORAGE_ACCOUNT_DEFAULT_RG_NAME(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_ACCOUNT_DEFAULT_RG_NAME) Disk(com.microsoft.azure.management.compute.Disk) AzureProvisioningCallback(com.vmware.photon.controller.model.adapters.azure.utils.AzureProvisioningCallback) CompletionStage(java.util.concurrent.CompletionStage) AzureSecurityGroupUtils(com.vmware.photon.controller.model.adapters.azure.utils.AzureSecurityGroupUtils) StorageAccountTypes(com.microsoft.azure.management.compute.StorageAccountTypes) SkuInner(com.microsoft.azure.management.storage.implementation.SkuInner) StorageDescriptionService(com.vmware.photon.controller.model.resources.StorageDescriptionService) DeferredResult(com.vmware.xenon.common.DeferredResult) UriUtils(com.vmware.xenon.common.UriUtils) STATUS_CODE_UNAUTHORIZED(com.vmware.xenon.common.Operation.STATUS_CODE_UNAUTHORIZED) InvalidKeyException(java.security.InvalidKeyException) StorageAccountCreateParametersInner(com.microsoft.azure.management.storage.implementation.StorageAccountCreateParametersInner) DiskCreateOptionTypes(com.microsoft.azure.management.compute.DiskCreateOptionTypes) ResourceGroupsInner(com.microsoft.azure.management.resources.implementation.ResourceGroupsInner) AzureProvisioningCallbackWithRetry(com.vmware.photon.controller.model.adapters.azure.utils.AzureProvisioningCallbackWithRetry) AZURE_DATA_DISK_CACHING(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_DATA_DISK_CACHING) INVALID_PARAMETER(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.INVALID_PARAMETER) STORAGE_ACCOUNT_ALREADY_EXIST(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.STORAGE_ACCOUNT_ALREADY_EXIST) VirtualNetworksInner(com.microsoft.azure.management.network.implementation.VirtualNetworksInner) ImageReferenceInner(com.microsoft.azure.management.compute.implementation.ImageReferenceInner) OperationContext(com.vmware.xenon.common.OperationContext) PROVISIONING_STATE_FAILED_NO_SUBNET(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.PROVISIONING_STATE_FAILED_NO_SUBNET) ComputeDescriptionService(com.vmware.photon.controller.model.resources.ComputeDescriptionService) ArrayList(java.util.ArrayList) OSProfile(com.microsoft.azure.management.compute.OSProfile) ComputeInstanceRequest(com.vmware.photon.controller.model.adapterapi.ComputeInstanceRequest) SecurityGroupState(com.vmware.photon.controller.model.resources.SecurityGroupService.SecurityGroupState) StorageException(com.microsoft.azure.storage.StorageException) AzureAsyncCallback(com.vmware.photon.controller.model.adapters.azure.AzureAsyncCallback) AzureImageSource(com.vmware.photon.controller.model.adapters.azure.instance.AzureInstanceContext.AzureImageSource) MISSING_SUBSCRIPTION_CODE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.MISSING_SUBSCRIPTION_CODE) StorageProfile(com.microsoft.azure.management.compute.StorageProfile) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) BiConsumer(java.util.function.BiConsumer) CloudPageBlob(com.microsoft.azure.storage.blob.CloudPageBlob) OperationStatusResponseInner(com.microsoft.azure.management.compute.implementation.OperationStatusResponseInner) VirtualMachineInner(com.microsoft.azure.management.compute.implementation.VirtualMachineInner) PublicIPAddressInner(com.microsoft.azure.management.network.implementation.PublicIPAddressInner) AdapterUtils(com.vmware.photon.controller.model.adapters.util.AdapterUtils) ServiceErrorResponse(com.vmware.xenon.common.ServiceErrorResponse) DataDisk(com.microsoft.azure.management.compute.DataDisk) ServiceCallback(com.microsoft.rest.ServiceCallback) STORAGE_NAMESPACE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.STORAGE_NAMESPACE) SubnetInner(com.microsoft.azure.management.network.implementation.SubnetInner) ImageSource(com.vmware.photon.controller.model.adapters.util.instance.BaseComputeInstanceContext.ImageSource) File(java.io.File) ManagedDiskParametersInner(com.microsoft.azure.management.compute.implementation.ManagedDiskParametersInner) NETWORK_NAMESPACE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.NETWORK_NAMESPACE) AzureUtils.getStorageAccountKeyName(com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils.getStorageAccountKeyName) ApplicationTokenCredentials(com.microsoft.azure.credentials.ApplicationTokenCredentials) PhotonModelUriUtils.createInventoryUri(com.vmware.photon.controller.model.util.PhotonModelUriUtils.createInventoryUri) AuthCredentialsServiceState(com.vmware.xenon.services.common.AuthCredentialsService.AuthCredentialsServiceState) IPAllocationMethod(com.microsoft.azure.management.network.IPAllocationMethod) CloudError(com.microsoft.azure.CloudError) HardwareProfile(com.microsoft.azure.management.compute.HardwareProfile) AzureNicContext(com.vmware.photon.controller.model.adapters.azure.instance.AzureInstanceContext.AzureNicContext) URISyntaxException(java.net.URISyntaxException) NetworkProfile(com.microsoft.azure.management.compute.NetworkProfile) VirtualMachineImageResourceInner(com.microsoft.azure.management.compute.implementation.VirtualMachineImageResourceInner) AzureUriPaths(com.vmware.photon.controller.model.adapters.azure.AzureUriPaths) SubscriptionInner(com.microsoft.azure.management.resources.implementation.SubscriptionInner) AZURE_MANAGED_DISK_TYPE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_MANAGED_DISK_TYPE) ComputeType(com.vmware.photon.controller.model.resources.ComputeDescriptionService.ComputeDescription.ComputeType) Creatable(com.microsoft.azure.management.resources.fluentcore.model.Creatable) AzureSdkClients(com.vmware.photon.controller.model.adapters.azure.utils.AzureSdkClients) ResourceManagementClientImpl(com.microsoft.azure.management.resources.implementation.ResourceManagementClientImpl) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) URI(java.net.URI) AzureConstants(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants) AvailabilitySet(com.microsoft.azure.management.compute.AvailabilitySet) ComputeDescription(com.vmware.photon.controller.model.resources.ComputeDescriptionService.ComputeDescription) SkuName(com.microsoft.azure.management.storage.SkuName) AddressSpace(com.microsoft.azure.management.network.AddressSpace) COMPUTER_NAME(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.COMPUTER_NAME) Predicate(java.util.function.Predicate) Collection(java.util.Collection) DiskConfiguration(com.vmware.photon.controller.model.resources.ImageService.ImageState.DiskConfiguration) AvailabilitySetInner(com.microsoft.azure.management.compute.implementation.AvailabilitySetInner) DiskState(com.vmware.photon.controller.model.resources.DiskService.DiskState) CloudStorageAccount(com.microsoft.azure.storage.CloudStorageAccount) InstanceRequestType(com.vmware.photon.controller.model.adapterapi.ComputeInstanceRequest.InstanceRequestType) Collectors(java.util.stream.Collectors) Base64(java.util.Base64) List(java.util.List) VirtualHardDisk(com.microsoft.azure.management.compute.VirtualHardDisk) AzureUtils(com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils) NetworkSecurityGroupsInner(com.microsoft.azure.management.network.implementation.NetworkSecurityGroupsInner) NetworkInterfacesInner(com.microsoft.azure.management.network.implementation.NetworkInterfacesInner) Optional(java.util.Optional) NetworkSecurityGroupInner(com.microsoft.azure.management.network.implementation.NetworkSecurityGroupInner) DiskService(com.vmware.photon.controller.model.resources.DiskService) UnsupportedEncodingException(java.io.UnsupportedEncodingException) Default(com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback.Default) SubResource(com.microsoft.azure.SubResource) AZURE_OSDISK_CACHING(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_OSDISK_CACHING) ResourceGroupInner(com.microsoft.azure.management.resources.implementation.ResourceGroupInner) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) Region(com.microsoft.azure.management.resources.fluentcore.arm.Region) HashMap(java.util.HashMap) CompletableFuture(java.util.concurrent.CompletableFuture) AzureDeferredResultServiceCallbackWithRetry(com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallbackWithRetry) VirtualMachineSizeTypes(com.microsoft.azure.management.compute.VirtualMachineSizeTypes) SubscriptionClientImpl(com.microsoft.azure.management.resources.implementation.SubscriptionClientImpl) Kind(com.microsoft.azure.management.storage.Kind) Level(java.util.logging.Level) FileUtils(com.vmware.xenon.common.FileUtils) AZURE_STORAGE_ACCOUNT_RG_NAME(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_ACCOUNT_RG_NAME) AuthCredentialsService(com.vmware.xenon.services.common.AuthCredentialsService) ComputeState(com.vmware.photon.controller.model.resources.ComputeService.ComputeState) PublicIPAddressesInner(com.microsoft.azure.management.network.implementation.PublicIPAddressesInner) STORAGE_CONNECTION_STRING(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.STORAGE_CONNECTION_STRING) AzureDecommissionCallback(com.vmware.photon.controller.model.adapters.azure.utils.AzureDecommissionCallback) ComputeManagementClientImpl(com.microsoft.azure.management.compute.implementation.ComputeManagementClientImpl) NetworkInterfaceIPConfigurationInner(com.microsoft.azure.management.network.implementation.NetworkInterfaceIPConfigurationInner) EncryptionUtils(com.vmware.photon.controller.model.security.util.EncryptionUtils) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) Operation(com.vmware.xenon.common.Operation) AvailabilitySetSkuTypes(com.microsoft.azure.management.compute.AvailabilitySetSkuTypes) StorageAccountKey(com.microsoft.azure.management.storage.StorageAccountKey) BaseAdapterStage(com.vmware.photon.controller.model.adapters.util.BaseAdapterContext.BaseAdapterStage) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) NetworkInterfaceReferenceInner(com.microsoft.azure.management.compute.implementation.NetworkInterfaceReferenceInner) AzureDeferredResultServiceCallback(com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback) AZURE_STORAGE_ACCOUNT_NAME(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_ACCOUNT_NAME) CloudException(com.microsoft.azure.CloudException) ImageState(com.vmware.photon.controller.model.resources.ImageService.ImageState) StorageAccountInner(com.microsoft.azure.management.storage.implementation.StorageAccountInner) NetworkInterfaceDescription(com.vmware.photon.controller.model.resources.NetworkInterfaceDescriptionService.NetworkInterfaceDescription) NetworkInterfaceInner(com.microsoft.azure.management.network.implementation.NetworkInterfaceInner) CLOUD_CONFIG_DEFAULT_FILE_INDEX(com.vmware.photon.controller.model.constants.PhotonModelConstants.CLOUD_CONFIG_DEFAULT_FILE_INDEX) Collections(java.util.Collections) OperationJoin(com.vmware.xenon.common.OperationJoin) RESOURCE_GROUP_NAME(com.vmware.photon.controller.model.ComputeProperties.RESOURCE_GROUP_NAME) PROVIDER_REGISTRED_STATE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.PROVIDER_REGISTRED_STATE) OperationContext(com.vmware.xenon.common.OperationContext) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) CloudStorageAccount(com.microsoft.azure.storage.CloudStorageAccount) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) DeferredResult(com.vmware.xenon.common.DeferredResult) CloudPageBlob(com.microsoft.azure.storage.blob.CloudPageBlob)

Aggregations

OperationContext (com.vmware.xenon.common.OperationContext)17 Operation (com.vmware.xenon.common.Operation)6 StatelessService (com.vmware.xenon.common.StatelessService)6 AWSClientManager (com.vmware.photon.controller.model.adapters.awsadapter.util.AWSClientManager)5 DeferredResult (com.vmware.xenon.common.DeferredResult)5 AmazonEC2AsyncClient (com.amazonaws.services.ec2.AmazonEC2AsyncClient)4 AWSClientManagerFactory (com.vmware.photon.controller.model.adapters.awsadapter.util.AWSClientManagerFactory)4 BaseAdapterStage (com.vmware.photon.controller.model.adapters.util.BaseAdapterContext.BaseAdapterStage)4 ComputeState (com.vmware.photon.controller.model.resources.ComputeService.ComputeState)4 DiskService (com.vmware.photon.controller.model.resources.DiskService)4 PhotonModelUriUtils.createInventoryUri (com.vmware.photon.controller.model.util.PhotonModelUriUtils.createInventoryUri)4 Utils (com.vmware.xenon.common.Utils)4 ArrayList (java.util.ArrayList)4 AsyncHandler (com.amazonaws.handlers.AsyncHandler)3 StartInstancesRequest (com.amazonaws.services.ec2.model.StartInstancesRequest)3 StartInstancesResult (com.amazonaws.services.ec2.model.StartInstancesResult)3 StopInstancesRequest (com.amazonaws.services.ec2.model.StopInstancesRequest)3 StopInstancesResult (com.amazonaws.services.ec2.model.StopInstancesResult)3 AmazonClientException (com.amazonaws.AmazonClientException)2 AmazonServiceException (com.amazonaws.AmazonServiceException)2