Search in sources :

Example 51 with Azure

use of com.microsoft.azure.management.Azure in project photon-model by vmware.

the class AzureStorageEnumerationAdapterService method disassociateDiskStates.

/*
     * Disassociate local disk states that no longer exist in Azure
     *
     * The logic works by recording a timestamp when enumeration starts. This timestamp is used to
     * lookup resources which haven't been touched as part of current enumeration cycle. The other
     * data point this method uses is the blob discovered as part of get blob call.
     *
     * A disassociate on a disk state is invoked only if it meets two criteria: - Timestamp older
     * than current enumeration cycle. - blob is not present on Azure.
     */
private void disassociateDiskStates(StorageEnumContext context, StorageEnumStages next) {
    Query.Builder qBuilder = Query.Builder.create().addKindFieldClause(DiskState.class).addFieldClause(DiskState.FIELD_NAME_COMPUTE_HOST_LINK, context.parentCompute.documentSelfLink).addRangeClause(DiskState.FIELD_NAME_UPDATE_TIME_MICROS, QueryTask.NumericRange.createLessThanRange(context.enumerationStartTimeInMicros));
    Query.Builder typeFilterQuery = Query.Builder.create(Occurance.MUST_OCCUR);
    Query blobFilter = Query.Builder.create(Occurance.SHOULD_OCCUR).addFieldClause(AZURE_STORAGE_TYPE, AZURE_STORAGE_BLOBS).build();
    QueryTask.Query diskFilter = QueryTask.Query.Builder.create(QueryTask.Query.Occurance.SHOULD_OCCUR).addFieldClause(AZURE_STORAGE_TYPE, AZURE_STORAGE_DISKS).build();
    typeFilterQuery.addClause(blobFilter);
    typeFilterQuery.addClause(diskFilter);
    qBuilder.addClause(typeFilterQuery.build());
    QueryByPages<DiskState> queryLocalStates = new QueryByPages<>(getHost(), qBuilder.build(), DiskState.class, context.parentCompute.tenantLinks, context.request.endpointLink).setMaxPageSize(getQueryResultLimit());
    queryLocalStates.setClusterType(ServiceTypeCluster.INVENTORY_SERVICE);
    List<DeferredResult<Operation>> ops = new ArrayList<>();
    queryLocalStates.queryDocuments(ds -> {
        if (context.blobIds.contains(ds.id)) {
            return;
        }
        ops.add(disassociateIfNotAttachedToCompute(context, ds));
    }).thenCompose(r -> DeferredResult.allOf(ops)).whenComplete((r, e) -> {
        logFine(() -> "Finished disassociation of disk states for Azure");
        context.subStage = next;
        handleSubStage(context);
    });
}
Also used : STORAGE_ACCOUNT_REST_API_VERSION(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.STORAGE_ACCOUNT_REST_API_VERSION) Arrays(java.util.Arrays) QUERY_PARAM_API_VERSION(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.QUERY_PARAM_API_VERSION) ComputeEnumerateResourceRequest(com.vmware.photon.controller.model.adapterapi.ComputeEnumerateResourceRequest) AZURE_STORAGE_BLOBS(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_BLOBS) ServiceTypeCluster(com.vmware.photon.controller.model.util.ClusterUtil.ServiceTypeCluster) StringUtils(org.apache.commons.lang3.StringUtils) ResourceGroupState(com.vmware.photon.controller.model.resources.ResourceGroupService.ResourceGroupState) Azure(com.microsoft.azure.management.Azure) Utils(com.vmware.xenon.common.Utils) Map(java.util.Map) StorageDescription(com.vmware.photon.controller.model.resources.StorageDescriptionService.StorageDescription) COMPUTE_HOST_LINK_PROP_NAME(com.vmware.photon.controller.model.ComputeProperties.COMPUTE_HOST_LINK_PROP_NAME) EnumSet(java.util.EnumSet) ListBlobItem(com.microsoft.azure.storage.blob.ListBlobItem) AZURE_STORAGE_ACCOUNT_KEY1(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_ACCOUNT_KEY1) StorageAccountListKeysResultInner(com.microsoft.azure.management.storage.implementation.StorageAccountListKeysResultInner) StatelessService(com.vmware.xenon.common.StatelessService) Set(java.util.Set) AdapterUtils.getDeletionState(com.vmware.photon.controller.model.adapters.util.AdapterUtils.getDeletionState) AZURE_STORAGE_CONTAINER_LEASE_STATUS(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_CONTAINER_LEASE_STATUS) TagService(com.vmware.photon.controller.model.resources.TagService) StorageDescriptionService(com.vmware.photon.controller.model.resources.StorageDescriptionService) CompletionHandler(com.vmware.xenon.common.Operation.CompletionHandler) DeferredResult(com.vmware.xenon.common.DeferredResult) UriUtils(com.vmware.xenon.common.UriUtils) ComputeService(com.vmware.photon.controller.model.resources.ComputeService) AZURE_STORAGE_CONTAINER_LEASE_STATE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_CONTAINER_LEASE_STATE) ComputeProperties(com.vmware.photon.controller.model.ComputeProperties) ResourceGroupStateType(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.ResourceGroupStateType) PhotonModelUtils(com.vmware.photon.controller.model.resources.util.PhotonModelUtils) ArrayList(java.util.ArrayList) StorageException(com.microsoft.azure.storage.StorageException) TagState(com.vmware.photon.controller.model.resources.TagService.TagState) Query(com.vmware.xenon.services.common.QueryTask.Query) AUTH_HEADER_BEARER_PREFIX(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AUTH_HEADER_BEARER_PREFIX) UriPaths(com.vmware.photon.controller.model.UriPaths) EnumerationStages(com.vmware.photon.controller.model.adapters.util.enums.EnumerationStages) CloudBlobContainer(com.microsoft.azure.storage.blob.CloudBlobContainer) StorageErrorCode(com.microsoft.azure.storage.StorageErrorCode) EnumUtils(org.apache.commons.lang3.EnumUtils) AZURE_STORAGE_CONTAINER_LEASE_LAST_MODIFIED(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_CONTAINER_LEASE_LAST_MODIFIED) AdapterUtils(com.vmware.photon.controller.model.adapters.util.AdapterUtils) LIST_STORAGE_ACCOUNTS(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.LIST_STORAGE_ACCOUNTS) ResourceState(com.vmware.photon.controller.model.resources.ResourceState) QueryUtils(com.vmware.photon.controller.model.query.QueryUtils) ResultSegment(com.microsoft.azure.storage.ResultSegment) EMPTY_STR(com.vmware.photon.controller.model.constants.PhotonModelConstants.EMPTY_STR) ContainerListingDetails(com.microsoft.azure.storage.blob.ContainerListingDetails) UnknownHostException(java.net.UnknownHostException) QueryTop(com.vmware.photon.controller.model.query.QueryUtils.QueryTop) ComputeStateWithDescription(com.vmware.photon.controller.model.resources.ComputeService.ComputeStateWithDescription) ComputeEnumerateAdapterRequest(com.vmware.photon.controller.model.adapters.util.ComputeEnumerateAdapterRequest) QuerySpecification(com.vmware.xenon.services.common.QueryTask.QuerySpecification) PhotonModelUriUtils.createInventoryUri(com.vmware.photon.controller.model.util.PhotonModelUriUtils.createInventoryUri) AuthCredentialsServiceState(com.vmware.xenon.services.common.AuthCredentialsService.AuthCredentialsServiceState) ResultContinuation(com.microsoft.azure.storage.ResultContinuation) URISyntaxException(java.net.URISyntaxException) QueryTask(com.vmware.xenon.services.common.QueryTask) AzureUriPaths(com.vmware.photon.controller.model.adapters.azure.AzureUriPaths) DEFAULT_DISK_TYPE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.DEFAULT_DISK_TYPE) AzureSdkClients(com.vmware.photon.controller.model.adapters.azure.utils.AzureSdkClients) CUSTOM_PROP_ENDPOINT_LINK(com.vmware.photon.controller.model.constants.PhotonModelConstants.CUSTOM_PROP_ENDPOINT_LINK) URI(java.net.URI) TagsUtil.newTagState(com.vmware.photon.controller.model.adapters.util.TagsUtil.newTagState) AzureConstants(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants) AzureConstants.getQueryResultLimit(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.getQueryResultLimit) Collection(java.util.Collection) ConcurrentHashMap(java.util.concurrent.ConcurrentHashMap) AZURE_STORAGE_CONTAINERS(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_CONTAINERS) DiskState(com.vmware.photon.controller.model.resources.DiskService.DiskState) Occurance(com.vmware.xenon.services.common.QueryTask.Query.Occurance) UUID(java.util.UUID) CloudStorageAccount(com.microsoft.azure.storage.CloudStorageAccount) Collectors(java.util.stream.Collectors) ResourceGroupService(com.vmware.photon.controller.model.resources.ResourceGroupService) List(java.util.List) AzureUtils(com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils) AzureUtils.getResourceGroupName(com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils.getResourceGroupName) TAG_KEY_TYPE(com.vmware.photon.controller.model.constants.PhotonModelConstants.TAG_KEY_TYPE) QueryOption(com.vmware.xenon.services.common.QueryTask.QuerySpecification.QueryOption) DiskService(com.vmware.photon.controller.model.resources.DiskService) Default(com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback.Default) QueryByPages(com.vmware.photon.controller.model.query.QueryUtils.QueryByPages) HashMap(java.util.HashMap) Level(java.util.logging.Level) HashSet(java.util.HashSet) AZURE_STORAGE_TYPE(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_TYPE) AuthCredentialsService(com.vmware.xenon.services.common.AuthCredentialsService) EnumerationAction(com.vmware.photon.controller.model.adapterapi.EnumerationAction) AzureResourceType(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AzureResourceType) STORAGE_CONNECTION_STRING(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.STORAGE_CONNECTION_STRING) ExecutorService(java.util.concurrent.ExecutorService) StorageAccountResultList(com.vmware.photon.controller.model.adapters.azure.model.storage.StorageAccountResultList) CloudBlobClient(com.microsoft.azure.storage.blob.CloudBlobClient) AdapterUriUtil(com.vmware.photon.controller.model.adapters.util.AdapterUriUtil) Operation(com.vmware.xenon.common.Operation) AZURE_STORAGE_DISKS(com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AZURE_STORAGE_DISKS) StorageAccountsInner(com.microsoft.azure.management.storage.implementation.StorageAccountsInner) TimeUnit(java.util.concurrent.TimeUnit) Consumer(java.util.function.Consumer) StorageCredentials(com.microsoft.azure.storage.StorageCredentials) AzureUtils.canonizeId(com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils.canonizeId) BlobListingDetails(com.microsoft.azure.storage.blob.BlobListingDetails) AzureDeferredResultServiceCallback(com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback) StorageAccountInner(com.microsoft.azure.management.storage.implementation.StorageAccountInner) StorageAccount(com.vmware.photon.controller.model.adapters.azure.model.storage.StorageAccount) OperationJoin(com.vmware.xenon.common.OperationJoin) CloudBlob(com.microsoft.azure.storage.blob.CloudBlob) QueryTask(com.vmware.xenon.services.common.QueryTask) Query(com.vmware.xenon.services.common.QueryTask.Query) Query(com.vmware.xenon.services.common.QueryTask.Query) DiskState(com.vmware.photon.controller.model.resources.DiskService.DiskState) ArrayList(java.util.ArrayList) DeferredResult(com.vmware.xenon.common.DeferredResult)

Example 52 with Azure

use of com.microsoft.azure.management.Azure in project photon-model by vmware.

the class TestAzureDeleteTask method setupStorageAccount.

@Before
public final void setupStorageAccount() throws Throwable {
    if (this.isMock) {
        return;
    }
    // Create Resource Group
    Azure azure = getAzureSdkClients().getAzureClient();
    resourceGroupName = SdkContext.randomResourceName(azureVMName, azureVMName.length() + 5);
    azure.resourceGroups().define(resourceGroupName).withRegion(AZURE_RESOURCE_GROUP_LOCATION).create();
    // Create Storage Account
    storageAccountName = azureVMName.replace("_", "") + "teststorageacc";
    azure.storageAccounts().define(storageAccountName).withRegion(AZURE_RESOURCE_GROUP_LOCATION).withExistingResourceGroup(resourceGroupName).create();
}
Also used : Azure(com.microsoft.azure.management.Azure) Before(org.junit.Before)

Example 53 with Azure

use of com.microsoft.azure.management.Azure in project cloudbreak by hortonworks.

the class TagsUtil method checkTagsAzure.

protected static void checkTagsAzure(String accesKey, String tenantId, String secretKey, String subscriptionId, CharSequence stackName, Map<String, String> tagsToCheckMap) {
    ApplicationTokenCredentials serviceClientCredentials = new ApplicationTokenCredentials(accesKey, tenantId, secretKey, null);
    Azure azure = Azure.authenticate(serviceClientCredentials).withSubscription(subscriptionId);
    PagedList<VirtualMachine> virtualMachinesList = azure.virtualMachines().list();
    for (VirtualMachine vm : virtualMachinesList) {
        if (vm.name().contains(stackName)) {
            Map<String, String> extractedTags = vm.tags();
            checkTags(tagsToCheckMap, extractedTags);
        }
    }
}
Also used : Azure(com.microsoft.azure.management.Azure) ApplicationTokenCredentials(com.microsoft.azure.credentials.ApplicationTokenCredentials) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine)

Example 54 with Azure

use of com.microsoft.azure.management.Azure in project cloudbreak by hortonworks.

the class AzureRmUseAnExistingSubnetInAnExistingVpcNetworkTest method createNetwork.

@Test
@Parameters({ "networkName", "description", "publicInAccount", "regionName", "resourceGroupName", "vpcName", "vpcSubnet" })
public void createNetwork(String networkName, @Optional("") String description, @Optional("false") boolean publicInAccount, String regionName, @Optional("it-vpc-resource-group") String resourceGroupName, @Optional("it-vpc") String vpcName, @Optional("it-vpc-subnet") String vpcSubnet) {
    ApplicationTokenCredentials serviceClientCredentials = new ApplicationTokenCredentials(defaultAccesKey, defaultTenantId, defaultSecretKey, null);
    Azure azure = Azure.authenticate(serviceClientCredentials).withSubscription(defaultSubscriptionId);
    azure.networks().define(vpcName).withRegion(regionName).withNewResourceGroup(resourceGroupName).withAddressSpace("10.0.0.0/16").withSubnet(vpcSubnet, "10.0.0.0/16").create();
    NetworkRequest networkRequest = new NetworkRequest();
    networkRequest.setName(networkName);
    networkRequest.setDescription(description);
    Map<String, Object> map = new HashMap<>();
    map.put("networkId", vpcName);
    map.put("subnetId", vpcSubnet);
    map.put("resourceGroupName", resourceGroupName);
    networkRequest.setParameters(map);
    networkRequest.setCloudPlatform("AZURE");
    String id = getCloudbreakClient().networkEndpoint().postPrivate(networkRequest).getId().toString();
    getItContext().putContextParam(CloudbreakITContextConstants.NETWORK_ID, id, true);
}
Also used : Azure(com.microsoft.azure.management.Azure) HashMap(java.util.HashMap) NetworkRequest(com.sequenceiq.cloudbreak.api.model.NetworkRequest) ApplicationTokenCredentials(com.microsoft.azure.credentials.ApplicationTokenCredentials) Parameters(org.testng.annotations.Parameters) Test(org.testng.annotations.Test)

Example 55 with Azure

use of com.microsoft.azure.management.Azure in project cloudbreak by hortonworks.

the class AzureDeleteVpcTest method deleteNetwork.

@AfterSuite
@Parameters({ "resourceGroupName", "vpcName" })
public void deleteNetwork(@Optional("it-vpc-resource-group") String resourceGroupName, @Optional("it-vpc") String vpcName) throws Exception {
    springTestContextPrepareTestInstance();
    ApplicationTokenCredentials serviceClientCredentials = new ApplicationTokenCredentials(defaultAccesKey, defaultTenantId, defaultSecretKey, null);
    Azure azure = Azure.authenticate(serviceClientCredentials).withSubscription(defaultSubscriptionId);
    azure.networks().deleteByResourceGroup(resourceGroupName, vpcName);
}
Also used : Azure(com.microsoft.azure.management.Azure) ApplicationTokenCredentials(com.microsoft.azure.credentials.ApplicationTokenCredentials) Parameters(org.testng.annotations.Parameters) AfterSuite(org.testng.annotations.AfterSuite)

Aggregations

Azure (com.microsoft.azure.management.Azure)144 File (java.io.File)75 ArrayList (java.util.ArrayList)20 HashMap (java.util.HashMap)20 ExecutorService (java.util.concurrent.ExecutorService)17 ComputeService (com.vmware.photon.controller.model.resources.ComputeService)16 DiskService (com.vmware.photon.controller.model.resources.DiskService)16 ComputeEnumerateResourceRequest (com.vmware.photon.controller.model.adapterapi.ComputeEnumerateResourceRequest)15 EnumerationAction (com.vmware.photon.controller.model.adapterapi.EnumerationAction)15 AzureUriPaths (com.vmware.photon.controller.model.adapters.azure.AzureUriPaths)15 AzureConstants (com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants)15 AzureResourceType (com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.AzureResourceType)15 AzureConstants.getQueryResultLimit (com.vmware.photon.controller.model.adapters.azure.constants.AzureConstants.getQueryResultLimit)15 AzureDeferredResultServiceCallback (com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback)15 Default (com.vmware.photon.controller.model.adapters.azure.utils.AzureDeferredResultServiceCallback.Default)15 AzureSdkClients (com.vmware.photon.controller.model.adapters.azure.utils.AzureSdkClients)15 AzureUtils (com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils)15 AzureUtils.getResourceGroupName (com.vmware.photon.controller.model.adapters.azure.utils.AzureUtils.getResourceGroupName)15 AdapterUtils (com.vmware.photon.controller.model.adapters.util.AdapterUtils)15 ComputeEnumerateAdapterRequest (com.vmware.photon.controller.model.adapters.util.ComputeEnumerateAdapterRequest)15