Search in sources :

Example 1 with AzureContext

use of com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext in project cloudbreak by hortonworks.

the class AzureComputeResourceService method initContext.

private AzureContext initContext(AuthenticatedContext ac, CloudStack cloudStack) {
    AzureContext context = contextBuilder.contextInit(ac.getCloudContext(), ac, cloudStack.getNetwork(), null, true);
    String stackCrn = cloudStack.getParameters().getOrDefault(PlatformParametersConsts.RESOURCE_CRN_PARAMETER, "");
    if (stackCrn.isEmpty()) {
        LOGGER.warn("Stack crn is not set in CloudStack, it can cause errors during infrastructure creation!");
    }
    context.putParameter(PlatformParametersConsts.RESOURCE_CRN_PARAMETER, cloudStack.getParameters().getOrDefault(PlatformParametersConsts.RESOURCE_CRN_PARAMETER, ""));
    return context;
}
Also used : AzureContext(com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext)

Example 2 with AzureContext

use of com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext in project cloudbreak by hortonworks.

the class AzureAttachmentResourceBuilder method build.

@Override
public List<CloudResource> build(AzureContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, List<CloudResource> buildableResource, CloudStack cloudStack) {
    CloudResource cloudResourceInstance = buildableResource.stream().filter(cloudResource -> cloudResource.getType().equals(ResourceType.AZURE_INSTANCE)).findFirst().orElseThrow(() -> new AzureResourceException("Instance resource not found"));
    LOGGER.info("Attach disk to the instance {}", cloudResourceInstance);
    CloudContext cloudContext = auth.getCloudContext();
    String resourceGroupName = azureResourceGroupMetadataProvider.getResourceGroupName(cloudContext, cloudStack);
    AzureClient client = getAzureClient(auth);
    VirtualMachine vm = client.getVirtualMachineByResourceGroup(resourceGroupName, cloudResourceInstance.getName());
    Set<String> diskIds = vm.dataDisks().values().stream().map(VirtualMachineDataDisk::id).collect(Collectors.toSet());
    CloudResource volumeSet = buildableResource.stream().filter(cloudResource -> cloudResource.getType().equals(ResourceType.AZURE_VOLUMESET)).filter(cloudResource -> !cloudResourceInstance.getInstanceId().equals(cloudResource.getInstanceId())).findFirst().orElseThrow(() -> new AzureResourceException("Volume set resource not found"));
    VolumeSetAttributes volumeSetAttributes = getVolumeSetAttributes(volumeSet);
    volumeSetAttributes.getVolumes().forEach(volume -> {
        Disk disk = client.getDiskById(volume.getId());
        if (!diskIds.contains(disk.id())) {
            if (disk.isAttachedToVirtualMachine()) {
                detachDiskFromVmByVmId(client, disk);
            }
            attachDiskToVm(client, disk, vm);
        } else {
            LOGGER.info("Managed disk {} is already attached to VM {}", disk, vm);
        }
    });
    volumeSet.setInstanceId(cloudResourceInstance.getInstanceId());
    volumeSet.setStatus(CommonStatus.CREATED);
    LOGGER.info("Volume set {} attached successfully", volumeSet);
    return List.of(volumeSet);
}
Also used : AzureContext(com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext) ResourceType(com.sequenceiq.common.api.type.ResourceType) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) Logger(org.slf4j.Logger) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) VirtualMachineDataDisk(com.microsoft.azure.management.compute.VirtualMachineDataDisk) AzureResourceGroupMetadataProvider(com.sequenceiq.cloudbreak.cloud.azure.AzureResourceGroupMetadataProvider) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) LoggerFactory(org.slf4j.LoggerFactory) Set(java.util.Set) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Image(com.sequenceiq.cloudbreak.cloud.model.Image) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) Inject(javax.inject.Inject) Disk(com.microsoft.azure.management.compute.Disk) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) Component(org.springframework.stereotype.Component) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Group(com.sequenceiq.cloudbreak.cloud.model.Group) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) VirtualMachineDataDisk(com.microsoft.azure.management.compute.VirtualMachineDataDisk) Disk(com.microsoft.azure.management.compute.Disk) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes)

Example 3 with AzureContext

use of com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext in project cloudbreak by hortonworks.

the class AzureVolumeResourceBuilder method build.

@Override
public List<CloudResource> build(AzureContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, List<CloudResource> buildableResource, CloudStack cloudStack) throws Exception {
    LOGGER.info("Create volumes on provider");
    AzureClient client = getAzureClient(auth);
    Map<String, List<VolumeSetAttributes.Volume>> volumeSetMap = Collections.synchronizedMap(new HashMap<>());
    List<Future<?>> futures = new ArrayList<>();
    List<CloudResource> requestedResources = buildableResource.stream().filter(cloudResource -> CommonStatus.REQUESTED.equals(cloudResource.getStatus())).collect(toList());
    CloudContext cloudContext = auth.getCloudContext();
    String resourceGroupName = azureResourceGroupMetadataProvider.getResourceGroupName(cloudContext, cloudStack);
    String region = cloudContext.getLocation().getRegion().getRegionName();
    String diskEncryptionSetId = getDiskEncryptionSetId(group);
    for (CloudResource resource : requestedResources) {
        volumeSetMap.put(resource.getName(), Collections.synchronizedList(new ArrayList<>()));
        VolumeSetAttributes volumeSet = getVolumeSetAttributes(resource);
        DeviceNameGenerator generator = new DeviceNameGenerator(DEVICE_NAME_TEMPLATE, 1);
        futures.addAll(volumeSet.getVolumes().stream().map(volume -> intermediateBuilderExecutor.submit(() -> {
            Disk result = client.getDiskByName(resourceGroupName, volume.getId());
            if (result == null) {
                result = client.createManagedDisk(volume.getId(), volume.getSize(), AzureDiskType.getByValue(volume.getType()), region, resourceGroupName, cloudStack.getTags(), diskEncryptionSetId);
            } else {
                LOGGER.info("Managed disk for resource group: {}, name: {} already exists: {}", resourceGroupName, volume.getId(), result);
            }
            String volumeId = result.id();
            volumeSetMap.get(resource.getName()).add(new VolumeSetAttributes.Volume(volumeId, generator.next(), volume.getSize(), volume.getType(), volume.getCloudVolumeUsageType()));
        })).collect(toList()));
    }
    for (Future<?> future : futures) {
        future.get();
    }
    return buildableResource.stream().peek(resource -> {
        List<VolumeSetAttributes.Volume> volumes = volumeSetMap.get(resource.getName());
        if (!CollectionUtils.isEmpty(volumes)) {
            getVolumeSetAttributes(resource).setVolumes(volumes);
        }
    }).map(copyResourceWithNewStatus(CommonStatus.CREATED)).collect(toList());
}
Also used : AzureContext(com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) AvailabilityZoneId(com.microsoft.azure.management.resources.fluentcore.arm.AvailabilityZoneId) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) PlatformParametersConsts(com.sequenceiq.cloudbreak.cloud.PlatformParametersConsts) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) Future(java.util.concurrent.Future) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Map(java.util.Map) AzureInstanceView(com.sequenceiq.cloudbreak.cloud.azure.view.AzureInstanceView) AzureUtils(com.sequenceiq.cloudbreak.cloud.azure.AzureUtils) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) AzureResourceGroupMetadataProvider(com.sequenceiq.cloudbreak.cloud.azure.AzureResourceGroupMetadataProvider) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) AzureDiskType(com.sequenceiq.cloudbreak.cloud.azure.AzureDiskType) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) Backoff(org.springframework.retry.annotation.Backoff) Objects(java.util.Objects) Disk(com.microsoft.azure.management.compute.Disk) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) CollectionUtils(org.springframework.util.CollectionUtils) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) HashMap(java.util.HashMap) Image(com.sequenceiq.cloudbreak.cloud.model.Image) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) AzureResourceNameService(com.sequenceiq.cloudbreak.cloud.azure.service.AzureResourceNameService) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) PagedList(com.microsoft.azure.PagedList) Qualifier(org.springframework.beans.factory.annotation.Qualifier) StreamSupport(java.util.stream.StreamSupport) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) Retryable(org.springframework.retry.annotation.Retryable) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) Collectors.toList(java.util.stream.Collectors.toList) Component(org.springframework.stereotype.Component) PreserveResourceException(com.sequenceiq.cloudbreak.cloud.template.compute.PreserveResourceException) Collections(java.util.Collections) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) ArrayList(java.util.ArrayList) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) Future(java.util.concurrent.Future) List(java.util.List) ArrayList(java.util.ArrayList) PagedList(com.microsoft.azure.PagedList) Collectors.toList(java.util.stream.Collectors.toList) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Disk(com.microsoft.azure.management.compute.Disk) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator)

Example 4 with AzureContext

use of com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext in project cloudbreak by hortonworks.

the class AzureVolumeResourceBuilder method checkResources.

@Override
protected List<CloudResourceStatus> checkResources(ResourceType type, AzureContext context, AuthenticatedContext auth, Iterable<CloudResource> resources) {
    AzureClient client = getAzureClient(auth);
    List<CloudResource> volumeResources = StreamSupport.stream(resources.spliterator(), false).filter(r -> r.getType().equals(resourceType())).collect(toList());
    CloudResource resourceGroup = context.getNetworkResources().stream().filter(r -> r.getType().equals(ResourceType.AZURE_RESOURCE_GROUP)).findFirst().orElseThrow(() -> new AzureResourceException("Resource group resource not found"));
    String resourceGroupName = resourceGroup.getName();
    PagedList<Disk> existingDisks = client.listDisksByResourceGroup(resourceGroupName);
    ResourceStatus volumeSetStatus = getResourceStatus(existingDisks, volumeResources);
    return volumeResources.stream().map(resource -> new CloudResourceStatus(resource, volumeSetStatus)).collect(toList());
}
Also used : AzureContext(com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) AvailabilityZoneId(com.microsoft.azure.management.resources.fluentcore.arm.AvailabilityZoneId) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) PlatformParametersConsts(com.sequenceiq.cloudbreak.cloud.PlatformParametersConsts) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) Future(java.util.concurrent.Future) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Map(java.util.Map) AzureInstanceView(com.sequenceiq.cloudbreak.cloud.azure.view.AzureInstanceView) AzureUtils(com.sequenceiq.cloudbreak.cloud.azure.AzureUtils) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) AzureResourceGroupMetadataProvider(com.sequenceiq.cloudbreak.cloud.azure.AzureResourceGroupMetadataProvider) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) AzureDiskType(com.sequenceiq.cloudbreak.cloud.azure.AzureDiskType) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) Backoff(org.springframework.retry.annotation.Backoff) Objects(java.util.Objects) Disk(com.microsoft.azure.management.compute.Disk) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) CollectionUtils(org.springframework.util.CollectionUtils) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) HashMap(java.util.HashMap) Image(com.sequenceiq.cloudbreak.cloud.model.Image) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) AzureResourceNameService(com.sequenceiq.cloudbreak.cloud.azure.service.AzureResourceNameService) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) PagedList(com.microsoft.azure.PagedList) Qualifier(org.springframework.beans.factory.annotation.Qualifier) StreamSupport(java.util.stream.StreamSupport) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) Retryable(org.springframework.retry.annotation.Retryable) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) Collectors.toList(java.util.stream.Collectors.toList) Component(org.springframework.stereotype.Component) PreserveResourceException(com.sequenceiq.cloudbreak.cloud.template.compute.PreserveResourceException) Collections(java.util.Collections) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Disk(com.microsoft.azure.management.compute.Disk)

Example 5 with AzureContext

use of com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext in project cloudbreak by hortonworks.

the class AzureVolumeResourceBuilder method create.

@Override
public List<CloudResource> create(AzureContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, Image image) {
    LOGGER.info("Creating volume resources");
    List<CloudResource> computeResources = context.getComputeResources(privateId);
    if (Objects.isNull(computeResources) || computeResources.isEmpty()) {
        return null;
    }
    CloudResource vm = context.getComputeResources(privateId).stream().filter(cloudResource -> ResourceType.AZURE_INSTANCE.equals(cloudResource.getType())).findFirst().get();
    Optional<CloudResource> reattachableVolumeSet = computeResources.stream().filter(resource -> ResourceType.AZURE_VOLUMESET.equals(resource.getType())).filter(cloudResource -> CommonStatus.DETACHED.equals(cloudResource.getStatus()) || vm.getInstanceId().equals(cloudResource.getInstanceId())).findFirst();
    LOGGER.info("Reattachable volume set {}", reattachableVolumeSet.map(cloudResource -> "is present with name:" + cloudResource.getName()).orElse("is not present"));
    return List.of(reattachableVolumeSet.orElseGet(createVolumeSet(privateId, auth, group, vm, context.getStringParameter(PlatformParametersConsts.RESOURCE_CRN_PARAMETER))));
}
Also used : AzureContext(com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext) VirtualMachine(com.microsoft.azure.management.compute.VirtualMachine) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) AvailabilityZoneId(com.microsoft.azure.management.resources.fluentcore.arm.AvailabilityZoneId) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) PlatformParametersConsts(com.sequenceiq.cloudbreak.cloud.PlatformParametersConsts) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) Future(java.util.concurrent.Future) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Map(java.util.Map) AzureInstanceView(com.sequenceiq.cloudbreak.cloud.azure.view.AzureInstanceView) AzureUtils(com.sequenceiq.cloudbreak.cloud.azure.AzureUtils) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) AzureResourceGroupMetadataProvider(com.sequenceiq.cloudbreak.cloud.azure.AzureResourceGroupMetadataProvider) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) AzureDiskType(com.sequenceiq.cloudbreak.cloud.azure.AzureDiskType) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) Backoff(org.springframework.retry.annotation.Backoff) Objects(java.util.Objects) Disk(com.microsoft.azure.management.compute.Disk) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) CollectionUtils(org.springframework.util.CollectionUtils) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) AzureClient(com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient) HashMap(java.util.HashMap) Image(com.sequenceiq.cloudbreak.cloud.model.Image) Function(java.util.function.Function) Supplier(java.util.function.Supplier) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) AzureResourceNameService(com.sequenceiq.cloudbreak.cloud.azure.service.AzureResourceNameService) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) PagedList(com.microsoft.azure.PagedList) Qualifier(org.springframework.beans.factory.annotation.Qualifier) StreamSupport(java.util.stream.StreamSupport) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) Retryable(org.springframework.retry.annotation.Retryable) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) Collectors.toList(java.util.stream.Collectors.toList) Component(org.springframework.stereotype.Component) PreserveResourceException(com.sequenceiq.cloudbreak.cloud.template.compute.PreserveResourceException) Collections(java.util.Collections) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource)

Aggregations

AzureContext (com.sequenceiq.cloudbreak.cloud.azure.context.AzureContext)6 Disk (com.microsoft.azure.management.compute.Disk)5 VirtualMachine (com.microsoft.azure.management.compute.VirtualMachine)5 AzureResourceGroupMetadataProvider (com.sequenceiq.cloudbreak.cloud.azure.AzureResourceGroupMetadataProvider)5 AzureClient (com.sequenceiq.cloudbreak.cloud.azure.client.AzureClient)5 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)5 CloudContext (com.sequenceiq.cloudbreak.cloud.context.CloudContext)5 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)5 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)5 CloudStack (com.sequenceiq.cloudbreak.cloud.model.CloudStack)5 Group (com.sequenceiq.cloudbreak.cloud.model.Group)5 Image (com.sequenceiq.cloudbreak.cloud.model.Image)5 VolumeSetAttributes (com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes)5 CommonStatus (com.sequenceiq.common.api.type.CommonStatus)5 ResourceType (com.sequenceiq.common.api.type.ResourceType)5 List (java.util.List)5 Collectors (java.util.stream.Collectors)5 PagedList (com.microsoft.azure.PagedList)4 AvailabilityZoneId (com.microsoft.azure.management.resources.fluentcore.arm.AvailabilityZoneId)4 PlatformParametersConsts (com.sequenceiq.cloudbreak.cloud.PlatformParametersConsts)4