Search in sources :

Example 16 with Operation

use of com.google.api.services.compute.model.Operation in project cloudbreak by hortonworks.

the class GcpAttachedDiskResourceBuilder method checkResources.

@Override
public List<CloudResourceStatus> checkResources(GcpContext context, AuthenticatedContext auth, List<CloudResource> resources) {
    List<CloudResourceStatus> result = new ArrayList<>();
    for (CloudResource resource : resources) {
        LOGGER.debug("Check {} resource: {}", resourceType(), resource);
        List<String> operationIds = Optional.ofNullable(resource.getParameter(OPERATION_ID, List.class)).orElse(List.of());
        boolean finished = operationIds.isEmpty() || operationIds.stream().allMatch(operationId -> {
            try {
                Operation operation = getResourceChecker().check(context, operationId, resources);
                return operation == null || gcpStackUtil.isOperationFinished(operation);
            } catch (Exception e) {
                CloudContext cloudContext = auth.getCloudContext();
                throw new GcpResourceException("Error during status check", resourceType(), cloudContext.getName(), cloudContext.getId(), resource.getName(), e);
            }
        });
        ResourceStatus successStatus = context.isBuild() ? ResourceStatus.CREATED : ResourceStatus.DELETED;
        result.add(new CloudResourceStatus(resource, finished ? successStatus : ResourceStatus.IN_PROGRESS));
        if (finished) {
            if (successStatus == ResourceStatus.CREATED) {
                LOGGER.debug("Creation of {} was successful", resource);
            } else {
                LOGGER.debug("Deletion of {} was successful", resource);
            }
        }
    }
    return result;
}
Also used : CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) GcpResourceNameService(com.sequenceiq.cloudbreak.cloud.gcp.service.GcpResourceNameService) HashMap(java.util.HashMap) GcpLabelUtil(com.sequenceiq.cloudbreak.cloud.gcp.util.GcpLabelUtil) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Disk(com.google.api.services.compute.model.Disk) Image(com.sequenceiq.cloudbreak.cloud.model.Image) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Future(java.util.concurrent.Future) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) Map(java.util.Map) Qualifier(org.springframework.beans.factory.annotation.Qualifier) CustomGcpDiskEncryptionService(com.sequenceiq.cloudbreak.cloud.gcp.service.CustomGcpDiskEncryptionService) Insert(com.google.api.services.compute.Compute.Disks.Insert) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) GcpContext(com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext) GcpStackUtil(com.sequenceiq.cloudbreak.cloud.gcp.util.GcpStackUtil) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Collection(java.util.Collection) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) IOException(java.io.IOException) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) Operation(com.google.api.services.compute.model.Operation) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) Component(org.springframework.stereotype.Component) TokenResponseException(com.google.api.client.auth.oauth2.TokenResponseException) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) Volume(com.sequenceiq.cloudbreak.cloud.model.Volume) Collections(java.util.Collections) Compute(com.google.api.services.compute.Compute) GcpResourceException(com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException) GcpDiskType(com.sequenceiq.cloudbreak.cloud.gcp.GcpPlatformParameters.GcpDiskType) CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) ArrayList(java.util.ArrayList) Operation(com.google.api.services.compute.model.Operation) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) IOException(java.io.IOException) TokenResponseException(com.google.api.client.auth.oauth2.TokenResponseException) GcpResourceException(com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) GcpResourceException(com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource)

Example 17 with Operation

use of com.google.api.services.compute.model.Operation in project cloudbreak by hortonworks.

the class GcpAttachedDiskResourceBuilder method build.

@Override
public List<CloudResource> build(GcpContext context, CloudInstance instance, long privateId, AuthenticatedContext auth, Group group, List<CloudResource> resources, CloudStack cloudStack) throws Exception {
    InstanceTemplate template = group.getReferenceInstanceTemplate();
    List<String> operations = new ArrayList<>();
    List<String> syncedOperations = Collections.synchronizedList(operations);
    String projectId = context.getProjectId();
    Compute compute = context.getCompute();
    Collection<Future<Void>> futures = new ArrayList<>();
    List<CloudResource> buildableResource = resources.stream().filter(cloudResource -> CommonStatus.REQUESTED.equals(cloudResource.getStatus())).collect(Collectors.toList());
    List<CloudResource> result = new ArrayList<>();
    for (CloudResource volumeSetResource : buildableResource) {
        VolumeSetAttributes volumeSetAttributes = volumeSetResource.getParameter(CloudResource.ATTRIBUTES, VolumeSetAttributes.class);
        for (VolumeSetAttributes.Volume volume : volumeSetAttributes.getVolumes()) {
            Map<String, String> labels = gcpLabelUtil.createLabelsFromTags(cloudStack);
            Disk disk = createDisk(projectId, volume, labels, volumeSetAttributes);
            customGcpDiskEncryptionService.addEncryptionKeyToDisk(template, disk);
            Future<Void> submit = intermediateBuilderExecutor.submit(() -> {
                Insert insDisk = compute.disks().insert(projectId, volumeSetAttributes.getAvailabilityZone(), disk);
                try {
                    Operation operation = insDisk.execute();
                    syncedOperations.add(operation.getName());
                    if (operation.getHttpErrorStatusCode() != null) {
                        throw new GcpResourceException(operation.getHttpErrorMessage(), resourceType(), disk.getName());
                    }
                } catch (TokenResponseException e) {
                    throw gcpStackUtil.getMissingServiceAccountKeyError(e, projectId);
                } catch (GoogleJsonResponseException e) {
                    throw new GcpResourceException(checkException(e), resourceType(), disk.getName());
                }
                return null;
            });
            futures.add(submit);
        }
        volumeSetResource.putParameter(OPERATION_ID, operations);
        result.add(new Builder().cloudResource(volumeSetResource).status(CommonStatus.CREATED).params(volumeSetResource.getParameters()).build());
    }
    for (Future<Void> future : futures) {
        future.get();
    }
    result.addAll(resources.stream().filter(cloudResource -> CommonStatus.CREATED.equals(cloudResource.getStatus())).collect(Collectors.toList()));
    return result;
}
Also used : CloudContext(com.sequenceiq.cloudbreak.cloud.context.CloudContext) AsyncTaskExecutor(org.springframework.core.task.AsyncTaskExecutor) LoggerFactory(org.slf4j.LoggerFactory) GcpResourceNameService(com.sequenceiq.cloudbreak.cloud.gcp.service.GcpResourceNameService) HashMap(java.util.HashMap) GcpLabelUtil(com.sequenceiq.cloudbreak.cloud.gcp.util.GcpLabelUtil) CloudInstance(com.sequenceiq.cloudbreak.cloud.model.CloudInstance) Disk(com.google.api.services.compute.model.Disk) Image(com.sequenceiq.cloudbreak.cloud.model.Image) DeviceNameGenerator(com.sequenceiq.cloudbreak.util.DeviceNameGenerator) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Future(java.util.concurrent.Future) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) Map(java.util.Map) Qualifier(org.springframework.beans.factory.annotation.Qualifier) CustomGcpDiskEncryptionService(com.sequenceiq.cloudbreak.cloud.gcp.service.CustomGcpDiskEncryptionService) Insert(com.google.api.services.compute.Compute.Disks.Insert) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate) GcpContext(com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext) GcpStackUtil(com.sequenceiq.cloudbreak.cloud.gcp.util.GcpStackUtil) CommonStatus(com.sequenceiq.common.api.type.CommonStatus) ResourceStatus(com.sequenceiq.cloudbreak.cloud.model.ResourceStatus) ResourceType(com.sequenceiq.common.api.type.ResourceType) Logger(org.slf4j.Logger) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Collection(java.util.Collection) CloudResourceStatus(com.sequenceiq.cloudbreak.cloud.model.CloudResourceStatus) IOException(java.io.IOException) PersistenceNotifier(com.sequenceiq.cloudbreak.cloud.notification.PersistenceNotifier) Operation(com.google.api.services.compute.model.Operation) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) Collectors(java.util.stream.Collectors) CloudStack(com.sequenceiq.cloudbreak.cloud.model.CloudStack) List(java.util.List) Component(org.springframework.stereotype.Component) TokenResponseException(com.google.api.client.auth.oauth2.TokenResponseException) Group(com.sequenceiq.cloudbreak.cloud.model.Group) Optional(java.util.Optional) Volume(com.sequenceiq.cloudbreak.cloud.model.Volume) Collections(java.util.Collections) Compute(com.google.api.services.compute.Compute) GcpResourceException(com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException) GcpDiskType(com.sequenceiq.cloudbreak.cloud.gcp.GcpPlatformParameters.GcpDiskType) Builder(com.sequenceiq.cloudbreak.cloud.model.CloudResource.Builder) ArrayList(java.util.ArrayList) Operation(com.google.api.services.compute.model.Operation) Insert(com.google.api.services.compute.Compute.Disks.Insert) VolumeSetAttributes(com.sequenceiq.cloudbreak.cloud.model.VolumeSetAttributes) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) Compute(com.google.api.services.compute.Compute) Future(java.util.concurrent.Future) GcpResourceException(com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) Disk(com.google.api.services.compute.model.Disk) TokenResponseException(com.google.api.client.auth.oauth2.TokenResponseException) InstanceTemplate(com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate)

Example 18 with Operation

use of com.google.api.services.compute.model.Operation in project cloudbreak by hortonworks.

the class GcpFirewallInResourceBuilderTest method testDeleteWhenEverythingGoesFine.

@Test
public void testDeleteWhenEverythingGoesFine() throws Exception {
    CloudResource resource = new CloudResource.Builder().type(ResourceType.GCP_FIREWALL_IN).status(CommonStatus.CREATED).group("master").name("super").instanceId("id-123").params(new HashMap<>()).persistent(true).build();
    GcpContext gcpContext = mock(GcpContext.class);
    AuthenticatedContext authenticatedContext = mock(AuthenticatedContext.class);
    Network network = mock(Network.class);
    Compute compute = mock(Compute.class);
    Compute.Firewalls firewalls = mock(Compute.Firewalls.class);
    Compute.Firewalls.Delete firewallsDelete = mock(Compute.Firewalls.Delete.class);
    Operation operation = mock(Operation.class);
    when(gcpContext.getCompute()).thenReturn(compute);
    when(gcpContext.getProjectId()).thenReturn("id");
    when(compute.firewalls()).thenReturn(firewalls);
    when(firewalls.delete(anyString(), anyString())).thenReturn(firewallsDelete);
    when(firewallsDelete.execute()).thenReturn(operation);
    when(operation.getName()).thenReturn("name");
    CloudResource delete = underTest.delete(gcpContext, authenticatedContext, resource, network);
    Assert.assertEquals(ResourceType.GCP_FIREWALL_IN, delete.getType());
    Assert.assertEquals(CommonStatus.CREATED, delete.getStatus());
    Assert.assertEquals("super", delete.getName());
    Assert.assertEquals("master", delete.getGroup());
    Assert.assertEquals("id-123", delete.getInstanceId());
}
Also used : GcpContext(com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext) Network(com.sequenceiq.cloudbreak.cloud.model.Network) Compute(com.google.api.services.compute.Compute) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Operation(com.google.api.services.compute.model.Operation) Test(org.junit.jupiter.api.Test)

Example 19 with Operation

use of com.google.api.services.compute.model.Operation in project cloudbreak by hortonworks.

the class GcpFirewallInternalResourceBuilderTest method testDeleteWhenEverythingGoesFine.

@Test
public void testDeleteWhenEverythingGoesFine() throws Exception {
    CloudResource resource = new CloudResource.Builder().type(ResourceType.GCP_FIREWALL_INTERNAL).status(CommonStatus.CREATED).group("master").name("super").instanceId("id-123").params(new HashMap<>()).persistent(true).build();
    GcpContext gcpContext = mock(GcpContext.class);
    AuthenticatedContext authenticatedContext = mock(AuthenticatedContext.class);
    Network network = mock(Network.class);
    Compute compute = mock(Compute.class);
    Compute.Firewalls firewalls = mock(Compute.Firewalls.class);
    Compute.Firewalls.Delete firewallsDelete = mock(Compute.Firewalls.Delete.class);
    Operation operation = mock(Operation.class);
    when(gcpContext.getCompute()).thenReturn(compute);
    when(gcpContext.getProjectId()).thenReturn("id");
    when(compute.firewalls()).thenReturn(firewalls);
    when(firewalls.delete(anyString(), anyString())).thenReturn(firewallsDelete);
    when(firewallsDelete.execute()).thenReturn(operation);
    when(operation.getName()).thenReturn("name");
    CloudResource delete = underTest.delete(gcpContext, authenticatedContext, resource, network);
    Assert.assertEquals(ResourceType.GCP_FIREWALL_INTERNAL, delete.getType());
    Assert.assertEquals(CommonStatus.CREATED, delete.getStatus());
    Assert.assertEquals("super", delete.getName());
    Assert.assertEquals("master", delete.getGroup());
    Assert.assertEquals("id-123", delete.getInstanceId());
}
Also used : GcpContext(com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext) Network(com.sequenceiq.cloudbreak.cloud.model.Network) Compute(com.google.api.services.compute.Compute) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Operation(com.google.api.services.compute.model.Operation) Test(org.junit.jupiter.api.Test)

Example 20 with Operation

use of com.google.api.services.compute.model.Operation in project cloudbreak by hortonworks.

the class GcpSubnetResourceBuilderTest method testDeleteWhenEverythingGoesFine.

@Test
public void testDeleteWhenEverythingGoesFine() throws Exception {
    CloudResource resource = new CloudResource.Builder().type(ResourceType.GCP_NETWORK).status(CommonStatus.CREATED).group("master").name("super").instanceId("id-123").params(new HashMap<>()).persistent(true).build();
    GcpContext gcpContext = mock(GcpContext.class);
    AuthenticatedContext authenticatedContext = mock(AuthenticatedContext.class);
    Network network = mock(Network.class);
    Compute compute = mock(Compute.class);
    Location location = mock(Location.class);
    Compute.Subnetworks networks = mock(Compute.Subnetworks.class);
    Compute.Subnetworks.Delete networksDelete = mock(Compute.Subnetworks.Delete.class);
    Operation operation = mock(Operation.class);
    when(gcpStackUtil.isNewNetworkAndSubnet(network)).thenReturn(true);
    when(gcpContext.getLocation()).thenReturn(location);
    when(location.getRegion()).thenReturn(Region.region("us-west-1"));
    when(gcpContext.getCompute()).thenReturn(compute);
    when(gcpContext.getProjectId()).thenReturn("id");
    when(compute.subnetworks()).thenReturn(networks);
    when(networks.delete(anyString(), anyString(), anyString())).thenReturn(networksDelete);
    when(networksDelete.execute()).thenReturn(operation);
    when(operation.getName()).thenReturn("name");
    CloudResource delete = underTest.delete(gcpContext, authenticatedContext, resource, network);
    Assert.assertEquals(ResourceType.GCP_NETWORK, delete.getType());
    Assert.assertEquals(CommonStatus.CREATED, delete.getStatus());
    Assert.assertEquals("super", delete.getName());
    Assert.assertEquals("master", delete.getGroup());
    Assert.assertEquals("id-123", delete.getInstanceId());
}
Also used : GcpContext(com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext) Network(com.sequenceiq.cloudbreak.cloud.model.Network) Compute(com.google.api.services.compute.Compute) CloudResource(com.sequenceiq.cloudbreak.cloud.model.CloudResource) AuthenticatedContext(com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext) Operation(com.google.api.services.compute.model.Operation) Location(com.sequenceiq.cloudbreak.cloud.model.Location) Test(org.junit.jupiter.api.Test)

Aggregations

Operation (com.google.api.services.compute.model.Operation)94 Compute (com.google.api.services.compute.Compute)38 GoogleJsonResponseException (com.google.api.client.googleapis.json.GoogleJsonResponseException)37 IOException (java.io.IOException)35 GcpResourceException (com.sequenceiq.cloudbreak.cloud.gcp.GcpResourceException)23 CloudResource (com.sequenceiq.cloudbreak.cloud.model.CloudResource)21 Test (org.junit.Test)19 ArrayList (java.util.ArrayList)17 CloudInstance (com.sequenceiq.cloudbreak.cloud.model.CloudInstance)15 Instance (com.google.api.services.compute.model.Instance)14 AutoScalingData (org.apache.druid.indexing.overlord.autoscaling.AutoScalingData)14 GcpContext (com.sequenceiq.cloudbreak.cloud.gcp.context.GcpContext)12 InstanceGroupManagersListManagedInstancesResponse (com.google.api.services.compute.model.InstanceGroupManagersListManagedInstancesResponse)10 Location (com.sequenceiq.cloudbreak.cloud.model.Location)10 HashMap (java.util.HashMap)10 InstanceTemplate (com.sequenceiq.cloudbreak.cloud.model.InstanceTemplate)9 Disk (com.google.api.services.compute.model.Disk)8 InstanceGroupManagersDeleteInstancesRequest (com.google.api.services.compute.model.InstanceGroupManagersDeleteInstancesRequest)8 AuthenticatedContext (com.sequenceiq.cloudbreak.cloud.context.AuthenticatedContext)8 CloudContext (com.sequenceiq.cloudbreak.cloud.context.CloudContext)8