Search in sources :

Example 1 with Operation

use of com.google.api.services.cloudresourcemanager.v3.model.Operation in project terra-resource-buffer by DataBiosphere.

the class CreateConsumerDefinedQuotaForBigQueryDailyUsageStep method doStep.

/**
 * Apply a Consumer Quota Override for the BigQuery Query Usage Quota.
 */
@Override
public StepResult doStep(FlightContext context) throws InterruptedException, RetryException {
    Optional<Long> overrideValue = GoogleProjectConfigUtils.bigQueryDailyUsageOverrideValueMebibytes(gcpProjectConfig);
    if (overrideValue.isEmpty()) {
        // Do not apply any quota override
        return StepResult.getStepResultSuccess();
    }
    long projectNumber = Optional.ofNullable(context.getWorkingMap().get(GOOGLE_PROJECT_NUMBER, Long.class)).orElseThrow();
    QuotaOverride overridePerProjectPerDay = buildQuotaOverride(projectNumber, overrideValue.get());
    // parent format and other details obtained by hitting the endpoint
    // https://serviceusage.googleapis.com/v1beta1/projects/${PROJECT_NUMBER}/services/bigquery.googleapis.com/consumerQuotaMetrics
    String parent = String.format("projects/%d/services/bigquery.googleapis.com/consumerQuotaMetrics/" + "bigquery.googleapis.com%%2Fquota%%2Fquery%%2Fusage/limits/%%2Fd%%2Fproject", projectNumber);
    try {
        // We are decreasing the quota by more than 10%, so we must tell Service Usage to bypass the
        // check with the force flag.
        Operation createOperation = serviceUsageCow.services().consumerQuotaMetrics().limits().consumerOverrides().create(parent, overridePerProjectPerDay).setForce(true).execute();
        OperationCow<Operation> operationCow = serviceUsageCow.operations().operationCow(createOperation);
        pollUntilSuccess(operationCow, Duration.ofSeconds(3), Duration.ofMinutes(5));
    } catch (IOException e) {
        throw new RetryException(e);
    }
    return StepResult.getStepResultSuccess();
}
Also used : QuotaOverride(com.google.api.services.serviceusage.v1beta1.model.QuotaOverride) Operation(com.google.api.services.serviceusage.v1beta1.model.Operation) IOException(java.io.IOException) RetryException(bio.terra.stairway.exception.RetryException) QuotaOverride(com.google.api.services.serviceusage.v1beta1.model.QuotaOverride)

Example 2 with Operation

use of com.google.api.services.cloudresourcemanager.v3.model.Operation in project terra-resource-buffer by DataBiosphere.

the class DeleteProjectStep method doStep.

@Override
public StepResult doStep(FlightContext flightContext) throws InterruptedException, RetryException {
    String projectId = flightContext.getInputParameters().get(CLOUD_RESOURCE_UID, CloudResourceUid.class).getGoogleProjectUid().getProjectId();
    try {
        Optional<Project> project = retrieveProject(rmCow, projectId);
        if (project.isEmpty() || isProjectDeleting(project.get())) {
            // Skip if project does not exist, or is being deleted. We know that the project is
            // created by Resource Buffer Service hence Resource Buffer Service should have owner
            // permission. So we assume 403 in this case
            // means that the project does not exist.
            logger.info("Project id: {} is deleted or being deleted", projectId);
            return StepResult.getStepResultSuccess();
        }
        OperationCow<?> operation = rmCow.operations().operationCow(rmCow.projects().delete(projectId).execute());
        pollUntilSuccess(operation, Duration.ofSeconds(5), Duration.ofMinutes(5));
    } catch (IOException e) {
        logger.info("Error when deleting GCP project", e);
        return new StepResult(StepStatus.STEP_RESULT_FAILURE_RETRY, e);
    }
    return StepResult.getStepResultSuccess();
}
Also used : Project(com.google.api.services.cloudresourcemanager.v3.model.Project) IOException(java.io.IOException) StepResult(bio.terra.stairway.StepResult)

Example 3 with Operation

use of com.google.api.services.cloudresourcemanager.v3.model.Operation in project FAAAST-Service by FraunhoferIOSB.

the class RequestHandlerManagerTest method testInvokeOperationAsyncRequest.

@Test
public void testInvokeOperationAsyncRequest() {
    CoreConfig coreConfig = CoreConfig.builder().build();
    Persistence persistence = mock(Persistence.class);
    MessageBus messageBus = mock(MessageBus.class);
    AssetConnectionManager assetConnectionManager = mock(AssetConnectionManager.class);
    AssetOperationProvider assetOperationProvider = mock(AssetOperationProvider.class);
    RequestHandlerManager manager = new RequestHandlerManager(coreConfig, persistence, messageBus, assetConnectionManager);
    Operation operation = getTestOperation();
    OperationHandle expectedOperationHandle = new OperationHandle.Builder().handleId("1").requestId("1").build();
    when(persistence.putOperationContext(any(), any(), any())).thenReturn(expectedOperationHandle);
    when(persistence.getOperationResult(any())).thenReturn(new OperationResult.Builder().requestId("1").build());
    when(assetConnectionManager.hasOperationProvider(any())).thenReturn(true);
    when(assetConnectionManager.getOperationProvider(any())).thenReturn(assetOperationProvider);
    InvokeOperationAsyncRequest invokeOperationAsyncRequest = new InvokeOperationAsyncRequest.Builder().requestId("1").id(new DefaultIdentifier.Builder().idType(IdentifierType.IRI).identifier("http://example.org").build()).inoutputArguments(operation.getInoutputVariables()).inputArguments(operation.getInputVariables()).build();
    InvokeOperationAsyncResponse response = manager.execute(invokeOperationAsyncRequest);
    OperationHandle actualOperationHandle = response.getPayload();
    Assert.assertEquals(expectedOperationHandle, actualOperationHandle);
}
Also used : CoreConfig(de.fraunhofer.iosb.ilt.faaast.service.config.CoreConfig) OperationResult(de.fraunhofer.iosb.ilt.faaast.service.model.api.operation.OperationResult) Operation(io.adminshell.aas.v3.model.Operation) DefaultOperation(io.adminshell.aas.v3.model.impl.DefaultOperation) OperationHandle(de.fraunhofer.iosb.ilt.faaast.service.model.api.operation.OperationHandle) Persistence(de.fraunhofer.iosb.ilt.faaast.service.persistence.Persistence) MessageBus(de.fraunhofer.iosb.ilt.faaast.service.messagebus.MessageBus) AssetConnectionManager(de.fraunhofer.iosb.ilt.faaast.service.assetconnection.AssetConnectionManager) RequestHandlerManager(de.fraunhofer.iosb.ilt.faaast.service.request.RequestHandlerManager) InvokeOperationAsyncRequest(de.fraunhofer.iosb.ilt.faaast.service.model.request.InvokeOperationAsyncRequest) AssetOperationProvider(de.fraunhofer.iosb.ilt.faaast.service.assetconnection.AssetOperationProvider) DefaultIdentifier(io.adminshell.aas.v3.model.impl.DefaultIdentifier) InvokeOperationAsyncResponse(de.fraunhofer.iosb.ilt.faaast.service.model.api.response.InvokeOperationAsyncResponse) Test(org.junit.Test)

Example 4 with Operation

use of com.google.api.services.cloudresourcemanager.v3.model.Operation in project terra-workspace-manager by DataBiosphere.

the class CompleteTransferOperationStep method getTransferOperationResult.

/**
 * Poll for completion of the named transfer operation and return the result.
 *
 * @param storageTransferService - svc to perform the transfer
 * @param transferJobName - name of job owning the transfer operation
 * @param operationName - server-generated name of running operation
 * @return StepResult indicating success or failure
 * @throws IOException
 * @throws InterruptedException
 */
private StepResult getTransferOperationResult(Storagetransfer storageTransferService, String transferJobName, String operationName) throws IOException, InterruptedException {
    // Now that we have an operation name, we can poll the operations endpoint for completion
    // information.
    int attempts = 0;
    Operation operation;
    do {
        operation = storageTransferService.transferOperations().get(operationName).execute();
        if (operation == null) {
            throw new RuntimeException(String.format("Failed to get transfer operation with name %s", operationName));
        } else if (operation.getDone() != null && operation.getDone()) {
            break;
        } else {
            // operation is not started or is in progress
            TimeUnit.MILLISECONDS.sleep(OPERATIONS_POLL_INTERVAL.toMillis());
            attempts++;
            logger.debug("Attempted to get transfer operation {} {} times", operationName, attempts);
        }
    } while (attempts < MAX_ATTEMPTS);
    if (MAX_ATTEMPTS <= attempts) {
        final String message = "Timed out waiting for operation result.";
        logger.info(message);
        return new StepResult(StepStatus.STEP_RESULT_FAILURE_FATAL, new StorageTransferServiceTimeoutException(message));
    }
    logger.info("Operation {} in transfer job {} has completed", operationName, transferJobName);
    // Inspect the completed operation for success
    if (operation.getError() != null) {
        logger.warn("Error in transfer operation {}: {}", operationName, operation.getError());
        final RuntimeException e = new RuntimeException("Failed transfer with error " + operation.getError().toString());
        return new StepResult(StepStatus.STEP_RESULT_FAILURE_FATAL, e);
    } else {
        logger.debug("Completed operation metadata: {}", operation.getMetadata());
        return StepResult.getStepResultSuccess();
    }
}
Also used : Operation(com.google.api.services.storagetransfer.v1.model.Operation) StepResult(bio.terra.stairway.StepResult) StorageTransferServiceTimeoutException(bio.terra.workspace.service.resource.controlled.exception.StorageTransferServiceTimeoutException)

Example 5 with Operation

use of com.google.api.services.cloudresourcemanager.v3.model.Operation in project google-cloud-intellij by GoogleCloudPlatform.

the class GoogleApiClientAppEngineAdminService method createApplication.

@Override
public Application createApplication(@NotNull String locationId, @NotNull final String projectId, @NotNull final Credential credential) throws IOException, GoogleApiException {
    Application arg = new Application();
    arg.setId(projectId);
    arg.setLocationId(locationId);
    Apps.Create createRequest = GoogleApiClientFactory.getInstance().getAppEngineApiClient(credential).apps().create(arg);
    Operation operation;
    try {
        // make the initial request to create the application
        operation = createRequest.execute();
        // poll for updates while the application is being created
        boolean done = false;
        while (!done) {
            try {
                Thread.sleep(CREATE_APPLICATION_POLLING_INTERVAL_MS);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
            operation = getOperation(projectId, operation.getName(), credential);
            if (operation.getDone() != null) {
                done = operation.getDone();
            }
        }
    } catch (GoogleJsonResponseException e) {
        throw GoogleApiException.from(e);
    }
    if (operation.getError() != null) {
        Status status = operation.getError();
        throw new GoogleApiException(status.getMessage(), status.getCode());
    } else {
        Application result = new Application();
        result.putAll(operation.getResponse());
        return result;
    }
}
Also used : Status(com.google.api.services.appengine.v1.model.Status) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) Operation(com.google.api.services.appengine.v1.model.Operation) Application(com.google.api.services.appengine.v1.model.Application) Apps(com.google.api.services.appengine.v1.Appengine.Apps)

Aggregations

IOException (java.io.IOException)16 Test (org.junit.Test)9 Operation (io.adminshell.aas.v3.model.Operation)7 StepResult (bio.terra.stairway.StepResult)6 GoogleJsonResponseException (com.google.api.client.googleapis.json.GoogleJsonResponseException)6 CloudHealthcare (com.google.api.services.healthcare.v1.CloudHealthcare)6 Operation (com.google.api.services.healthcare.v1.model.Operation)6 Project (com.google.api.services.cloudresourcemanager.v3.model.Project)5 Operation (com.google.api.services.notebooks.v1.model.Operation)5 MessageBus (de.fraunhofer.iosb.ilt.faaast.service.messagebus.MessageBus)5 Operation (com.google.api.services.appengine.v1.model.Operation)4 Create (com.google.api.services.container.v1beta1.Container.Projects.Locations.Clusters.Create)4 Operation (com.google.api.services.container.v1beta1.model.Operation)4 AIPlatformNotebooksCow (bio.terra.cloudres.google.notebooks.AIPlatformNotebooksCow)3 InstanceName (bio.terra.cloudres.google.notebooks.InstanceName)3 GcpCloudContext (bio.terra.workspace.service.workspace.model.GcpCloudContext)3 Status (com.google.api.services.appengine.v1.model.Status)3 Get (com.google.api.services.container.v1beta1.Container.Projects.Locations.Clusters.Get)3 Operation (com.google.api.services.serviceusage.v1beta1.model.Operation)3 AssetConnectionManager (de.fraunhofer.iosb.ilt.faaast.service.assetconnection.AssetConnectionManager)3