Search in sources :

Example 1 with QuotaOverride

use of com.google.api.services.serviceusage.v1beta1.model.QuotaOverride in project terra-resource-buffer by DataBiosphere.

the class CreateProjectFlightIntegrationTest method testCreateGoogleProject_createsConsumerOverride.

@Test
public void testCreateGoogleProject_createsConsumerOverride() throws Exception {
    FlightManager manager = new FlightManager(bufferDao, new StubSubmissionFlightFactory(LatchBeforeAssertResourceStep.class), stairwayComponent, transactionTemplate);
    GcpProjectConfig gcpProjectConfig = newBasicGcpConfig().serviceUsage(new ServiceUsage().bigQuery(new BigQueryQuotas().overrideBigQueryDailyUsageQuota(true).bigQueryDailyUsageQuotaOverrideValueMebibytes(new BigDecimal(CONSUMER_QUOTA_OVERRIDE_VALUE_MEBIBYTES))));
    Pool pool = preparePool(bufferDao, gcpProjectConfig);
    String flightId = manager.submitCreationFlight(pool).orElseThrow();
    ResourceId resourceId = extractResourceIdFromFlightState(blockUntilFlightComplete(stairwayComponent, flightId));
    Project project = assertProjectExists(resourceId);
    String projectNumber = getProjectNumberFromName(project.getName());
    String parent = String.format("projects/%s/services/bigquery.googleapis.com/consumerQuotaMetrics/" + "bigquery.googleapis.com%%2Fquota%%2Fquery%%2Fusage/limits/%%2Fd%%2Fproject", projectNumber);
    ServiceUsageCow.Services.ConsumerQuotaMetrics.Limits.ConsumerOverrides.List list = serviceUsageCow.services().consumerQuotaMetrics().limits().consumerOverrides().list(parent);
    ListConsumerOverridesResponse response = list.execute();
    assertEquals(1, response.getOverrides().size(), "single override expected");
    QuotaOverride quotaOverride = response.getOverrides().get(0);
    assertEquals(CONSUMER_QUOTA_OVERRIDE_VALUE_MEBIBYTES, quotaOverride.getOverrideValue());
}
Also used : ListConsumerOverridesResponse(com.google.api.services.serviceusage.v1beta1.model.ListConsumerOverridesResponse) FlightManager(bio.terra.buffer.service.resource.FlightManager) StubSubmissionFlightFactory(bio.terra.buffer.integration.IntegrationUtils.StubSubmissionFlightFactory) BigDecimal(java.math.BigDecimal) Project(com.google.api.services.cloudresourcemanager.v3.model.Project) QuotaOverride(com.google.api.services.serviceusage.v1beta1.model.QuotaOverride) ResourceId(bio.terra.buffer.common.ResourceId) Pool(bio.terra.buffer.common.Pool) IntegrationUtils.preparePool(bio.terra.buffer.integration.IntegrationUtils.preparePool) ServiceUsageCow(bio.terra.cloudres.google.serviceusage.ServiceUsageCow) Test(org.junit.jupiter.api.Test) BaseIntegrationTest(bio.terra.buffer.common.BaseIntegrationTest)

Example 2 with QuotaOverride

use of com.google.api.services.serviceusage.v1beta1.model.QuotaOverride in project terra-resource-buffer by DataBiosphere.

the class CreateConsumerDefinedQuotaForBigQueryDailyUsageStep method doStep.

/**
 * Apply a Consumer Quota Override for the BigQuery Query Usage Quota.
 */
@Override
public StepResult doStep(FlightContext context) throws InterruptedException, RetryException {
    Optional<Long> overrideValue = GoogleProjectConfigUtils.bigQueryDailyUsageOverrideValueMebibytes(gcpProjectConfig);
    if (overrideValue.isEmpty()) {
        // Do not apply any quota override
        return StepResult.getStepResultSuccess();
    }
    long projectNumber = Optional.ofNullable(context.getWorkingMap().get(GOOGLE_PROJECT_NUMBER, Long.class)).orElseThrow();
    QuotaOverride overridePerProjectPerDay = buildQuotaOverride(projectNumber, overrideValue.get());
    // parent format and other details obtained by hitting the endpoint
    // https://serviceusage.googleapis.com/v1beta1/projects/${PROJECT_NUMBER}/services/bigquery.googleapis.com/consumerQuotaMetrics
    String parent = String.format("projects/%d/services/bigquery.googleapis.com/consumerQuotaMetrics/" + "bigquery.googleapis.com%%2Fquota%%2Fquery%%2Fusage/limits/%%2Fd%%2Fproject", projectNumber);
    try {
        // We are decreasing the quota by more than 10%, so we must tell Service Usage to bypass the
        // check with the force flag.
        Operation createOperation = serviceUsageCow.services().consumerQuotaMetrics().limits().consumerOverrides().create(parent, overridePerProjectPerDay).setForce(true).execute();
        OperationCow<Operation> operationCow = serviceUsageCow.operations().operationCow(createOperation);
        pollUntilSuccess(operationCow, Duration.ofSeconds(3), Duration.ofMinutes(5));
    } catch (IOException e) {
        throw new RetryException(e);
    }
    return StepResult.getStepResultSuccess();
}
Also used : QuotaOverride(com.google.api.services.serviceusage.v1beta1.model.QuotaOverride) Operation(com.google.api.services.serviceusage.v1beta1.model.Operation) IOException(java.io.IOException) RetryException(bio.terra.stairway.exception.RetryException) QuotaOverride(com.google.api.services.serviceusage.v1beta1.model.QuotaOverride)

Aggregations

QuotaOverride (com.google.api.services.serviceusage.v1beta1.model.QuotaOverride)2 BaseIntegrationTest (bio.terra.buffer.common.BaseIntegrationTest)1 Pool (bio.terra.buffer.common.Pool)1 ResourceId (bio.terra.buffer.common.ResourceId)1 StubSubmissionFlightFactory (bio.terra.buffer.integration.IntegrationUtils.StubSubmissionFlightFactory)1 IntegrationUtils.preparePool (bio.terra.buffer.integration.IntegrationUtils.preparePool)1 FlightManager (bio.terra.buffer.service.resource.FlightManager)1 ServiceUsageCow (bio.terra.cloudres.google.serviceusage.ServiceUsageCow)1 RetryException (bio.terra.stairway.exception.RetryException)1 Project (com.google.api.services.cloudresourcemanager.v3.model.Project)1 ListConsumerOverridesResponse (com.google.api.services.serviceusage.v1beta1.model.ListConsumerOverridesResponse)1 Operation (com.google.api.services.serviceusage.v1beta1.model.Operation)1 IOException (java.io.IOException)1 BigDecimal (java.math.BigDecimal)1 Test (org.junit.jupiter.api.Test)1