Search in sources :

Example 6 with ApiException

use of bio.terra.workspace.client.ApiException in project terra-workspace-manager by DataBiosphere.

the class EnumerateJobs method doUserJourney.

@Override
public void doUserJourney(TestUserSpecification testUser, WorkspaceApi workspaceApi) throws Exception {
    // Case 1: fetch all
    EnumerateJobsResult fetchall = alpha1Api.enumerateJobs(getWorkspaceId(), null, null, null, null, null, null);
    logResult("fetchall", fetchall);
    // TODO: [PF-1281] we need another type of filtering to be able to do better validation of the
    // result return.
    // Case 2: fetch by pages
    String pageToken = null;
    for (int pageCount = 1; true; pageCount++) {
        EnumerateJobsResult page = alpha1Api.enumerateJobs(getWorkspaceId(), PAGE_SIZE, pageToken, null, null, null, null);
        logResult("page " + pageCount, page);
        assertThat("Not more than page size items returned", page.getResults().size(), lessThanOrEqualTo(PAGE_SIZE));
        pageToken = page.getPageToken();
        if (page.getResults().size() == 0) {
            break;
        }
    }
    // Case 4: filter by resource type
    EnumerateJobsResult buckets = alpha1Api.enumerateJobs(getWorkspaceId(), /*limit=*/
    null, /*pageToken=*/
    null, ResourceType.GCS_BUCKET, /*stewardship=*/
    null, /*name=*/
    null, /*jobState=*/
    null);
    logResult("buckets", buckets);
    for (EnumeratedJob job : buckets.getResults()) {
        assertThat("Job is a bucket", job.getResourceType(), equalTo(ResourceType.GCS_BUCKET));
        assertNotNull(job.getResourceAttributes().getGcpGcsBucket(), "Bucket resource present");
        assertThat("Resource is a bucket", job.getMetadata().getResourceType(), equalTo(ResourceType.GCS_BUCKET));
    }
    // Case 5: filter by stewardship type
    EnumerateJobsResult controlled = alpha1Api.enumerateJobs(getWorkspaceId(), null, null, null, StewardshipType.CONTROLLED, null, null);
    logResult("controlled", controlled);
    for (EnumeratedJob job : controlled.getResults()) {
        ResourceMetadata metadata = job.getMetadata();
        assertNotNull(metadata, "Resource has metadata");
        assertThat("Resource is controlled", metadata.getStewardshipType(), equalTo(StewardshipType.CONTROLLED));
    }
    // Case 6: filter by resource and stewardship
    EnumerateJobsResult controlledBuckets = alpha1Api.enumerateJobs(getWorkspaceId(), null, null, ResourceType.GCS_BUCKET, StewardshipType.CONTROLLED, null, null);
    logResult("controlledBuckets", controlledBuckets);
    for (EnumeratedJob job : controlledBuckets.getResults()) {
        ResourceMetadata metadata = job.getMetadata();
        assertNotNull(metadata, "Resource has metadata");
        assertThat("Resource is controlled", metadata.getStewardshipType(), equalTo(StewardshipType.CONTROLLED));
        assertThat("Resource is a bucket", metadata.getResourceType(), equalTo(ResourceType.GCS_BUCKET));
    }
    // Case 7: validate error on invalid pagination params
    ApiException invalidPaginationException = assertThrows(ApiException.class, () -> alpha1Api.enumerateJobs(getWorkspaceId(), -5, null, null, null, null, null));
    assertThat(invalidPaginationException.getMessage(), containsString("Invalid pagination"));
    invalidPaginationException = assertThrows(ApiException.class, () -> alpha1Api.enumerateJobs(getWorkspaceId(), 22, "junktoken", null, null, null, null));
    assertThat(invalidPaginationException.getMessage(), containsString("Invalid page token"));
}
Also used : EnumerateJobsResult(bio.terra.workspace.model.EnumerateJobsResult) Matchers.containsString(org.hamcrest.Matchers.containsString) ResourceMetadata(bio.terra.workspace.model.ResourceMetadata) EnumeratedJob(bio.terra.workspace.model.EnumeratedJob) ApiException(bio.terra.workspace.client.ApiException)

Example 7 with ApiException

use of bio.terra.workspace.client.ApiException in project terra-workspace-manager by DataBiosphere.

the class EnumerateResources method doUserJourney.

@Override
public void doUserJourney(TestUserSpecification testUser, WorkspaceApi workspaceApi) throws Exception {
    // Add second user to the workspace as a reader
    workspaceApi.grantRole(new GrantRoleRequestBody().memberEmail(workspaceReader.userEmail), getWorkspaceId(), IamRole.READER);
    // Case 1: fetch all
    ResourceList enumList = ownerResourceApi.enumerateResources(getWorkspaceId(), 0, RESOURCE_COUNT, null, null);
    logResult("fetchall", enumList);
    // Make sure we got all of the expected ids
    matchFullResourceList(enumList.getResources());
    // Repeat case 1 as the workspace reader.
    // As this is the first operation after modifying workspace IAM groups, retry here to compensate
    // for the delay in GCP IAM propagation.
    ResourceList readerEnumList = ClientTestUtils.getWithRetryOnException(() -> readerResourceApi.enumerateResources(getWorkspaceId(), 0, RESOURCE_COUNT, null, null));
    logResult("fetchall reader", readerEnumList);
    matchFullResourceList(readerEnumList.getResources());
    // Case 2: fetch by pages
    ResourceList page1List = ownerResourceApi.enumerateResources(getWorkspaceId(), 0, PAGE_SIZE, null, null);
    logResult("page1", page1List);
    assertThat(page1List.getResources().size(), equalTo(PAGE_SIZE));
    ResourceList page2List = ownerResourceApi.enumerateResources(getWorkspaceId(), PAGE_SIZE, PAGE_SIZE, null, null);
    logResult("page2", page2List);
    assertThat(page2List.getResources().size(), equalTo(PAGE_SIZE));
    ResourceList page3List = ownerResourceApi.enumerateResources(getWorkspaceId(), 2 * PAGE_SIZE, PAGE_SIZE, null, null);
    logResult("page3", page3List);
    assertThat(page3List.getResources().size(), lessThan(PAGE_SIZE));
    List<ResourceDescription> descriptionList = new ArrayList<>();
    descriptionList.addAll(page1List.getResources());
    descriptionList.addAll(page2List.getResources());
    descriptionList.addAll(page3List.getResources());
    matchFullResourceList(descriptionList);
    // Case 3: no results if offset is too high
    ResourceList enumEmptyList = ownerResourceApi.enumerateResources(getWorkspaceId(), 10 * PAGE_SIZE, PAGE_SIZE, null, null);
    assertThat(enumEmptyList.getResources().size(), equalTo(0));
    // Case 4: filter by resource type
    ResourceList buckets = ownerResourceApi.enumerateResources(getWorkspaceId(), 0, RESOURCE_COUNT, ResourceType.GCS_BUCKET, null);
    logResult("buckets", buckets);
    long expectedBuckets = resourceList.stream().filter(m -> m.getResourceType() == ResourceType.GCS_BUCKET).count();
    logger.info("Counted {} buckets created", expectedBuckets);
    // Note - assertThat exits out on an int -> long compare, so just don't do that.
    long actualBuckets = buckets.getResources().size();
    assertThat(actualBuckets, equalTo(expectedBuckets));
    // Case 5: filter by stewardship type
    ResourceList referencedList = ownerResourceApi.enumerateResources(getWorkspaceId(), 0, RESOURCE_COUNT, null, StewardshipType.REFERENCED);
    logResult("referenced", referencedList);
    long expectedReferenced = resourceList.stream().filter(m -> m.getStewardshipType() == StewardshipType.REFERENCED).count();
    logger.info("Counted {} referenced created", expectedReferenced);
    long actualReferenced = referencedList.getResources().size();
    assertThat(actualReferenced, equalTo(expectedReferenced));
    // Case 6: filter by resource and stewardship
    ResourceList controlledBucketList = ownerResourceApi.enumerateResources(getWorkspaceId(), 0, RESOURCE_COUNT, ResourceType.GCS_BUCKET, StewardshipType.CONTROLLED);
    logResult("controlledBucket", controlledBucketList);
    long expectedControlledBuckets = resourceList.stream().filter(m -> (m.getStewardshipType() == StewardshipType.CONTROLLED && m.getResourceType() == ResourceType.GCS_BUCKET)).count();
    logger.info("Counted {} controlled buckets created", expectedControlledBuckets);
    long actualControlledBuckets = controlledBucketList.getResources().size();
    assertThat(actualControlledBuckets, equalTo(expectedControlledBuckets));
    // Case 7: validate error on invalid pagination params
    ApiException invalidPaginationException = assertThrows(ApiException.class, () -> ownerResourceApi.enumerateResources(getWorkspaceId(), -11, 2, ResourceType.GCS_BUCKET, StewardshipType.CONTROLLED));
    assertThat(invalidPaginationException.getMessage(), containsString("Invalid pagination"));
    invalidPaginationException = assertThrows(ApiException.class, () -> ownerResourceApi.enumerateResources(getWorkspaceId(), 0, 0, ResourceType.GCS_BUCKET, StewardshipType.CONTROLLED));
    assertThat(invalidPaginationException.getMessage(), containsString("Invalid pagination"));
}
Also used : Assertions.assertThrows(org.junit.jupiter.api.Assertions.assertThrows) ControlledGcpResourceApi(bio.terra.workspace.api.ControlledGcpResourceApi) ClientTestUtils(scripts.utils.ClientTestUtils) CloudContextMaker(scripts.utils.CloudContextMaker) LoggerFactory(org.slf4j.LoggerFactory) ResourceList(bio.terra.workspace.model.ResourceList) ArrayList(java.util.ArrayList) WorkspaceAllocateTestScriptBase(scripts.utils.WorkspaceAllocateTestScriptBase) Matchers.lessThan(org.hamcrest.Matchers.lessThan) MultiResourcesUtils(scripts.utils.MultiResourcesUtils) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) ReferencedGcpResourceApi(bio.terra.workspace.api.ReferencedGcpResourceApi) StewardshipType(bio.terra.workspace.model.StewardshipType) ControlledResourceMetadata(bio.terra.workspace.model.ControlledResourceMetadata) WorkspaceApi(bio.terra.workspace.api.WorkspaceApi) ResourceMetadata(bio.terra.workspace.model.ResourceMetadata) ResourceType(bio.terra.workspace.model.ResourceType) Logger(org.slf4j.Logger) ApiException(bio.terra.workspace.client.ApiException) ApiClient(bio.terra.workspace.client.ApiClient) UUID(java.util.UUID) Collectors(java.util.stream.Collectors) IamRole(bio.terra.workspace.model.IamRole) List(java.util.List) TestUserSpecification(bio.terra.testrunner.runner.config.TestUserSpecification) Matchers.equalTo(org.hamcrest.Matchers.equalTo) GrantRoleRequestBody(bio.terra.workspace.model.GrantRoleRequestBody) ResourceDescription(bio.terra.workspace.model.ResourceDescription) Matchers.greaterThan(org.hamcrest.Matchers.greaterThan) Matchers.containsString(org.hamcrest.Matchers.containsString) ResourceApi(bio.terra.workspace.api.ResourceApi) ResourceList(bio.terra.workspace.model.ResourceList) GrantRoleRequestBody(bio.terra.workspace.model.GrantRoleRequestBody) ResourceDescription(bio.terra.workspace.model.ResourceDescription) ArrayList(java.util.ArrayList) ApiException(bio.terra.workspace.client.ApiException)

Example 8 with ApiException

use of bio.terra.workspace.client.ApiException in project terra-workspace-manager by DataBiosphere.

the class PrivateControlledAiNotebookInstanceLifecycle method doUserJourney.

@Override
@SuppressFBWarnings(value = "DLS_DEAD_LOCAL_STORE")
protected void doUserJourney(TestUserSpecification testUser, WorkspaceApi workspaceApi) throws Exception {
    CloudContextMaker.createGcpCloudContext(getWorkspaceId(), workspaceApi);
    workspaceApi.grantRole(new GrantRoleRequestBody().memberEmail(resourceUser.userEmail), getWorkspaceId(), IamRole.WRITER);
    workspaceApi.grantRole(new GrantRoleRequestBody().memberEmail(otherWorkspaceUser.userEmail), getWorkspaceId(), IamRole.WRITER);
    ControlledGcpResourceApi resourceUserApi = ClientTestUtils.getControlledGcpResourceClient(resourceUser, server);
    CreatedControlledGcpAiNotebookInstanceResult creationResult = NotebookUtils.makeControlledNotebookUserPrivate(getWorkspaceId(), instanceId, /*location=*/
    null, resourceUserApi);
    UUID resourceId = creationResult.getAiNotebookInstance().getMetadata().getResourceId();
    GcpAiNotebookInstanceResource resource = resourceUserApi.getAiNotebookInstance(getWorkspaceId(), resourceId);
    assertEquals(instanceId, resource.getAttributes().getInstanceId(), "Notebook instance id is correct in GET response from WSM");
    assertEquals(instanceId, creationResult.getAiNotebookInstance().getAttributes().getInstanceId(), "Notebook instance id is correct in create response from WSM");
    assertEquals(resourceUser.userEmail, resource.getMetadata().getControlledResourceMetadata().getPrivateResourceUser().getUserName(), "User is the private user of the notebook");
    assertEquals("us-central1-a", resource.getAttributes().getLocation(), "The notebook uses the default location because location is not specified.");
    // Any workspace user should be able to enumerate notebooks, even though they can't
    // read or write them.
    ResourceApi otherUserApi = ClientTestUtils.getResourceClient(otherWorkspaceUser, server);
    ResourceList notebookList = otherUserApi.enumerateResources(getWorkspaceId(), 0, 5, ResourceType.AI_NOTEBOOK, StewardshipType.CONTROLLED);
    assertEquals(1, notebookList.getResources().size());
    MultiResourcesUtils.assertResourceType(ResourceType.AI_NOTEBOOK, notebookList);
    createAControlledAiNotebookInstanceWithoutSpecifiedInstanceId_validInstanceIdIsGenerated(resourceUserApi);
    createAControlledAiNotebookInstanceWithoutSpecifiedInstanceId_specifyLocation(resourceUserApi);
    String instanceName = String.format("projects/%s/locations/%s/instances/%s", resource.getAttributes().getProjectId(), resource.getAttributes().getLocation(), resource.getAttributes().getInstanceId());
    AIPlatformNotebooks userNotebooks = ClientTestUtils.getAIPlatformNotebooksClient(resourceUser);
    assertTrue(NotebookUtils.userHasProxyAccess(creationResult, resourceUser, resource.getAttributes().getProjectId()), "Private resource user has access to their notebook");
    assertFalse(NotebookUtils.userHasProxyAccess(creationResult, otherWorkspaceUser, resource.getAttributes().getProjectId()), "Other workspace user does not have access to a private notebook");
    // The user should be able to stop their notebook.
    userNotebooks.projects().locations().instances().stop(instanceName, new StopInstanceRequest());
    // The user should not be able to directly delete their notebook.
    GoogleJsonResponseException directDeleteForbidden = assertThrows(GoogleJsonResponseException.class, () -> userNotebooks.projects().locations().instances().delete(instanceName).execute());
    assertEquals(HttpStatus.SC_FORBIDDEN, directDeleteForbidden.getStatusCode(), "User may not delete notebook directly on GCP");
    // Update the AI Notebook through WSM.
    var newName = "new-instance-notebook-name";
    var newDescription = "new description for the new instance notebook name";
    var newMetadata = ImmutableMap.of("foo", "bar", "count", "3");
    GcpAiNotebookInstanceResource updatedResource = resourceUserApi.updateAiNotebookInstance(new UpdateControlledGcpAiNotebookInstanceRequestBody().description(newDescription).name(newName).updateParameters(new GcpAiNotebookUpdateParameters().metadata(newMetadata)), getWorkspaceId(), resourceId);
    assertEquals(newName, updatedResource.getMetadata().getName());
    assertEquals(newDescription, updatedResource.getMetadata().getDescription());
    var metadata = userNotebooks.projects().locations().instances().get(instanceName).execute().getMetadata();
    for (var entrySet : newMetadata.entrySet()) {
        assertThat(metadata, IsMapContaining.hasEntry(entrySet.getKey(), entrySet.getValue()));
    }
    // Delete the AI Notebook through WSM.
    DeleteControlledGcpAiNotebookInstanceResult deleteResult = resourceUserApi.deleteAiNotebookInstance(new DeleteControlledGcpAiNotebookInstanceRequest().jobControl(new JobControl().id(UUID.randomUUID().toString())), getWorkspaceId(), resourceId);
    String deleteJobId = deleteResult.getJobReport().getId();
    deleteResult = ClientTestUtils.pollWhileRunning(deleteResult, () -> resourceUserApi.getDeleteAiNotebookInstanceResult(getWorkspaceId(), deleteJobId), DeleteControlledGcpAiNotebookInstanceResult::getJobReport, Duration.ofSeconds(10));
    ClientTestUtils.assertJobSuccess("delete ai notebook", deleteResult.getJobReport(), deleteResult.getErrorReport());
    // Verify the notebook was deleted from WSM metadata.
    ApiException notebookIsMissing = assertThrows(ApiException.class, () -> resourceUserApi.getAiNotebookInstance(getWorkspaceId(), resourceId), "Notebook is deleted from WSM");
    assertEquals(HttpStatus.SC_NOT_FOUND, notebookIsMissing.getCode(), "Error from WSM is 404");
    // Verify the notebook was deleted from GCP.
    GoogleJsonResponseException notebookNotFound = assertThrows(GoogleJsonResponseException.class, () -> userNotebooks.projects().locations().instances().get(instanceName).execute(), "Notebook is deleted from GCP");
    // GCP may respond with either 403 or 404 depending on how quickly this is called after deleting
    // the notebook. Either response is valid in this case.
    assertThat("Error from GCP is 403 or 404", notebookNotFound.getStatusCode(), anyOf(equalTo(HttpStatus.SC_NOT_FOUND), equalTo(HttpStatus.SC_FORBIDDEN)));
}
Also used : UpdateControlledGcpAiNotebookInstanceRequestBody(bio.terra.workspace.model.UpdateControlledGcpAiNotebookInstanceRequestBody) GrantRoleRequestBody(bio.terra.workspace.model.GrantRoleRequestBody) CreatedControlledGcpAiNotebookInstanceResult(bio.terra.workspace.model.CreatedControlledGcpAiNotebookInstanceResult) DeleteControlledGcpAiNotebookInstanceRequest(bio.terra.workspace.model.DeleteControlledGcpAiNotebookInstanceRequest) GcpAiNotebookUpdateParameters(bio.terra.workspace.model.GcpAiNotebookUpdateParameters) DeleteControlledGcpAiNotebookInstanceResult(bio.terra.workspace.model.DeleteControlledGcpAiNotebookInstanceResult) JobControl(bio.terra.workspace.model.JobControl) AIPlatformNotebooks(com.google.api.services.notebooks.v1.AIPlatformNotebooks) GcpAiNotebookInstanceResource(bio.terra.workspace.model.GcpAiNotebookInstanceResource) StopInstanceRequest(com.google.api.services.notebooks.v1.model.StopInstanceRequest) GoogleJsonResponseException(com.google.api.client.googleapis.json.GoogleJsonResponseException) ControlledGcpResourceApi(bio.terra.workspace.api.ControlledGcpResourceApi) ResourceApi(bio.terra.workspace.api.ResourceApi) ResourceList(bio.terra.workspace.model.ResourceList) ControlledGcpResourceApi(bio.terra.workspace.api.ControlledGcpResourceApi) UUID(java.util.UUID) ApiException(bio.terra.workspace.client.ApiException) SuppressFBWarnings(edu.umd.cs.findbugs.annotations.SuppressFBWarnings)

Aggregations

ApiException (bio.terra.workspace.client.ApiException)8 ControlledGcpResourceApi (bio.terra.workspace.api.ControlledGcpResourceApi)6 ResourceApi (bio.terra.workspace.api.ResourceApi)6 ResourceList (bio.terra.workspace.model.ResourceList)6 UUID (java.util.UUID)6 GrantRoleRequestBody (bio.terra.workspace.model.GrantRoleRequestBody)5 CreatedControlledGcpGcsBucket (bio.terra.workspace.model.CreatedControlledGcpGcsBucket)3 Matchers.containsString (org.hamcrest.Matchers.containsString)3 ApiClient (bio.terra.workspace.client.ApiClient)2 GcpGcsBucketResource (bio.terra.workspace.model.GcpGcsBucketResource)2 ResourceMetadata (bio.terra.workspace.model.ResourceMetadata)2 Bucket (com.google.cloud.storage.Bucket)2 Storage (com.google.cloud.storage.Storage)2 GcsBucketAccessTester (scripts.utils.GcsBucketAccessTester)2 TestUserSpecification (bio.terra.testrunner.runner.config.TestUserSpecification)1 ReferencedGcpResourceApi (bio.terra.workspace.api.ReferencedGcpResourceApi)1 WorkspaceApi (bio.terra.workspace.api.WorkspaceApi)1 WorkspaceApplicationApi (bio.terra.workspace.api.WorkspaceApplicationApi)1 ClonedControlledGcpBigQueryDataset (bio.terra.workspace.model.ClonedControlledGcpBigQueryDataset)1 ClonedControlledGcpGcsBucket (bio.terra.workspace.model.ClonedControlledGcpGcsBucket)1