Search in sources :

Example 1 with EnumerateSnapshotModel

use of bio.terra.model.EnumerateSnapshotModel in project jade-data-repo by DataBiosphere.

the class RepositoryApiController method enumerateSnapshots.

@Override
public ResponseEntity<EnumerateSnapshotModel> enumerateSnapshots(@Valid @RequestParam(value = "offset", required = false, defaultValue = "0") Integer offset, @Valid @RequestParam(value = "limit", required = false, defaultValue = "10") Integer limit, @Valid @RequestParam(value = "sort", required = false, defaultValue = "created_date") String sort, @Valid @RequestParam(value = "direction", required = false, defaultValue = "asc") String direction, @Valid @RequestParam(value = "filter", required = false) String filter) {
    ControllerUtils.validateEnumerateParams(offset, limit, sort, direction);
    List<UUID> resources = iamService.listAuthorizedResources(getAuthenticatedInfo(), IamResourceType.DATASNAPSHOT);
    EnumerateSnapshotModel edm = snapshotService.enumerateSnapshots(offset, limit, sort, direction, filter, resources);
    return new ResponseEntity<>(edm, HttpStatus.OK);
}
Also used : ResponseEntity(org.springframework.http.ResponseEntity) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) UUID(java.util.UUID)

Example 2 with EnumerateSnapshotModel

use of bio.terra.model.EnumerateSnapshotModel in project jade-data-repo by DataBiosphere.

the class SnapshotConnectedTest method testEnumeration.

@Test
public void testEnumeration() throws Exception {
    SnapshotRequestModel snapshotRequest = makeSnapshotTestRequest(datasetSummary, "snapshot-test-snapshot.json");
    // Other unit tests exercise the array bounds, so here we don't fuss with that here.
    // Just make sure we get the same snapshot summary that we made.
    List<SnapshotSummaryModel> snapshotList = new ArrayList<>();
    for (int i = 0; i < 5; i++) {
        MockHttpServletResponse response = performCreateSnapshot(snapshotRequest, "_en_");
        SnapshotSummaryModel summaryModel = validateSnapshotCreated(snapshotRequest, response);
        snapshotList.add(summaryModel);
    }
    List<UUID> snapshotIds = snapshotList.stream().map(snapshot -> UUID.fromString(snapshot.getId())).collect(Collectors.toList());
    when(samService.listAuthorizedResources(any(), any())).thenReturn(snapshotIds);
    EnumerateSnapshotModel enumResponse = enumerateTestSnapshots();
    List<SnapshotSummaryModel> enumeratedArray = enumResponse.getItems();
    assertThat("total is correct", enumResponse.getTotal(), equalTo(5));
    // The enumeratedArray may contain more snapshots than just the set we created,
    // but ours should be in order in the enumeration. So we do a merge waiting until we match
    // by id and then comparing contents.
    int compareIndex = 0;
    for (SnapshotSummaryModel anEnumeratedSnapshot : enumeratedArray) {
        if (anEnumeratedSnapshot.getId().equals(snapshotList.get(compareIndex).getId())) {
            assertThat("MetadataEnumeration summary matches create summary", anEnumeratedSnapshot, equalTo(snapshotList.get(compareIndex)));
            compareIndex++;
        }
    }
    assertThat("we found all snapshots", compareIndex, equalTo(5));
    for (int i = 0; i < 5; i++) {
        connectedOperations.deleteTestSnapshot(enumeratedArray.get(i).getId());
    }
}
Also used : BillingProfileModel(bio.terra.model.BillingProfileModel) FieldValue(com.google.cloud.bigquery.FieldValue) CoreMatchers(org.hamcrest.CoreMatchers) SnapshotModel(bio.terra.model.SnapshotModel) Autowired(org.springframework.beans.factory.annotation.Autowired) ActiveProfiles(org.springframework.test.context.ActiveProfiles) StorageOptions(com.google.cloud.storage.StorageOptions) StringUtils(org.apache.commons.lang3.StringUtils) BigQuery(com.google.cloud.bigquery.BigQuery) Assert.assertThat(org.junit.Assert.assertThat) RelationshipModel(bio.terra.model.RelationshipModel) FieldValueList(com.google.cloud.bigquery.FieldValueList) After(org.junit.After) DrsId(bio.terra.service.filedata.DrsId) TableResult(com.google.cloud.bigquery.TableResult) SpringRunner(org.springframework.test.context.junit4.SpringRunner) URI(java.net.URI) DataLocationService(bio.terra.service.resourcemanagement.DataLocationService) MockBean(org.springframework.boot.test.mock.mockito.MockBean) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) PDAO_PREFIX(bio.terra.common.PdaoConstant.PDAO_PREFIX) MediaType(org.springframework.http.MediaType) QueryJobConfiguration(com.google.cloud.bigquery.QueryJobConfiguration) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) UUID(java.util.UUID) Category(org.junit.experimental.categories.Category) SnapshotRequestContentsModel(bio.terra.model.SnapshotRequestContentsModel) Collectors(java.util.stream.Collectors) DatasetSummaryModel(bio.terra.model.DatasetSummaryModel) StandardCharsets(java.nio.charset.StandardCharsets) ConfigEnum(bio.terra.service.configuration.ConfigEnum) IOUtils(org.apache.commons.io.IOUtils) List(java.util.List) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) ST(org.stringtemplate.v4.ST) Assert.assertFalse(org.junit.Assert.assertFalse) TestUtils(bio.terra.common.TestUtils) Matchers.equalTo(org.hamcrest.Matchers.equalTo) Optional(java.util.Optional) Storage(com.google.cloud.storage.Storage) ConfigurationService(bio.terra.service.configuration.ConfigurationService) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) GoogleResourceConfiguration(bio.terra.service.resourcemanagement.google.GoogleResourceConfiguration) ConnectedOperations(bio.terra.common.fixtures.ConnectedOperations) Names(bio.terra.common.fixtures.Names) DRSObject(bio.terra.model.DRSObject) RunWith(org.junit.runner.RunWith) BigQueryProject(bio.terra.service.tabulardata.google.BigQueryProject) DrsIdService(bio.terra.service.filedata.DrsIdService) TableModel(bio.terra.model.TableModel) MockMvcResultMatchers.content(org.springframework.test.web.servlet.result.MockMvcResultMatchers.content) MockHttpServletResponse(org.springframework.mock.web.MockHttpServletResponse) ArrayList(java.util.ArrayList) MockMvc(org.springframework.test.web.servlet.MockMvc) SnapshotRequestModel(bio.terra.model.SnapshotRequestModel) MockMvcRequestBuilders.delete(org.springframework.test.web.servlet.request.MockMvcRequestBuilders.delete) MockMvcRequestBuilders.post(org.springframework.test.web.servlet.request.MockMvcRequestBuilders.post) MvcResult(org.springframework.test.web.servlet.MvcResult) JsonLoader(bio.terra.common.fixtures.JsonLoader) ProfileDao(bio.terra.service.resourcemanagement.ProfileDao) ConnectedTestConfiguration(bio.terra.app.configuration.ConnectedTestConfiguration) StringStartsWith.startsWith(org.hamcrest.core.StringStartsWith.startsWith) Before(org.junit.Before) BlobInfo(com.google.cloud.storage.BlobInfo) FileLoadModel(bio.terra.model.FileLoadModel) FileModel(bio.terra.model.FileModel) Assert.assertNotNull(org.junit.Assert.assertNotNull) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Connected(bio.terra.common.category.Connected) Assert.assertTrue(org.junit.Assert.assertTrue) DeleteResponseModel(bio.terra.model.DeleteResponseModel) Test(org.junit.Test) IamProviderInterface(bio.terra.service.iam.IamProviderInterface) Mockito.when(org.mockito.Mockito.when) SnapshotSourceModel(bio.terra.model.SnapshotSourceModel) DatasetDao(bio.terra.service.dataset.DatasetDao) TimeUnit(java.util.concurrent.TimeUnit) ErrorModel(bio.terra.model.ErrorModel) HttpStatus(org.springframework.http.HttpStatus) IngestRequestModel(bio.terra.model.IngestRequestModel) AutoConfigureMockMvc(org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc) Assert.assertNull(org.junit.Assert.assertNull) SnapshotSummaryModel(bio.terra.model.SnapshotSummaryModel) MockMvcRequestBuilders.get(org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get) Assert.assertEquals(org.junit.Assert.assertEquals) SnapshotSummaryModel(bio.terra.model.SnapshotSummaryModel) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) ArrayList(java.util.ArrayList) UUID(java.util.UUID) SnapshotRequestModel(bio.terra.model.SnapshotRequestModel) MockHttpServletResponse(org.springframework.mock.web.MockHttpServletResponse) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) Test(org.junit.Test)

Example 3 with EnumerateSnapshotModel

use of bio.terra.model.EnumerateSnapshotModel in project jade-data-repo by DataBiosphere.

the class SnapshotConnectedTest method testExcludeLockedFromSnapshotLookups.

@Test
public void testExcludeLockedFromSnapshotLookups() throws Exception {
    // create a snapshot
    SnapshotSummaryModel snapshotSummary = connectedOperations.createSnapshot(datasetSummary, "snapshot-test-snapshot.json", "_d2_");
    // check that the snapshot metadata row is unlocked
    String exclusiveLock = snapshotDao.getExclusiveLockState(UUID.fromString(snapshotSummary.getId()));
    assertNull("snapshot row is unlocked", exclusiveLock);
    // retrieve the snapshot and check that it finds it
    SnapshotModel snapshotModel = connectedOperations.getSnapshot(snapshotSummary.getId());
    assertEquals("Lookup unlocked snapshot succeeds", snapshotSummary.getName(), snapshotModel.getName());
    // enumerate snapshots and check that this snapshot is included in the set
    EnumerateSnapshotModel enumerateSnapshotModelModel = connectedOperations.enumerateSnapshots(snapshotSummary.getName());
    List<SnapshotSummaryModel> enumeratedSnapshots = enumerateSnapshotModelModel.getItems();
    boolean foundSnapshotWithMatchingId = false;
    for (SnapshotSummaryModel enumeratedSnapshot : enumeratedSnapshots) {
        if (enumeratedSnapshot.getId().equals(snapshotSummary.getId())) {
            foundSnapshotWithMatchingId = true;
            break;
        }
    }
    assertTrue("Unlocked included in enumeration", foundSnapshotWithMatchingId);
    // NO ASSERTS inside the block below where hang is enabled to reduce chance of failing before disabling the hang
    // ====================================================
    // enable hang in DeleteSnapshotPrimaryDataStep
    configService.setFault(ConfigEnum.SNAPSHOT_DELETE_LOCK_CONFLICT_STOP_FAULT.name(), true);
    // kick off a request to delete the snapshot. this should hang before unlocking the snapshot object.
    MvcResult deleteResult = mvc.perform(delete("/api/repository/v1/snapshots/" + snapshotSummary.getId())).andReturn();
    // give the flight time to launch
    TimeUnit.SECONDS.sleep(5);
    // note: asserts are below outside the hang block
    exclusiveLock = snapshotDao.getExclusiveLockState(UUID.fromString(snapshotSummary.getId()));
    // retrieve the snapshot and check that it returns not found
    // note: asserts are below outside the hang block
    MvcResult retrieveResult = mvc.perform(get("/api/repository/v1/snapshots/" + snapshotSummary.getId())).andReturn();
    // enumerate snapshots and check that this snapshot is not included in the set
    // note: asserts are below outside the hang block
    MvcResult enumerateResult = connectedOperations.enumerateSnapshotsRaw(snapshotSummary.getName());
    // disable hang in DeleteSnapshotPrimaryDataStep
    configService.setFault(ConfigEnum.SNAPSHOT_DELETE_LOCK_CONFLICT_CONTINUE_FAULT.name(), true);
    // ====================================================
    // check that the snapshot metadata row has an exclusive lock after kicking off the delete
    assertNotNull("snapshot row is exclusively locked", exclusiveLock);
    // check that the retrieve snapshot returned not found
    connectedOperations.handleFailureCase(retrieveResult.getResponse(), HttpStatus.NOT_FOUND);
    // check that the enumerate snapshots returned successfully and that this snapshot is not included in the set
    enumerateSnapshotModelModel = connectedOperations.handleSuccessCase(enumerateResult.getResponse(), EnumerateSnapshotModel.class);
    enumeratedSnapshots = enumerateSnapshotModelModel.getItems();
    foundSnapshotWithMatchingId = false;
    for (SnapshotSummaryModel enumeratedSnapshot : enumeratedSnapshots) {
        if (enumeratedSnapshot.getId().equals(snapshotSummary.getId())) {
            foundSnapshotWithMatchingId = true;
            break;
        }
    }
    assertFalse("Exclusively locked not included in enumeration", foundSnapshotWithMatchingId);
    // check the response from the delete request
    MockHttpServletResponse deleteResponse = connectedOperations.validateJobModelAndWait(deleteResult);
    DeleteResponseModel deleteResponseModel = connectedOperations.handleSuccessCase(deleteResponse, DeleteResponseModel.class);
    assertEquals("Snapshot delete returned successfully", DeleteResponseModel.ObjectStateEnum.DELETED, deleteResponseModel.getObjectState());
    // try to fetch the snapshot again and confirm nothing is returned
    connectedOperations.getSnapshotExpectError(snapshotSummary.getId(), HttpStatus.NOT_FOUND);
}
Also used : SnapshotSummaryModel(bio.terra.model.SnapshotSummaryModel) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) CoreMatchers.containsString(org.hamcrest.CoreMatchers.containsString) MvcResult(org.springframework.test.web.servlet.MvcResult) SnapshotModel(bio.terra.model.SnapshotModel) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) MockHttpServletResponse(org.springframework.mock.web.MockHttpServletResponse) DeleteResponseModel(bio.terra.model.DeleteResponseModel) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) Test(org.junit.Test)

Example 4 with EnumerateSnapshotModel

use of bio.terra.model.EnumerateSnapshotModel in project jade-data-repo by DataBiosphere.

the class SnapshotConnectedTest method enumerateTestSnapshots.

private EnumerateSnapshotModel enumerateTestSnapshots() throws Exception {
    MvcResult result = mvc.perform(get("/api/repository/v1/snapshots?offset=0&limit=1000")).andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8_VALUE)).andReturn();
    MockHttpServletResponse response = result.getResponse();
    EnumerateSnapshotModel summary = TestUtils.mapFromJson(response.getContentAsString(), EnumerateSnapshotModel.class);
    return summary;
}
Also used : EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) MvcResult(org.springframework.test.web.servlet.MvcResult) MockHttpServletResponse(org.springframework.mock.web.MockHttpServletResponse)

Example 5 with EnumerateSnapshotModel

use of bio.terra.model.EnumerateSnapshotModel in project jade-data-repo by DataBiosphere.

the class SnapshotTest method snapshotUnauthorizedPermissionsTest.

@Test
public void snapshotUnauthorizedPermissionsTest() throws Exception {
    DataRepoResponse<JobModel> createSnapLaunchResp = dataRepoFixtures.createSnapshotLaunch(reader(), datasetSummaryModel, "ingest-test-snapshot.json");
    assertThat("Reader is not authorized to create a dataSnapshot", createSnapLaunchResp.getStatusCode(), equalTo(HttpStatus.UNAUTHORIZED));
    SnapshotSummaryModel snapshotSummary = dataRepoFixtures.createSnapshot(custodian(), datasetSummaryModel, "ingest-test-snapshot.json");
    createdSnapshotIds.add(snapshotSummary.getId());
    DataRepoResponse<JobModel> deleteSnapResp = dataRepoFixtures.deleteSnapshotLaunch(reader(), snapshotSummary.getId());
    assertThat("Reader is not authorized to delete a dataSnapshot", deleteSnapResp.getStatusCode(), equalTo(HttpStatus.UNAUTHORIZED));
    DataRepoResponse<SnapshotModel> getSnapResp = dataRepoFixtures.getSnapshotRaw(discoverer(), snapshotSummary.getId());
    assertThat("Discoverer is not authorized to get a dataSnapshot", getSnapResp.getStatusCode(), equalTo(HttpStatus.UNAUTHORIZED));
    EnumerateSnapshotModel enumSnap = dataRepoFixtures.enumerateSnapshots(discoverer());
    assertThat("Discoverer does not have access to dataSnapshots", enumSnap.getTotal(), equalTo(0));
}
Also used : SnapshotSummaryModel(bio.terra.model.SnapshotSummaryModel) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) JobModel(bio.terra.model.JobModel) SnapshotModel(bio.terra.model.SnapshotModel) EnumerateSnapshotModel(bio.terra.model.EnumerateSnapshotModel) Test(org.junit.Test) SpringBootTest(org.springframework.boot.test.context.SpringBootTest)

Aggregations

EnumerateSnapshotModel (bio.terra.model.EnumerateSnapshotModel)6 SnapshotSummaryModel (bio.terra.model.SnapshotSummaryModel)4 SnapshotModel (bio.terra.model.SnapshotModel)3 Test (org.junit.Test)3 SpringBootTest (org.springframework.boot.test.context.SpringBootTest)3 MockHttpServletResponse (org.springframework.mock.web.MockHttpServletResponse)3 MvcResult (org.springframework.test.web.servlet.MvcResult)3 DeleteResponseModel (bio.terra.model.DeleteResponseModel)2 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)2 ConnectedTestConfiguration (bio.terra.app.configuration.ConnectedTestConfiguration)1 PDAO_PREFIX (bio.terra.common.PdaoConstant.PDAO_PREFIX)1 TestUtils (bio.terra.common.TestUtils)1 Connected (bio.terra.common.category.Connected)1 ConnectedOperations (bio.terra.common.fixtures.ConnectedOperations)1 JsonLoader (bio.terra.common.fixtures.JsonLoader)1 Names (bio.terra.common.fixtures.Names)1 BillingProfileModel (bio.terra.model.BillingProfileModel)1 DRSObject (bio.terra.model.DRSObject)1 DatasetSummaryModel (bio.terra.model.DatasetSummaryModel)1 ErrorModel (bio.terra.model.ErrorModel)1