Search in sources :

Example 16 with SnapshotRequestModel

use of bio.terra.model.SnapshotRequestModel in project jade-data-repo by DataBiosphere.

the class SnapshotValidationTest method makeSnapshotAssetRequest.

// Generate a valid snapshot-by-asset request, we will tweak individual pieces to test validation below
public SnapshotRequestModel makeSnapshotAssetRequest() {
    SnapshotRequestAssetModel assetSpec = new SnapshotRequestAssetModel().assetName("asset").rootValues(Arrays.asList("sample 1", "sample 2", "sample 3"));
    SnapshotRequestContentsModel snapshotRequestContentsModel = new SnapshotRequestContentsModel().datasetName("dataset").mode(SnapshotRequestContentsModel.ModeEnum.BYASSET).assetSpec(assetSpec);
    return new SnapshotRequestModel().name("snapshot").description("snapshot description").addContentsItem(snapshotRequestContentsModel);
}
Also used : SnapshotRequestAssetModel(bio.terra.model.SnapshotRequestAssetModel) SnapshotRequestContentsModel(bio.terra.model.SnapshotRequestContentsModel) SnapshotRequestModel(bio.terra.model.SnapshotRequestModel)

Example 17 with SnapshotRequestModel

use of bio.terra.model.SnapshotRequestModel in project jade-data-repo by DataBiosphere.

the class DatasetIntegrationTest method testSoftDeleteNotInFullView.

@Test
public void testSoftDeleteNotInFullView() throws Exception {
    datasetId = ingestedDataset();
    // get row ids
    DatasetModel dataset = dataRepoFixtures.getDataset(steward(), datasetId);
    BigQuery bigQuery = BigQueryFixtures.getBigQuery(dataset.getDataProject(), stewardToken);
    List<String> participantRowIds = getRowIds(bigQuery, dataset, "participant", 3L);
    List<String> sampleRowIds = getRowIds(bigQuery, dataset, "sample", 2L);
    // swap in these row ids in the request
    SnapshotRequestModel requestModelAll = jsonLoader.loadObject("ingest-test-snapshot-fullviews.json", SnapshotRequestModel.class);
    requestModelAll.getContents().get(0).datasetName(dataset.getName());
    SnapshotSummaryModel snapshotSummaryAll = dataRepoFixtures.createSnapshotWithRequest(steward(), dataset.getName(), requestModelAll);
    snapshotIds.add(snapshotSummaryAll.getId());
    SnapshotModel snapshotAll = dataRepoFixtures.getSnapshot(steward(), snapshotSummaryAll.getId());
    // The steward is the custodian in this case, so is a reader in big query.
    BigQuery bigQueryAll = BigQueryFixtures.getBigQuery(snapshotAll.getDataProject(), stewardToken);
    assertSnapshotTableCount(bigQueryAll, snapshotAll, "participant", 5L);
    assertSnapshotTableCount(bigQueryAll, snapshotAll, "sample", 7L);
    // write them to GCS
    String participantPath = writeListToScratch("softDel", participantRowIds);
    String samplePath = writeListToScratch("softDel", sampleRowIds);
    // build the deletion request with pointers to the two files with row ids to soft delete
    List<DataDeletionTableModel> dataDeletionTableModels = Arrays.asList(deletionTableFile("participant", participantPath), deletionTableFile("sample", samplePath));
    DataDeletionRequest request = dataDeletionRequest().tables(dataDeletionTableModels);
    // send off the soft delete request
    dataRepoFixtures.deleteData(steward(), datasetId, request);
    // make sure the new counts make sense
    assertTableCount(bigQuery, dataset, "participant", 2L);
    assertTableCount(bigQuery, dataset, "sample", 5L);
    // make full views snapshot
    SnapshotRequestModel requestModelLess = jsonLoader.loadObject("ingest-test-snapshot-fullviews.json", SnapshotRequestModel.class);
    requestModelLess.getContents().get(0).datasetName(dataset.getName());
    SnapshotSummaryModel snapshotSummaryLess = dataRepoFixtures.createSnapshotWithRequest(steward(), dataset.getName(), requestModelLess);
    snapshotIds.add(snapshotSummaryLess.getId());
    SnapshotModel snapshotLess = dataRepoFixtures.getSnapshot(steward(), snapshotSummaryLess.getId());
    BigQuery bigQueryLess = BigQueryFixtures.getBigQuery(snapshotLess.getDataProject(), stewardToken);
    // make sure the old counts stayed the same
    assertSnapshotTableCount(bigQueryAll, snapshotAll, "participant", 5L);
    assertSnapshotTableCount(bigQueryAll, snapshotAll, "sample", 7L);
    // make sure the new counts make sense
    assertSnapshotTableCount(bigQueryLess, snapshotLess, "participant", 2L);
    assertSnapshotTableCount(bigQueryLess, snapshotLess, "sample", 5L);
}
Also used : BigQuery(com.google.cloud.bigquery.BigQuery) SnapshotSummaryModel(bio.terra.model.SnapshotSummaryModel) DataDeletionRequest(bio.terra.model.DataDeletionRequest) EnumerateDatasetModel(bio.terra.model.EnumerateDatasetModel) DatasetModel(bio.terra.model.DatasetModel) SnapshotRequestModel(bio.terra.model.SnapshotRequestModel) SnapshotModel(bio.terra.model.SnapshotModel) DataDeletionTableModel(bio.terra.model.DataDeletionTableModel) SpringBootTest(org.springframework.boot.test.context.SpringBootTest) Test(org.junit.Test)

Example 18 with SnapshotRequestModel

use of bio.terra.model.SnapshotRequestModel in project jade-data-repo by DataBiosphere.

the class ConnectedOperations method launchCreateSnapshot.

public MockHttpServletResponse launchCreateSnapshot(DatasetSummaryModel datasetSummaryModel, String resourcePath, String infix) throws Exception {
    SnapshotRequestModel snapshotRequest = jsonLoader.loadObject(resourcePath, SnapshotRequestModel.class);
    String snapshotName = Names.randomizeNameInfix(snapshotRequest.getName(), infix);
    return launchCreateSnapshotName(datasetSummaryModel, snapshotRequest, snapshotName);
}
Also used : SnapshotRequestModel(bio.terra.model.SnapshotRequestModel)

Aggregations

SnapshotRequestModel (bio.terra.model.SnapshotRequestModel)18 Test (org.junit.Test)11 SpringBootTest (org.springframework.boot.test.context.SpringBootTest)11 SnapshotSummaryModel (bio.terra.model.SnapshotSummaryModel)9 SnapshotModel (bio.terra.model.SnapshotModel)8 EnumerateSnapshotModel (bio.terra.model.EnumerateSnapshotModel)7 MockHttpServletResponse (org.springframework.mock.web.MockHttpServletResponse)7 SnapshotRequestContentsModel (bio.terra.model.SnapshotRequestContentsModel)6 DatasetSummaryModel (bio.terra.model.DatasetSummaryModel)5 CoreMatchers.containsString (org.hamcrest.CoreMatchers.containsString)5 DatasetModel (bio.terra.model.DatasetModel)4 ErrorModel (bio.terra.model.ErrorModel)4 BigQuery (com.google.cloud.bigquery.BigQuery)4 JsonLoader (bio.terra.common.fixtures.JsonLoader)3 IngestRequestModel (bio.terra.model.IngestRequestModel)3 BigQueryProject (bio.terra.service.tabulardata.google.BigQueryProject)3 ConnectedTestConfiguration (bio.terra.app.configuration.ConnectedTestConfiguration)2 PDAO_PREFIX (bio.terra.common.PdaoConstant.PDAO_PREFIX)2 TestUtils (bio.terra.common.TestUtils)2 Connected (bio.terra.common.category.Connected)2