Search in sources :

Example 6 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class VcfImportServiceIT method testDoImportVcfWithSamples.

private void testDoImportVcfWithSamples() {
    String entityTypeId = "variantsWithSamples";
    String fileName = entityTypeId + ".vcf";
    File file = getFile("/vcf/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of(entityTypeId, 10, entityTypeId + "Sample", 10), ImmutableSet.of(entityTypeId, entityTypeId + "Sample"));
    assertVariants(entityTypeId, true);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 7 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class EmxImportServiceIT method testDoImportEmxCsvZip.

private void testDoImportEmxCsvZip() {
    String fileName = "emx-csv.zip";
    File file = getFile("/csv/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of(CSV_HOSPITAL, 3, CSV_PATIENTS, 3), ImmutableSet.of(CSV_HOSPITAL, CSV_PATIENTS));
    verifyFirstAndLastRows(CSV_HOSPITAL, hospitalFirstRow, hospitalLastRow);
    verifyFirstAndLastRows(CSV_PATIENTS, patientsFirstRow, patientsLastRow);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 8 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class EmxImportServiceIT method testDoImportEmxTsvZip.

private void testDoImportEmxTsvZip() {
    String fileName = "emx-tsv.zip";
    File file = getFile("/tsv/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of(TSV_HOSPITAL, 3, TSV_PATIENTS, 3), ImmutableSet.of(TSV_HOSPITAL, TSV_PATIENTS));
    verifyFirstAndLastRows(TSV_HOSPITAL, hospitalFirstRow, hospitalLastRow);
    verifyFirstAndLastRows(TSV_PATIENTS, patientsFirstRow, patientsLastRow);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 9 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class EmxImportServiceIT method executeAddUpdateOrUpdateTest.

private void executeAddUpdateOrUpdateTest(File file, File addUpdateFile, Map<String, Integer> entityCountMap, Set<String> addedEntityTypes, Runnable entityValidationMethod) {
    FileRepositoryCollection addRepoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService addImportService = importServiceFactory.getImportService(file, addRepoCollection);
    addImportService.doImport(addRepoCollection, ADD, PACKAGE_DEFAULT);
    FileRepositoryCollection addUpdateRepoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(addUpdateFile);
    ImportService addUpdateImportService = importServiceFactory.getImportService(addUpdateFile, addUpdateRepoCollection);
    EntityImportReport importReport = addUpdateImportService.doImport(addUpdateRepoCollection, ADD_UPDATE_EXISTING, PACKAGE_DEFAULT);
    validateImportReport(importReport, entityCountMap, addedEntityTypes);
    entityValidationMethod.run();
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 10 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class EmxImportServiceIT method testDoImportAddEmx.

private void testDoImportAddEmx(File file, Map<String, Integer> entityCountMap, Set<String> addedEntityTypes, Runnable entityValidationMethod) {
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, entityCountMap, addedEntityTypes);
    entityValidationMethod.run();
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Aggregations

EntityImportReport (org.molgenis.data.importer.EntityImportReport)17 ImportService (org.molgenis.data.importer.ImportService)11 FileRepositoryCollection (org.molgenis.data.file.support.FileRepositoryCollection)10 File (java.io.File)9 EntityType (org.molgenis.data.meta.model.EntityType)4 Stream (java.util.stream.Stream)3 DefaultPackage (org.molgenis.data.meta.DefaultPackage)3 Package (org.molgenis.data.meta.model.Package)3 AbstractMockitoTest (org.molgenis.test.AbstractMockitoTest)3 Test (org.testng.annotations.Test)3 IOException (java.io.IOException)2 FileMeta (org.molgenis.data.file.model.FileMeta)2 AbstractRepository (org.molgenis.data.support.AbstractRepository)2 Transactional (org.springframework.transaction.annotation.Transactional)2 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 MolgenisDataException (org.molgenis.data.MolgenisDataException)1 RepositoryCollection (org.molgenis.data.RepositoryCollection)1 Attribute (org.molgenis.data.meta.model.Attribute)1 FileIngestJobExecution (org.molgenis.file.ingest.meta.FileIngestJobExecution)1