Search in sources :

Example 1 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class OntologyImportService method doImport.

@Override
@Transactional
public EntityImportReport doImport(RepositoryCollection source, DatabaseAction databaseAction, String packageId) {
    if (databaseAction != DatabaseAction.ADD) {
        throw new IllegalArgumentException("Only ADD is supported");
    }
    EntityImportReport report = new EntityImportReport();
    for (String entityTypeId : source.getEntityTypeIds()) {
        try (Repository<Entity> sourceRepository = source.getRepository(entityTypeId)) {
            Repository<Entity> targetRepository = dataService.getRepository(entityTypeId);
            Integer count = targetRepository.add(asStream(sourceRepository));
            report.addEntityCount(entityTypeId, count);
        } catch (IOException e) {
            LOG.error("", e);
            throw new MolgenisDataException(e);
        }
    }
    return report;
}
Also used : IOException(java.io.IOException) EntityImportReport(org.molgenis.data.importer.EntityImportReport) Transactional(org.springframework.transaction.annotation.Transactional)

Example 2 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class VcfImporterService method doImport.

@Transactional
@Override
public EntityImportReport doImport(RepositoryCollection source, DatabaseAction databaseAction, String packageId) {
    if (databaseAction != DatabaseAction.ADD)
        throw new IllegalArgumentException("Only ADD is supported");
    List<EntityType> addedEntities = Lists.newArrayList();
    EntityImportReport report;
    Iterator<String> it = source.getEntityTypeIds().iterator();
    if (it.hasNext()) {
        try (Repository<Entity> repo = source.getRepository(it.next())) {
            report = importVcf(repo, addedEntities, packageId);
        } catch (IOException e) {
            LOG.error("", e);
            throw new MolgenisDataException(e);
        }
    } else {
        report = new EntityImportReport();
    }
    return report;
}
Also used : EntityType(org.molgenis.data.meta.model.EntityType) IOException(java.io.IOException) EntityImportReport(org.molgenis.data.importer.EntityImportReport) Transactional(org.springframework.transaction.annotation.Transactional)

Example 3 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class OntologyImportServiceIT method testDoImportOwl.

private void testDoImportOwl() {
    String fileName = "ontology-small.owl.zip";
    File file = getFile("/owl/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of("sys_ont_OntologyTermDynamicAnnotation", 4, "sys_ont_OntologyTermSynonym", 9, "sys_ont_OntologyTermNodePath", 10, "sys_ont_Ontology", 1, "sys_ont_OntologyTerm", 9), emptySet());
    // Verify the import as system as we need write permissions on sys tables to carry out the verification
    runAsSystem(this::verifyOwlAsSystem);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 4 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class VcfImportServiceIT method testDoImportVcfWithoutSamples.

private void testDoImportVcfWithoutSamples() {
    String entityTypeId = "variantsWithoutSamples";
    String fileName = entityTypeId + ".vcf";
    File file = getFile("/vcf/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of(entityTypeId, 10), ImmutableSet.of(entityTypeId));
    assertVariants(entityTypeId, false);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Example 5 with EntityImportReport

use of org.molgenis.data.importer.EntityImportReport in project molgenis by molgenis.

the class VcfImportServiceIT method testDoImportVcfGzWithSamples.

private void testDoImportVcfGzWithSamples() {
    String entityTypeId = "variantsWithSamplesGz";
    String fileName = entityTypeId + ".vcf.gz";
    File file = getFile("/vcf/" + fileName);
    FileRepositoryCollection repoCollection = fileRepositoryCollectionFactory.createFileRepositoryCollection(file);
    ImportService importService = importServiceFactory.getImportService(file, repoCollection);
    EntityImportReport importReport = importService.doImport(repoCollection, ADD, PACKAGE_DEFAULT);
    validateImportReport(importReport, ImmutableMap.of("variantsWithSamplesGz", 10, "variantsWithSamplesGzSample", 10), ImmutableSet.of("variantsWithSamplesGz", "variantsWithSamplesGzSample"));
    assertVariants(entityTypeId, true);
}
Also used : ImportService(org.molgenis.data.importer.ImportService) FileRepositoryCollection(org.molgenis.data.file.support.FileRepositoryCollection) File(java.io.File) EntityImportReport(org.molgenis.data.importer.EntityImportReport)

Aggregations

EntityImportReport (org.molgenis.data.importer.EntityImportReport)17 ImportService (org.molgenis.data.importer.ImportService)11 FileRepositoryCollection (org.molgenis.data.file.support.FileRepositoryCollection)10 File (java.io.File)9 EntityType (org.molgenis.data.meta.model.EntityType)4 Stream (java.util.stream.Stream)3 DefaultPackage (org.molgenis.data.meta.DefaultPackage)3 Package (org.molgenis.data.meta.model.Package)3 AbstractMockitoTest (org.molgenis.test.AbstractMockitoTest)3 Test (org.testng.annotations.Test)3 IOException (java.io.IOException)2 FileMeta (org.molgenis.data.file.model.FileMeta)2 AbstractRepository (org.molgenis.data.support.AbstractRepository)2 Transactional (org.springframework.transaction.annotation.Transactional)2 AmazonS3 (com.amazonaws.services.s3.AmazonS3)1 AtomicInteger (java.util.concurrent.atomic.AtomicInteger)1 MolgenisDataException (org.molgenis.data.MolgenisDataException)1 RepositoryCollection (org.molgenis.data.RepositoryCollection)1 Attribute (org.molgenis.data.meta.model.Attribute)1 FileIngestJobExecution (org.molgenis.file.ingest.meta.FileIngestJobExecution)1