Search in sources :

Example 51 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImporterFactoryImplIT method createSIAD.

@Test
public void createSIAD() throws StudyImporterException, DatasetRegistryException {
    final DatasetRegistryGitHubRemote datasetFinderGitHubRemote = new DatasetRegistryGitHubRemote(inStream -> inStream);
    DatasetImporter importer = importerFor(datasetFinderGitHubRemote, "globalbioticinteractions/siad");
    assertThat(importer, is(notNullValue()));
    Dataset dataset = ((BaseDatasetImporter) importer).getDataset();
    final JsonNode config = dataset.getConfig();
    assertThat(config, is(notNullValue()));
    assertThat(dataset.getOrDefault(DatasetConstant.SHOULD_RESOLVE_REFERENCES, "donald"), is("false"));
}
Also used : DatasetRegistryGitHubRemote(org.globalbioticinteractions.dataset.DatasetRegistryGitHubRemote) Dataset(org.globalbioticinteractions.dataset.Dataset) JsonNode(com.fasterxml.jackson.databind.JsonNode) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) DatasetImporter(org.eol.globi.data.DatasetImporter) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) Test(org.junit.Test)

Example 52 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImporterFactoryImplIT method defaultTSVImporterCached.

@Test
public void defaultTSVImporterCached() throws StudyImporterException, DatasetRegistryException, IOException {
    final DatasetRegistry datasetRegistry = new DatasetRegistryWithCache(new DatasetRegistryGitHubArchive(inStream -> inStream), dataset -> CacheUtil.cacheFor(dataset.getNamespace(), "target/datasets", inStream -> inStream));
    DatasetImporter importer = getTemplateImporter(datasetRegistry, "globalbioticinteractions/template-dataset");
    DatasetImporterForTSV importerTSV = (DatasetImporterForTSV) importer;
    assertThat(importerTSV.getBaseUrl(), startsWith("https://github.com/globalbioticinteractions/template-dataset/"));
    assertThat(importerTSV.getDataset().retrieve(URI.create("globi.json")), is(notNullValue()));
}
Also used : CacheUtil(org.globalbioticinteractions.cache.CacheUtil) DatasetImporterForGoMexSI2(org.eol.globi.data.DatasetImporterForGoMexSI2) DatasetRegistryGitHubRemote(org.globalbioticinteractions.dataset.DatasetRegistryGitHubRemote) IsInstanceOf.instanceOf(org.hamcrest.core.IsInstanceOf.instanceOf) DatasetImporterForHurlbert(org.eol.globi.data.DatasetImporterForHurlbert) DatasetConstant(org.globalbioticinteractions.dataset.DatasetConstant) DatasetImporterForMetaTable(org.eol.globi.data.DatasetImporterForMetaTable) DatasetImporterForHafner(org.eol.globi.data.DatasetImporterForHafner) DatasetImporterForSzoboszlai(org.eol.globi.data.DatasetImporterForSzoboszlai) StudyImporterException(org.eol.globi.data.StudyImporterException) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) Is.is(org.hamcrest.core.Is.is) JsonNode(com.fasterxml.jackson.databind.JsonNode) DatasetImporter(org.eol.globi.data.DatasetImporter) DatasetImporterForJSONLD(org.eol.globi.data.DatasetImporterForJSONLD) URI(java.net.URI) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) StringStartsWith.startsWith(org.hamcrest.core.StringStartsWith.startsWith) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) DatasetRegistryZenodo(org.globalbioticinteractions.dataset.DatasetRegistryZenodo) DatasetRegistryException(org.globalbioticinteractions.dataset.DatasetRegistryException) Assert.assertNotNull(org.junit.Assert.assertNotNull) Test(org.junit.Test) IOException(java.io.IOException) DatasetImporterForCoetzer(org.eol.globi.data.DatasetImporterForCoetzer) DatasetRegistryGitHubArchive(org.globalbioticinteractions.dataset.DatasetRegistryGitHubArchive) DatasetImporterForPlanque(org.eol.globi.data.DatasetImporterForPlanque) DatasetImporterForTSV(org.eol.globi.data.DatasetImporterForTSV) DatasetFactory(org.globalbioticinteractions.dataset.DatasetFactory) Dataset(org.globalbioticinteractions.dataset.Dataset) DatasetImporterForRSS(org.eol.globi.data.DatasetImporterForRSS) IsNull.notNullValue(org.hamcrest.core.IsNull.notNullValue) DatasetImporterForWood(org.eol.globi.data.DatasetImporterForWood) DatasetRegistryWithCache(org.globalbioticinteractions.dataset.DatasetRegistryWithCache) Matchers.containsString(org.hamcrest.Matchers.containsString) InputStream(java.io.InputStream) DatasetImporterForTSV(org.eol.globi.data.DatasetImporterForTSV) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) DatasetRegistryWithCache(org.globalbioticinteractions.dataset.DatasetRegistryWithCache) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) DatasetImporter(org.eol.globi.data.DatasetImporter) DatasetRegistryGitHubArchive(org.globalbioticinteractions.dataset.DatasetRegistryGitHubArchive) Test(org.junit.Test)

Example 53 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImporterFactoryImplIT method getTemplateImporter.

DatasetImporter getTemplateImporter(DatasetRegistry datasetRegistry, String repo) throws DatasetRegistryException, StudyImporterException {
    Dataset dataset = new DatasetFactory(datasetRegistry).datasetFor(repo);
    DatasetImporter importer = new StudyImporterFactoryImpl(null).createImporter(dataset);
    assertThat(importer, is(notNullValue()));
    assertThat(importer, is(instanceOf(DatasetImporterForTSV.class)));
    return importer;
}
Also used : Dataset(org.globalbioticinteractions.dataset.Dataset) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) DatasetImporter(org.eol.globi.data.DatasetImporter) DatasetFactory(org.globalbioticinteractions.dataset.DatasetFactory)

Example 54 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class InteractionListenerImplTest method processMessageWithTranslatedInteractionType.

@Test
public void processMessageWithTranslatedInteractionType() throws StudyImporterException, IOException {
    ResourceService resourceService = Mockito.mock(ResourceService.class);
    when(resourceService.retrieve(URI.create("interaction_types_ignored.csv"))).thenReturn(IOUtils.toInputStream("provided_interaction_type_id\nshouldBeIgnored", StandardCharsets.UTF_8)).thenReturn(IOUtils.toInputStream("provided_interaction_type_id\nshouldBeIgnored", StandardCharsets.UTF_8));
    when(resourceService.retrieve(URI.create("interaction_types_mapping.csv"))).thenReturn(IOUtils.toInputStream(getTestMap(), StandardCharsets.UTF_8));
    Dataset dataset = new DatasetImpl("bla", resourceService, URI.create("foo:bar"));
    InteractionListenerImpl interactionListener = new InteractionListenerImpl(nodeFactory, null, null, dataset);
    HashMap<String, String> interaction = new HashMap<>();
    interaction.put(SOURCE_TAXON_NAME, "sourceName");
    interaction.put(DatasetImporterForTSV.INTERACTION_TYPE_NAME, "shouldBeMapped");
    interaction.put(TARGET_TAXON_NAME, "targetName");
    interaction.put(REFERENCE_ID, "citation");
    assertStudyCount(0L);
    interactionListener.on(interaction);
    assertStudyCount(1L);
}
Also used : HashMap(java.util.HashMap) Dataset(org.globalbioticinteractions.dataset.Dataset) ResourceService(org.eol.globi.service.ResourceService) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) Test(org.junit.Test)

Example 55 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImplTest method useCitationFromTableDefinition.

@Test
public void useCitationFromTableDefinition() throws IOException {
    Dataset dataset = new DatasetImpl("some/namespace", URI.create("some:uri"), inStream -> inStream);
    dataset.setConfig(new ObjectMapper().readTree(getClass().getResourceAsStream("/org/eol/globi/data/test-meta-globi-default-external-schema.json")));
    assertThat(dataset.getCitation(), is("Seltzer, Carrie; Wysocki, William; Palacios, Melissa; Eickhoff, Anna; Pilla, Hannah; Aungst, Jordan; Mercer, Aaron; Quicho, Jamie; Voss, Neil; Xu, Man; J. Ndangalasi, Henry; C. Lovett, Jon; J. Cordeiro, Norbert (2015): Plant-animal interactions from Africa. figshare. https://dx.doi.org/10.6084/m9.figshare.1526128"));
}
Also used : Dataset(org.globalbioticinteractions.dataset.Dataset) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test)

Aggregations

Dataset (org.globalbioticinteractions.dataset.Dataset)68 Test (org.junit.Test)46 DatasetImpl (org.globalbioticinteractions.dataset.DatasetImpl)25 URI (java.net.URI)20 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)17 JsonNode (com.fasterxml.jackson.databind.JsonNode)14 Matchers.containsString (org.hamcrest.Matchers.containsString)14 IOException (java.io.IOException)11 MatcherAssert.assertThat (org.hamcrest.MatcherAssert.assertThat)10 InputStream (java.io.InputStream)9 Is.is (org.hamcrest.core.Is.is)9 StudyNode (org.eol.globi.domain.StudyNode)8 Node (org.neo4j.graphdb.Node)8 URL (java.net.URL)7 ArrayList (java.util.ArrayList)7 StringStartsWith.startsWith (org.hamcrest.core.StringStartsWith.startsWith)7 Assert.assertNotNull (org.junit.Assert.assertNotNull)7 BaseDatasetImporter (org.eol.globi.data.BaseDatasetImporter)6 DatasetImporter (org.eol.globi.data.DatasetImporter)6 DatasetNode (org.eol.globi.domain.DatasetNode)6