Search in sources :

Example 1 with DatasetImpl

use of org.globalbioticinteractions.dataset.DatasetImpl in project eol-globi-data by jhpoelen.

the class DatasetImporterForMetaTableIT method importAll.

public static void importAll(InteractionListener interactionListener, DatasetImporterForMetaTable.TableParserFactory tableFactory, String baseUrl, String resource) throws IOException, StudyImporterException {
    final InputStream inputStream = ResourceUtil.asInputStream(resource, inStream -> inStream);
    final JsonNode config = new ObjectMapper().readTree(inputStream);
    final Dataset dataset = new DatasetImpl("some/namespace", URI.create("http://example.com"), inStream -> inStream);
    dataset.setConfig(config);
    for (JsonNode table : DatasetImporterForMetaTable.collectTables(dataset)) {
        DatasetImporterForMetaTable.importTable(interactionListener, tableFactory, table, new DatasetImpl(null, URI.create(baseUrl), inStream -> inStream), null);
    }
}
Also used : TaxonUtil(org.eol.globi.service.TaxonUtil) CSVTSVUtil(org.eol.globi.util.CSVTSVUtil) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) URL(java.net.URL) InteractionListener(org.eol.globi.process.InteractionListener) ArrayList(java.util.ArrayList) ResourceUtil(org.eol.globi.util.ResourceUtil) Map(java.util.Map) Is.is(org.hamcrest.core.Is.is) JsonNode(com.fasterxml.jackson.databind.JsonNode) URI(java.net.URI) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) StringStartsWith.startsWith(org.hamcrest.core.StringStartsWith.startsWith) CoreMatchers.nullValue(org.hamcrest.CoreMatchers.nullValue) IsNot.not(org.hamcrest.core.IsNot.not) Assert.assertNotNull(org.junit.Assert.assertNotNull) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test) IOException(java.io.IOException) StandardCharsets(java.nio.charset.StandardCharsets) IOUtils(org.apache.commons.io.IOUtils) List(java.util.List) Dataset(org.globalbioticinteractions.dataset.Dataset) Matchers.containsString(org.hamcrest.Matchers.containsString) InputStream(java.io.InputStream) InputStream(java.io.InputStream) Dataset(org.globalbioticinteractions.dataset.Dataset) JsonNode(com.fasterxml.jackson.databind.JsonNode) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 2 with DatasetImpl

use of org.globalbioticinteractions.dataset.DatasetImpl in project eol-globi-data by jhpoelen.

the class DatasetImporterForPensoftIT method importStudy.

@Test
public void importStudy() throws StudyImporterException, URISyntaxException {
    final DatasetImporterForPensoft importer = new DatasetImporterForPensoft(new ParserFactoryLocal(), null);
    final Dataset dataset = new DatasetImpl("some/name", URI.create("some:uri"), in -> in);
    final ObjectNode objectNode = new ObjectMapper().createObjectNode();
    final URL resource = getClass().getResource("pensoft/annotated-tables-first-two.json");
    objectNode.put("url", resource.toURI().toString());
    objectNode.put("citation", "some dataset citation");
    dataset.setConfig(objectNode);
    List<Map<String, String>> links = new ArrayList<>();
    importer.setDataset(dataset);
    importer.setInteractionListener(new InteractionListener() {

        @Override
        public void on(Map<String, String> interaction) throws StudyImporterException {
            links.add(interaction);
        }
    });
    importer.importStudy();
    assertThat(links.size(), is(121));
    assertThat(links.get(0), hasEntry("Family Name", "Acanthaceae"));
    assertThat(links.get(0), hasEntry("Family Name_expanded_taxon_name", "Acanthaceae"));
    assertThat(links.get(0), hasEntry("Family Name_expanded_taxon_id", "http://openbiodiv.net/4B689A17-2541-4F5F-A896-6F0C2EEA3FB4"));
    assertThat(links.get(0), hasEntry("referenceUrl", "https://doi.org/10.3897/zookeys.306.5455"));
    assertThat(links.get(0), hasEntry("referenceDoi", "10.3897/zookeys.306.5455"));
    assertThat(links.get(0), hasEntry("referenceCitation", "Dewi Sartiami, Laurence A. Mound. 2013. Identification of the terebrantian thrips (Insecta, Thysanoptera) associated with cultivated plantsĀ inĀ Java, Indonesia. ZooKeys. https://doi.org/10.3897/zookeys.306.5455"));
}
Also used : ObjectNode(com.fasterxml.jackson.databind.node.ObjectNode) Dataset(org.globalbioticinteractions.dataset.Dataset) ArrayList(java.util.ArrayList) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) URL(java.net.URL) InteractionListener(org.eol.globi.process.InteractionListener) Map(java.util.Map) TreeMap(java.util.TreeMap) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Test(org.junit.Test) TestUtil.getResourceServiceTest(org.eol.globi.data.TestUtil.getResourceServiceTest)

Example 3 with DatasetImpl

use of org.globalbioticinteractions.dataset.DatasetImpl in project eol-globi-data by jhpoelen.

the class DatasetImporterForRSSTest method useUrlOnly.

@Test
public void useUrlOnly() throws IOException {
    DatasetImpl dataset = datasetFor("{ \"url\": \"bar\" }");
    assertThat(DatasetImporterForRSS.getRSSEndpoint(dataset), is("bar"));
}
Also used : DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) Test(org.junit.Test)

Example 4 with DatasetImpl

use of org.globalbioticinteractions.dataset.DatasetImpl in project eol-globi-data by jhpoelen.

the class DatasetImporterForRSSTest method useUrl.

@Test
public void useUrl() throws IOException {
    DatasetImpl dataset = datasetFor("{ \"url\": \"bar\", " + "\"resources\": { \"rss\": \"foo\" } " + "}");
    assertThat(DatasetImporterForRSS.getRSSEndpoint(dataset), is("bar"));
}
Also used : DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) Test(org.junit.Test)

Example 5 with DatasetImpl

use of org.globalbioticinteractions.dataset.DatasetImpl in project eol-globi-data by jhpoelen.

the class DatasetImporterForRSSTest method useRssNamedProperty.

@Test
public void useRssNamedProperty() throws IOException {
    DatasetImpl dataset = datasetFor("{ \"url\": \"bar\", " + "\"resources\": { \"rss\": \"foo\" } " + "}");
    assertThat(DatasetImporterForRSS.getRSSEndpoint(dataset), is("bar"));
}
Also used : DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) Test(org.junit.Test)

Aggregations

DatasetImpl (org.globalbioticinteractions.dataset.DatasetImpl)83 Test (org.junit.Test)73 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)50 URI (java.net.URI)32 JsonNode (com.fasterxml.jackson.databind.JsonNode)31 Dataset (org.globalbioticinteractions.dataset.Dataset)25 ArrayList (java.util.ArrayList)22 IOException (java.io.IOException)21 MatcherAssert.assertThat (org.hamcrest.MatcherAssert.assertThat)21 InteractionListener (org.eol.globi.process.InteractionListener)19 URL (java.net.URL)18 Map (java.util.Map)17 StudyNode (org.eol.globi.domain.StudyNode)17 HashMap (java.util.HashMap)16 Is.is (org.hamcrest.core.Is.is)16 List (java.util.List)15 StudyImpl (org.eol.globi.domain.StudyImpl)15 StringUtils (org.apache.commons.lang3.StringUtils)14 CoreMatchers.nullValue (org.hamcrest.CoreMatchers.nullValue)14 Is (org.hamcrest.core.Is)14