Search in sources :

Example 16 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImporterFactoryImplIT method defaultTSVImporterCachedZenodo.

@Test
public void defaultTSVImporterCachedZenodo() throws StudyImporterException, DatasetRegistryException {
    final DatasetRegistry datasetRegistry = new DatasetRegistryWithCache(new DatasetRegistryZenodo(inStream -> inStream), dataset -> CacheUtil.cacheFor(dataset.getNamespace(), "target/datasets", inStream -> inStream));
    DatasetImporter importer = getTemplateImporter(datasetRegistry, "globalbioticinteractions/template-dataset");
    DatasetImporterForTSV importerTSV = (DatasetImporterForTSV) importer;
    assertThat(importerTSV.getSourceCitation(), containsString("doi.org"));
}
Also used : CacheUtil(org.globalbioticinteractions.cache.CacheUtil) DatasetImporterForGoMexSI2(org.eol.globi.data.DatasetImporterForGoMexSI2) DatasetRegistryGitHubRemote(org.globalbioticinteractions.dataset.DatasetRegistryGitHubRemote) IsInstanceOf.instanceOf(org.hamcrest.core.IsInstanceOf.instanceOf) DatasetImporterForHurlbert(org.eol.globi.data.DatasetImporterForHurlbert) DatasetConstant(org.globalbioticinteractions.dataset.DatasetConstant) DatasetImporterForMetaTable(org.eol.globi.data.DatasetImporterForMetaTable) DatasetImporterForHafner(org.eol.globi.data.DatasetImporterForHafner) DatasetImporterForSzoboszlai(org.eol.globi.data.DatasetImporterForSzoboszlai) StudyImporterException(org.eol.globi.data.StudyImporterException) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) Is.is(org.hamcrest.core.Is.is) JsonNode(com.fasterxml.jackson.databind.JsonNode) DatasetImporter(org.eol.globi.data.DatasetImporter) DatasetImporterForJSONLD(org.eol.globi.data.DatasetImporterForJSONLD) URI(java.net.URI) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) StringStartsWith.startsWith(org.hamcrest.core.StringStartsWith.startsWith) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) DatasetRegistryZenodo(org.globalbioticinteractions.dataset.DatasetRegistryZenodo) DatasetRegistryException(org.globalbioticinteractions.dataset.DatasetRegistryException) Assert.assertNotNull(org.junit.Assert.assertNotNull) Test(org.junit.Test) IOException(java.io.IOException) DatasetImporterForCoetzer(org.eol.globi.data.DatasetImporterForCoetzer) DatasetRegistryGitHubArchive(org.globalbioticinteractions.dataset.DatasetRegistryGitHubArchive) DatasetImporterForPlanque(org.eol.globi.data.DatasetImporterForPlanque) DatasetImporterForTSV(org.eol.globi.data.DatasetImporterForTSV) DatasetFactory(org.globalbioticinteractions.dataset.DatasetFactory) Dataset(org.globalbioticinteractions.dataset.Dataset) DatasetImporterForRSS(org.eol.globi.data.DatasetImporterForRSS) IsNull.notNullValue(org.hamcrest.core.IsNull.notNullValue) DatasetImporterForWood(org.eol.globi.data.DatasetImporterForWood) DatasetRegistryWithCache(org.globalbioticinteractions.dataset.DatasetRegistryWithCache) Matchers.containsString(org.hamcrest.Matchers.containsString) InputStream(java.io.InputStream) DatasetImporterForTSV(org.eol.globi.data.DatasetImporterForTSV) DatasetRegistryZenodo(org.globalbioticinteractions.dataset.DatasetRegistryZenodo) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) DatasetRegistryWithCache(org.globalbioticinteractions.dataset.DatasetRegistryWithCache) BaseDatasetImporter(org.eol.globi.data.BaseDatasetImporter) DatasetImporter(org.eol.globi.data.DatasetImporter) Test(org.junit.Test)

Example 17 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImporterForRegistryTest method filteredDatasets.

@Test
public void filteredDatasets() throws StudyImporterException {
    DatasetImporterForRegistry importer = new DatasetImporterForRegistry(null, null, new DatasetRegistry() {

        @Override
        public Collection<String> findNamespaces() throws DatasetRegistryException {
            return Collections.singletonList("some/namespace");
        }

        @Override
        public Dataset datasetFor(String namespace) throws DatasetRegistryException {
            DatasetImpl dataset = new DatasetImpl("some/namespace", URI.create("some:uri"), in -> in) {

                @Override
                public InputStream retrieve(URI resource) throws IOException {
                    if (!StringUtils.endsWith(resource.toString(), "globi.json")) {
                        throw new IOException();
                    }
                    return IOUtils.toInputStream("{\"some\":\"thing\"}", StandardCharsets.UTF_8);
                }
            };
            return dataset;
        }
    });
    importer.setDatasetFilter(x -> false);
    importer.importStudy();
}
Also used : DatasetRegistryException(org.globalbioticinteractions.dataset.DatasetRegistryException) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) Collection(java.util.Collection) Test(org.junit.Test) IOException(java.io.IOException) StringUtils(org.apache.commons.lang3.StringUtils) StandardCharsets(java.nio.charset.StandardCharsets) Is(org.hamcrest.core.Is) IOUtils(org.apache.commons.io.IOUtils) Dataset(org.globalbioticinteractions.dataset.Dataset) URI(java.net.URI) MatcherAssert.assertThat(org.hamcrest.MatcherAssert.assertThat) Collections(java.util.Collections) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) InputStream(java.io.InputStream) DatasetRegistryException(org.globalbioticinteractions.dataset.DatasetRegistryException) Dataset(org.globalbioticinteractions.dataset.Dataset) InputStream(java.io.InputStream) Collection(java.util.Collection) DatasetRegistry(org.globalbioticinteractions.dataset.DatasetRegistry) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) IOException(java.io.IOException) URI(java.net.URI) Test(org.junit.Test)

Example 18 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImplTest method doiExpectedPrefix.

@Test
public void doiExpectedPrefix() throws IOException {
    Dataset dataset = new DatasetImpl("some/namespace", URI.create("some:uri"), inStream -> inStream);
    dataset.setConfig(new ObjectMapper().readTree("{\"doi\": \"doi:10.2980/1195-6860(2006)13[23:TDOFUB]2.0.CO;2\" }"));
    assertThat(dataset.getDOI(), is(new DOI("2980", "1195-6860(2006)13[23:TDOFUB]2.0.CO;2")));
}
Also used : Dataset(org.globalbioticinteractions.dataset.Dataset) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DOI(org.globalbioticinteractions.doi.DOI) Test(org.junit.Test)

Example 19 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetImplTest method doiExpected.

@Test
public void doiExpected() throws IOException {
    Dataset dataset = new DatasetImpl("some/namespace", URI.create("some:uri"), inStream -> inStream);
    dataset.setConfig(new ObjectMapper().readTree("{\"doi\": \"http://dx.doi.org/10.2980/1195-6860(2006)13[23:TDOFUB]2.0.CO;2\" }"));
    assertThat(dataset.getDOI(), is(new DOI("2980", "1195-6860(2006)13[23:TDOFUB]2.0.CO;2")));
}
Also used : Dataset(org.globalbioticinteractions.dataset.Dataset) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DOI(org.globalbioticinteractions.doi.DOI) Test(org.junit.Test)

Example 20 with Dataset

use of org.globalbioticinteractions.dataset.Dataset in project eol-globi-data by jhpoelen.

the class DatasetTest method resourceURL.

@Test
public void resourceURL() throws IOException, URISyntaxException {
    URL resource = getClass().getResource("archive.zip");
    File archive = new File(resource.toURI());
    assertThat(archive.exists(), is(true));
    URI parentDir = archive.getParentFile().toURI();
    Dataset dataset = new DatasetImpl("some/namespace", parentDir, inStream -> inStream);
    InputStream retrieve = dataset.retrieve(resource.toURI());
    assertArchiveHash(retrieve);
    assertArchiveHash(dataset.retrieve(URI.create("archive.zip")));
    assertArchiveHash(dataset.retrieve(URI.create("/archive.zip")));
}
Also used : Dataset(org.globalbioticinteractions.dataset.Dataset) InputStream(java.io.InputStream) DatasetImpl(org.globalbioticinteractions.dataset.DatasetImpl) File(java.io.File) URI(java.net.URI) URL(java.net.URL) Test(org.junit.Test)

Aggregations

Dataset (org.globalbioticinteractions.dataset.Dataset)68 Test (org.junit.Test)46 DatasetImpl (org.globalbioticinteractions.dataset.DatasetImpl)25 URI (java.net.URI)20 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)17 JsonNode (com.fasterxml.jackson.databind.JsonNode)14 Matchers.containsString (org.hamcrest.Matchers.containsString)14 IOException (java.io.IOException)11 MatcherAssert.assertThat (org.hamcrest.MatcherAssert.assertThat)10 InputStream (java.io.InputStream)9 Is.is (org.hamcrest.core.Is.is)9 StudyNode (org.eol.globi.domain.StudyNode)8 Node (org.neo4j.graphdb.Node)8 URL (java.net.URL)7 ArrayList (java.util.ArrayList)7 StringStartsWith.startsWith (org.hamcrest.core.StringStartsWith.startsWith)7 Assert.assertNotNull (org.junit.Assert.assertNotNull)7 BaseDatasetImporter (org.eol.globi.data.BaseDatasetImporter)6 DatasetImporter (org.eol.globi.data.DatasetImporter)6 DatasetNode (org.eol.globi.domain.DatasetNode)6