Search in sources :

Example 6 with Dataset

use of org.eol.globi.service.Dataset in project eol-globi-data by jhpoelen.

the class StudyImporterForArthopodEasyCaptureTest method readRSS.

@Test
public void readRSS() throws StudyImporterException, IOException {
    final ParserFactory parserFactory = null;
    final NodeFactory nodeFactory = null;
    final Dataset dataset = getDatasetGroup();
    List<StudyImporter> importers = StudyImporterForArthopodEasyCapture.getStudyImportersForRSSFeed(dataset, parserFactory, nodeFactory);
    assertThat(importers.size(), is(3));
}
Also used : Dataset(org.eol.globi.service.Dataset) Test(org.junit.Test)

Example 7 with Dataset

use of org.eol.globi.service.Dataset in project eol-globi-data by jhpoelen.

the class StudyImporterForArthopodEasyCaptureTest method embeddedDatasetWithConfig.

@Test
public void embeddedDatasetWithConfig() throws IOException {
    Dataset embeddedDataset = StudyImporterForArthopodEasyCapture.embeddedDatasetFor(getDatasetGroupWithProperty(), "some other citation", URI.create("http://example.com/archive.zip"));
    assertThat(embeddedDataset.getCitation(), is("some other citation"));
    assertThat(embeddedDataset.getOrDefault(DatasetConstant.SHOULD_RESOLVE_REFERENCES, "true"), is("false"));
    assertThat(DatasetUtil.getNamedResourceURI(embeddedDataset, "archive"), is("http://example.com/archive.zip"));
}
Also used : Dataset(org.eol.globi.service.Dataset) Test(org.junit.Test)

Example 8 with Dataset

use of org.eol.globi.service.Dataset in project eol-globi-data by jhpoelen.

the class DatasetFinderWithCacheIT method hafnerTest.

@Test
public void hafnerTest() throws DatasetFinderException, IOException {
    DatasetFinder finder = new DatasetFinderWithCache(new DatasetFinderGitHubArchive(), cachePath);
    Dataset dataset = DatasetFactory.datasetFor("globalbioticinteractions/hafner", finder);
    assertNotNull(dataset.getResource("hafner/gopher_lice_int.csv"));
}
Also used : DatasetFinder(org.eol.globi.service.DatasetFinder) Dataset(org.eol.globi.service.Dataset) DatasetFinderGitHubArchive(org.eol.globi.service.DatasetFinderGitHubArchive) Test(org.junit.Test)

Example 9 with Dataset

use of org.eol.globi.service.Dataset in project eol-globi-data by jhpoelen.

the class NodeFactoryNeo4j method getOrCreateDataset.

@Override
public Dataset getOrCreateDataset(Dataset originatingDataset) {
    Transaction transaction = graphDb.beginTx();
    try {
        Dataset datasetCreated = getOrCreateDatasetNoTx(originatingDataset);
        transaction.success();
        return datasetCreated;
    } finally {
        transaction.finish();
    }
}
Also used : Transaction(org.neo4j.graphdb.Transaction) Dataset(org.eol.globi.service.Dataset)

Example 10 with Dataset

use of org.eol.globi.service.Dataset in project eol-globi-data by jhpoelen.

the class StudyImporterForArthopodEasyCapture method getStudyImportersForRSSFeed.

public static List<StudyImporter> getStudyImportersForRSSFeed(Dataset datasetOrig, ParserFactory parserFactory, NodeFactory nodeFactory) throws StudyImporterException {
    SyndFeedInput input = new SyndFeedInput();
    SyndFeed feed;
    String rss = getRss(datasetOrig);
    try {
        feed = input.build(new XmlReader(new URL(rss)));
    } catch (FeedException | IOException e) {
        throw new StudyImporterException("failed to read rss feed [" + rss + "]", e);
    }
    List<StudyImporter> importers = new ArrayList<StudyImporter>();
    final List entries = feed.getEntries();
    for (Object entry : entries) {
        if (entry instanceof SyndEntry) {
            SyndEntry syndEntry = (SyndEntry) entry;
            Dataset dataset = embeddedDatasetFor(datasetOrig, StringUtils.trim(syndEntry.getDescription().getValue()), URI.create(StringUtils.trim(syndEntry.getLink())));
            final StudyImporterForSeltmann studyImporter = new StudyImporterForSeltmann(parserFactory, nodeFactory);
            studyImporter.setDataset(dataset);
            importers.add(studyImporter);
        }
    }
    return importers;
}
Also used : SyndEntry(com.sun.syndication.feed.synd.SyndEntry) Dataset(org.eol.globi.service.Dataset) FeedException(com.sun.syndication.io.FeedException) ArrayList(java.util.ArrayList) XmlReader(com.sun.syndication.io.XmlReader) IOException(java.io.IOException) URL(java.net.URL) SyndFeed(com.sun.syndication.feed.synd.SyndFeed) SyndFeedInput(com.sun.syndication.io.SyndFeedInput) ArrayList(java.util.ArrayList) List(java.util.List)

Aggregations

Dataset (org.eol.globi.service.Dataset)31 Test (org.junit.Test)16 JsonNode (org.codehaus.jackson.JsonNode)11 ObjectMapper (org.codehaus.jackson.map.ObjectMapper)9 DatasetImpl (org.eol.globi.service.DatasetImpl)9 IOException (java.io.IOException)6 InputStream (java.io.InputStream)5 URI (java.net.URI)5 URL (java.net.URL)5 ArrayList (java.util.ArrayList)4 List (java.util.List)4 DatasetFinder (org.eol.globi.service.DatasetFinder)4 Is.is (org.hamcrest.core.Is.is)4 Assert.assertNotNull (org.junit.Assert.assertNotNull)4 Assert.assertThat (org.junit.Assert.assertThat)4 CSVParser (com.Ostermiller.util.CSVParser)3 LabeledCSVParser (com.Ostermiller.util.LabeledCSVParser)3 Map (java.util.Map)3 Assert (junit.framework.Assert)3 IOUtils (org.apache.commons.io.IOUtils)3