Search in sources :

Example 1 with GraphService

use of org.eol.globi.db.GraphService in project eol-globi-data by jhpoelen.

the class NanoPress method createLinkers.

public static List<Linker> createLinkers(CommandLine cmdLine, GraphDatabaseService graphService, File pubDir) {
    List<Linker> linkers = new ArrayList<>();
    if (!cmdLine.hasOption("nanoOnly")) {
        String batchSize = cmdLine.getOptionValue("batchSize", "1000");
        linkers.add(new IndexInteractions(graphService, Integer.parseInt(batchSize)));
    }
    linkers.add(new LinkerTrustyNanoPubs(graphService, nanopub -> {
        String code = TrustyUriUtils.getArtifactCode(nanopub.getUri().toString());
        OutputStream os = null;
        if (StringUtils.isNotBlank(code)) {
            try {
                os = new GZIPOutputStream(new FileOutputStream(new File(pubDir, code + ".trig.gz")));
            } catch (IOException e) {
                LOG.error("failed to produce nanopub file for [" + code + "]", e);
            }
        }
        return os == null ? new NullOutputStream() : os;
    }));
    return linkers;
}
Also used : StringUtils(org.apache.commons.lang.StringUtils) Arrays(java.util.Arrays) Options(org.apache.commons.cli.Options) HelpFormatter(org.apache.commons.cli.HelpFormatter) ArrayList(java.util.ArrayList) TrustyUriUtils(net.trustyuri.TrustyUriUtils) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) Map(java.util.Map) BasicParser(org.apache.commons.cli.BasicParser) CommandLine(org.apache.commons.cli.CommandLine) GraphService(org.eol.globi.db.GraphService) MapUtil(org.neo4j.helpers.collection.MapUtil) Option(org.apache.commons.cli.Option) OutputStream(java.io.OutputStream) CommandLineParser(org.apache.commons.cli.CommandLineParser) FileOutputStream(java.io.FileOutputStream) FileUtils(org.apache.commons.io.FileUtils) IOException(java.io.IOException) File(java.io.File) List(java.util.List) ParseException(org.apache.commons.cli.ParseException) NullOutputStream(org.apache.commons.io.output.NullOutputStream) Log(org.apache.commons.logging.Log) GZIPOutputStream(java.util.zip.GZIPOutputStream) LogFactory(org.apache.commons.logging.LogFactory) OutputStream(java.io.OutputStream) FileOutputStream(java.io.FileOutputStream) NullOutputStream(org.apache.commons.io.output.NullOutputStream) GZIPOutputStream(java.util.zip.GZIPOutputStream) ArrayList(java.util.ArrayList) IOException(java.io.IOException) GZIPOutputStream(java.util.zip.GZIPOutputStream) FileOutputStream(java.io.FileOutputStream) File(java.io.File) NullOutputStream(org.apache.commons.io.output.NullOutputStream)

Example 2 with GraphService

use of org.eol.globi.db.GraphService in project eol-globi-data by jhpoelen.

the class Normalizer method importData.

void importData(GraphDatabaseService graphService, String cacheDir) {
    NodeFactoryNeo4j factory = new NodeFactoryNeo4j(graphService);
    factory.setEcoregionFinder(getEcoregionFinder());
    factory.setDoiResolver(new DOIResolverImpl());
    try {
        CacheFactory cacheFactory = dataset -> new CacheLocalReadonly(dataset.getNamespace(), cacheDir);
        DatasetFinder finder = new DatasetFinderLocal(cacheDir, cacheFactory);
        StudyImporter importer = new StudyImporterForGitHubData(new ParserFactoryLocal(), factory, finder);
        importer.setDataset(new DatasetLocal());
        importer.setLogger(new StudyImportLogger());
        importer.importStudy();
    } catch (StudyImporterException e) {
        LOG.error("problem encountered while importing [" + StudyImporterForGitHubData.class.getName() + "]", e);
    }
    EcoregionFinder regionFinder = getEcoregionFinder();
    if (regionFinder != null) {
        regionFinder.shutdown();
    }
}
Also used : OpenTreeTaxonIndex(org.eol.globi.opentree.OpenTreeTaxonIndex) NonResolvingTaxonIndex(org.eol.globi.taxon.NonResolvingTaxonIndex) EcoregionFinder(org.eol.globi.geo.EcoregionFinder) DatasetFinderLocal(org.globalbioticinteractions.dataset.DatasetFinderLocal) URISyntaxException(java.net.URISyntaxException) Options(org.apache.commons.cli.Options) TaxonCacheService(org.eol.globi.taxon.TaxonCacheService) DOIResolverCache(org.eol.globi.service.DOIResolverCache) ResolvingTaxonIndex(org.eol.globi.taxon.ResolvingTaxonIndex) HelpFormatter(org.apache.commons.cli.HelpFormatter) StringUtils(org.apache.commons.lang3.StringUtils) ArrayList(java.util.ArrayList) StudyImporterException(org.eol.globi.data.StudyImporterException) EcoregionFinderProxy(org.eol.globi.service.EcoregionFinderProxy) ParserFactoryLocal(org.eol.globi.data.ParserFactoryLocal) GraphDatabaseService(org.neo4j.graphdb.GraphDatabaseService) BasicParser(org.apache.commons.cli.BasicParser) CommandLine(org.apache.commons.cli.CommandLine) GraphService(org.eol.globi.db.GraphService) URI(java.net.URI) EcoregionFinderFactoryImpl(org.eol.globi.geo.EcoregionFinderFactoryImpl) HttpUtil(org.eol.globi.util.HttpUtil) Option(org.apache.commons.cli.Option) Taxon(org.eol.globi.domain.Taxon) CommandLineParser(org.apache.commons.cli.CommandLineParser) MalformedURLException(java.net.MalformedURLException) CacheLocalReadonly(org.globalbioticinteractions.cache.CacheLocalReadonly) DatasetFinder(org.eol.globi.service.DatasetFinder) DatasetLocal(org.eol.globi.service.DatasetLocal) StudyImporterForGitHubData(org.eol.globi.data.StudyImporterForGitHubData) DOIResolverImpl(org.eol.globi.service.DOIResolverImpl) NodeFactoryNeo4j(org.eol.globi.data.NodeFactoryNeo4j) List(java.util.List) StudyImporter(org.eol.globi.data.StudyImporter) ParseException(org.apache.commons.cli.ParseException) GraphExporterImpl(org.eol.globi.export.GraphExporterImpl) Version(org.eol.globi.Version) Log(org.apache.commons.logging.Log) LogFactory(org.apache.commons.logging.LogFactory) CacheFactory(org.globalbioticinteractions.cache.CacheFactory) EcoregionFinder(org.eol.globi.geo.EcoregionFinder) StudyImporterForGitHubData(org.eol.globi.data.StudyImporterForGitHubData) DOIResolverImpl(org.eol.globi.service.DOIResolverImpl) NodeFactoryNeo4j(org.eol.globi.data.NodeFactoryNeo4j) DatasetLocal(org.eol.globi.service.DatasetLocal) StudyImporter(org.eol.globi.data.StudyImporter) CacheLocalReadonly(org.globalbioticinteractions.cache.CacheLocalReadonly) StudyImporterException(org.eol.globi.data.StudyImporterException) DatasetFinder(org.eol.globi.service.DatasetFinder) CacheFactory(org.globalbioticinteractions.cache.CacheFactory) DatasetFinderLocal(org.globalbioticinteractions.dataset.DatasetFinderLocal) ParserFactoryLocal(org.eol.globi.data.ParserFactoryLocal)

Aggregations

ArrayList (java.util.ArrayList)2 List (java.util.List)2 BasicParser (org.apache.commons.cli.BasicParser)2 CommandLine (org.apache.commons.cli.CommandLine)2 CommandLineParser (org.apache.commons.cli.CommandLineParser)2 HelpFormatter (org.apache.commons.cli.HelpFormatter)2 Option (org.apache.commons.cli.Option)2 Options (org.apache.commons.cli.Options)2 ParseException (org.apache.commons.cli.ParseException)2 Log (org.apache.commons.logging.Log)2 LogFactory (org.apache.commons.logging.LogFactory)2 GraphService (org.eol.globi.db.GraphService)2 GraphDatabaseService (org.neo4j.graphdb.GraphDatabaseService)2 File (java.io.File)1 FileOutputStream (java.io.FileOutputStream)1 IOException (java.io.IOException)1 OutputStream (java.io.OutputStream)1 MalformedURLException (java.net.MalformedURLException)1 URI (java.net.URI)1 URISyntaxException (java.net.URISyntaxException)1