Search in sources :

Example 1 with DatasetFinderException

use of org.eol.globi.service.DatasetFinderException in project eol-globi-data by jhpoelen.

the class DatasetFinderLocal method collectNamespaces.

private Collection<String> collectNamespaces(File directory) throws DatasetFinderException {
    Collection<File> accessFiles = FileUtils.listFiles(directory, new FileFileFilter() {

        @Override
        public boolean accept(File file) {
            return CacheLog.ACCESS_LOG_FILENAME.endsWith(file.getName());
        }
    }, TrueFileFilter.INSTANCE);
    Collection<String> namespaces = new TreeSet<>();
    for (File accessFile : accessFiles) {
        try {
            String[] rows = IOUtils.toString(accessFile.toURI()).split("\n");
            for (String row : rows) {
                namespaces.add(row.split("\t")[0]);
            }
        } catch (IOException e) {
            throw new DatasetFinderException("failed to read ", e);
        }
    }
    return namespaces;
}
Also used : TreeSet(java.util.TreeSet) FileFileFilter(org.apache.commons.io.filefilter.FileFileFilter) IOException(java.io.IOException) DatasetFinderException(org.eol.globi.service.DatasetFinderException) File(java.io.File)

Example 2 with DatasetFinderException

use of org.eol.globi.service.DatasetFinderException in project eol-globi-data by jhpoelen.

the class DatasetFinderLocal method datasetFor.

@Override
public Dataset datasetFor(String namespace) throws DatasetFinderException {
    Dataset dataset;
    try {
        final URI sourceURI = findLastCachedDatasetURI(namespace);
        dataset = sourceURI == null ? null : DatasetFactory.datasetFor(namespace, new DatasetFinder() {

            @Override
            public Collection<String> findNamespaces() throws DatasetFinderException {
                return Collections.singletonList(namespace);
            }

            @Override
            public Dataset datasetFor(String s) throws DatasetFinderException {
                Dataset dataset = new DatasetImpl(namespace, sourceURI);
                return new DatasetWithCache(dataset, cacheFactory.cacheFor(dataset));
            }
        });
    } catch (IOException e) {
        throw new DatasetFinderException("failed to access [" + namespace + "]", e);
    }
    if (dataset == null) {
        throw new DatasetFinderException("failed to retrieve/cache dataset in namespace [" + namespace + "]");
    }
    return dataset;
}
Also used : Dataset(org.eol.globi.service.Dataset) DatasetFinder(org.eol.globi.service.DatasetFinder) Collection(java.util.Collection) DatasetImpl(org.eol.globi.service.DatasetImpl) IOException(java.io.IOException) DatasetFinderException(org.eol.globi.service.DatasetFinderException) URI(java.net.URI)

Example 3 with DatasetFinderException

use of org.eol.globi.service.DatasetFinderException in project eol-globi-data by jhpoelen.

the class StudyImporterForGitHubData method importData.

public void importData(String repository) throws StudyImporterException {
    try {
        LOG.info("importing github repo [" + repository + "]...");
        Dataset dataset = DatasetFactory.datasetFor(repository, getDatasetFinder());
        nodeFactory.getOrCreateDataset(dataset);
        importData(dataset);
        LOG.info("importing github repo [" + repository + "] done.");
    } catch (StudyImporterException | DatasetFinderException ex) {
        String msg = "failed to import data from repo [" + repository + "]";
        LOG.error(msg, ex);
        throw new StudyImporterException(msg, ex);
    }
}
Also used : Dataset(org.eol.globi.service.Dataset) DatasetFinderException(org.eol.globi.service.DatasetFinderException)

Aggregations

DatasetFinderException (org.eol.globi.service.DatasetFinderException)3 IOException (java.io.IOException)2 Dataset (org.eol.globi.service.Dataset)2 File (java.io.File)1 URI (java.net.URI)1 Collection (java.util.Collection)1 TreeSet (java.util.TreeSet)1 FileFileFilter (org.apache.commons.io.filefilter.FileFileFilter)1 DatasetFinder (org.eol.globi.service.DatasetFinder)1 DatasetImpl (org.eol.globi.service.DatasetImpl)1