Search in sources :

Example 11 with FetcherException

use of org.jabref.logic.importer.FetcherException in project jabref by JabRef.

the class ArXiv method callApi.

/**
     * Queries the API.
     *
     * If only {@code searchQuery} is given, then the API will return results for each article that matches the query.
     * If only {@code ids} is given, then the API will return results for each article in the list.
     * If both {@code searchQuery} and {@code ids} are given, then the API will return each article in
     * {@code ids} that matches {@code searchQuery}. This allows the API to act as a results filter.
     *
     * @param searchQuery the search query used to find articles;
     *                    <a href="http://arxiv.org/help/api/user-manual#query_details">details</a>
     * @param ids         a list of arXiv identifiers
     * @param start       the index of the first returned result (zero-based)
     * @param maxResults  the number of maximal results (has to be smaller than 2000)
     * @return the response from the API as a XML document (Atom 1.0)
     * @throws FetcherException if there was a problem while building the URL or the API was not accessible
     */
private Document callApi(String searchQuery, List<ArXivIdentifier> ids, int start, int maxResults) throws FetcherException {
    if (maxResults > 2000) {
        throw new IllegalArgumentException("The arXiv API limits the number of maximal results to be 2000");
    }
    try {
        URIBuilder uriBuilder = new URIBuilder(API_URL);
        // The arXiv API has problems with accents, so we remove them (i.e. Fréchet -> Frechet)
        if (StringUtil.isNotBlank(searchQuery)) {
            uriBuilder.addParameter("search_query", StringUtil.stripAccents(searchQuery));
        }
        if (!ids.isEmpty()) {
            uriBuilder.addParameter("id_list", ids.stream().map(ArXivIdentifier::getNormalized).collect(Collectors.joining(",")));
        }
        uriBuilder.addParameter("start", String.valueOf(start));
        uriBuilder.addParameter("max_results", String.valueOf(maxResults));
        URL url = uriBuilder.build().toURL();
        DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
        DocumentBuilder builder = factory.newDocumentBuilder();
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        if (connection.getResponseCode() == 400) {
            // Bad request error from server, try to get more information
            throw getException(builder.parse(connection.getErrorStream()));
        } else {
            return builder.parse(connection.getInputStream());
        }
    } catch (SAXException | ParserConfigurationException | IOException | URISyntaxException exception) {
        throw new FetcherException("arXiv API request failed", exception);
    }
}
Also used : DocumentBuilderFactory(javax.xml.parsers.DocumentBuilderFactory) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) URL(java.net.URL) URIBuilder(org.apache.http.client.utils.URIBuilder) SAXException(org.xml.sax.SAXException) FetcherException(org.jabref.logic.importer.FetcherException) HttpURLConnection(java.net.HttpURLConnection) DocumentBuilder(javax.xml.parsers.DocumentBuilder) ArXivIdentifier(org.jabref.model.entry.identifier.ArXivIdentifier) ParserConfigurationException(javax.xml.parsers.ParserConfigurationException)

Example 12 with FetcherException

use of org.jabref.logic.importer.FetcherException in project jabref by JabRef.

the class ArXiv method getException.

private FetcherException getException(Document error) {
    List<Node> entries = XMLUtil.asList(error.getElementsByTagName("entry"));
    // </entry>
    if (entries.size() == 1) {
        Node node = entries.get(0);
        Optional<String> id = XMLUtil.getNodeContent(node, "id");
        Boolean isError = id.map(idContent -> idContent.startsWith("http://arxiv.org/api/errors")).orElse(false);
        if (isError) {
            String errorMessage = XMLUtil.getNodeContent(node, "summary").orElse("Unknown error");
            return new FetcherException(errorMessage);
        }
    }
    return new FetcherException("arXiv API request failed");
}
Also used : HttpURLConnection(java.net.HttpURLConnection) StringSimilarity(org.jabref.logic.util.strings.StringSimilarity) FieldName(org.jabref.model.entry.FieldName) URL(java.net.URL) XMLUtil(org.jabref.logic.util.io.XMLUtil) StringUtil(org.jabref.model.strings.StringUtil) URISyntaxException(java.net.URISyntaxException) ImportFormatPreferences(org.jabref.logic.importer.ImportFormatPreferences) OptionalUtil(org.jabref.model.util.OptionalUtil) LinkedFile(org.jabref.model.entry.LinkedFile) ArrayList(java.util.ArrayList) OAI2Handler(org.jabref.logic.importer.util.OAI2Handler) BibtexEntryTypes(org.jabref.model.entry.BibtexEntryTypes) Document(org.w3c.dom.Document) Node(org.w3c.dom.Node) ArXivIdentifier(org.jabref.model.entry.identifier.ArXivIdentifier) HelpFile(org.jabref.logic.help.HelpFile) MalformedURLException(java.net.MalformedURLException) URIBuilder(org.apache.http.client.utils.URIBuilder) FulltextFetcher(org.jabref.logic.importer.FulltextFetcher) FetcherException(org.jabref.logic.importer.FetcherException) BibEntry(org.jabref.model.entry.BibEntry) IdFetcher(org.jabref.logic.importer.IdFetcher) SearchBasedFetcher(org.jabref.logic.importer.SearchBasedFetcher) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) Objects(java.util.Objects) List(java.util.List) DOI(org.jabref.model.entry.identifier.DOI) ParserConfigurationException(javax.xml.parsers.ParserConfigurationException) DocumentBuilder(javax.xml.parsers.DocumentBuilder) SAXException(org.xml.sax.SAXException) IdBasedFetcher(org.jabref.logic.importer.IdBasedFetcher) Optional(java.util.Optional) Log(org.apache.commons.logging.Log) DocumentBuilderFactory(javax.xml.parsers.DocumentBuilderFactory) LogFactory(org.apache.commons.logging.LogFactory) Collections(java.util.Collections) FetcherException(org.jabref.logic.importer.FetcherException) Node(org.w3c.dom.Node)

Example 13 with FetcherException

use of org.jabref.logic.importer.FetcherException in project jabref by JabRef.

the class DoiFetcher method performSearchById.

@Override
public Optional<BibEntry> performSearchById(String identifier) throws FetcherException {
    Optional<DOI> doi = DOI.parse(identifier);
    try {
        if (doi.isPresent()) {
            URL doiURL = new URL(doi.get().getURIAsASCIIString());
            // BibTeX data
            URLDownload download = new URLDownload(doiURL);
            download.addHeader("Accept", "application/x-bibtex");
            String bibtexString = download.asString();
            // BibTeX entry
            Optional<BibEntry> fetchedEntry = BibtexParser.singleFromString(bibtexString, preferences);
            fetchedEntry.ifPresent(this::doPostCleanup);
            return fetchedEntry;
        } else {
            throw new FetcherException(Localization.lang("Invalid_DOI:_'%0'.", identifier));
        }
    } catch (IOException e) {
        throw new FetcherException(Localization.lang("Connection error"), e);
    } catch (ParseException e) {
        throw new FetcherException("Could not parse BibTeX entry", e);
    }
}
Also used : BibEntry(org.jabref.model.entry.BibEntry) FetcherException(org.jabref.logic.importer.FetcherException) IOException(java.io.IOException) ParseException(org.jabref.logic.importer.ParseException) URLDownload(org.jabref.logic.net.URLDownload) URL(java.net.URL) DOI(org.jabref.model.entry.identifier.DOI)

Example 14 with FetcherException

use of org.jabref.logic.importer.FetcherException in project jabref by JabRef.

the class AstrophysicsDataSystem method performSearch.

@Override
public List<BibEntry> performSearch(String query) throws FetcherException {
    if (StringUtil.isBlank(query)) {
        return Collections.emptyList();
    }
    try {
        URLConnection connection = getURLForQuery(query).openConnection();
        connection.setRequestProperty("User-Agent", URLDownload.USER_AGENT);
        try (InputStream stream = connection.getInputStream()) {
            List<BibEntry> fetchedEntries = getParser().parseEntries(stream);
            // Post-cleanup
            fetchedEntries.forEach(this::doPostCleanup);
            return fetchedEntries;
        } catch (IOException e) {
            throw new FetcherException("An I/O exception occurred", e);
        }
    } catch (URISyntaxException | MalformedURLException e) {
        throw new FetcherException("Search URI is malformed", e);
    } catch (IOException e) {
        throw new FetcherException("An I/O exception occurred", e);
    } catch (ParseException e) {
        throw new FetcherException("Error occurred when parsing entry", Localization.lang("Error occurred when parsing entry"), e);
    }
}
Also used : BibEntry(org.jabref.model.entry.BibEntry) FetcherException(org.jabref.logic.importer.FetcherException) MalformedURLException(java.net.MalformedURLException) InputStream(java.io.InputStream) IOException(java.io.IOException) URISyntaxException(java.net.URISyntaxException) ParseException(org.jabref.logic.importer.ParseException) URLConnection(java.net.URLConnection)

Example 15 with FetcherException

use of org.jabref.logic.importer.FetcherException in project jabref by JabRef.

the class GoogleScholar method obtainAndModifyCookie.

private void obtainAndModifyCookie() throws FetcherException {
    try {
        URLDownload downloader = new URLDownload("https://scholar.google.com");
        List<HttpCookie> cookies = downloader.getCookieFromUrl();
        for (HttpCookie cookie : cookies) {
            // append "CF=4" which represents "Citation format bibtex"
            cookie.setValue(cookie.getValue() + ":CF=4");
        }
    } catch (IOException e) {
        throw new FetcherException("Cookie configuration for Google Scholar failed.", e);
    }
}
Also used : FetcherException(org.jabref.logic.importer.FetcherException) IOException(java.io.IOException) URLDownload(org.jabref.logic.net.URLDownload) HttpCookie(java.net.HttpCookie)

Aggregations

FetcherException (org.jabref.logic.importer.FetcherException)18 IOException (java.io.IOException)12 BibEntry (org.jabref.model.entry.BibEntry)12 URISyntaxException (java.net.URISyntaxException)7 URL (java.net.URL)6 ArrayList (java.util.ArrayList)6 ParserResult (org.jabref.logic.importer.ParserResult)5 DOI (org.jabref.model.entry.identifier.DOI)5 URIBuilder (org.apache.http.client.utils.URIBuilder)4 URLDownload (org.jabref.logic.net.URLDownload)4 StringReader (java.io.StringReader)3 MalformedURLException (java.net.MalformedURLException)3 Optional (java.util.Optional)3 ParseException (org.jabref.logic.importer.ParseException)3 BufferedReader (java.io.BufferedReader)2 HttpURLConnection (java.net.HttpURLConnection)2 URLConnection (java.net.URLConnection)2 List (java.util.List)2 DocumentBuilder (javax.xml.parsers.DocumentBuilder)2 DocumentBuilderFactory (javax.xml.parsers.DocumentBuilderFactory)2