Search in sources :

Example 6 with JsonParserException

use of com.grack.nanojson.JsonParserException in project NewPipeExtractor by TeamNewPipe.

the class BandcampChannelLinkHandlerFactory method getId.

@Override
public String getId(final String url) throws ParsingException {
    try {
        final String response = NewPipe.getDownloader().get(url).responseBody();
        // Use band data embedded in website to extract ID
        final JsonObject bandData = JsonUtils.getJsonData(response, "data-band");
        return String.valueOf(bandData.getLong("id"));
    } catch (final IOException | ReCaptchaException | ArrayIndexOutOfBoundsException | JsonParserException e) {
        throw new ParsingException("Download failed", e);
    }
}
Also used : JsonParserException(com.grack.nanojson.JsonParserException) ParsingException(org.schabi.newpipe.extractor.exceptions.ParsingException) JsonObject(com.grack.nanojson.JsonObject) IOException(java.io.IOException) ReCaptchaException(org.schabi.newpipe.extractor.exceptions.ReCaptchaException)

Example 7 with JsonParserException

use of com.grack.nanojson.JsonParserException in project NewPipeExtractor by TeamNewPipe.

the class PeertubeAccountExtractor method getSubscriberCount.

@Override
public long getSubscriberCount() throws ParsingException {
    // The subscriber count cannot be retrieved directly. It needs to be calculated.
    // An accounts subscriber count is the number of the channel owner's subscriptions
    // plus the sum of all sub channels subscriptions.
    long subscribersCount = json.getLong("followersCount");
    String accountVideoChannelUrl = baseUrl + PeertubeChannelLinkHandlerFactory.API_ENDPOINT;
    if (getId().contains(ACCOUNTS)) {
        accountVideoChannelUrl += getId();
    } else {
        accountVideoChannelUrl += ACCOUNTS + getId();
    }
    accountVideoChannelUrl += "/video-channels";
    try {
        final String responseBody = getDownloader().get(accountVideoChannelUrl).responseBody();
        final JsonObject jsonResponse = JsonParser.object().from(responseBody);
        final JsonArray videoChannels = jsonResponse.getArray("data");
        for (final Object videoChannel : videoChannels) {
            final JsonObject videoChannelJsonObject = (JsonObject) videoChannel;
            subscribersCount += videoChannelJsonObject.getInt("followersCount");
        }
    } catch (final IOException | JsonParserException | ReCaptchaException ignored) {
    // something went wrong during video channels extraction, only return subscribers of ownerAccount
    }
    return subscribersCount;
}
Also used : JsonArray(com.grack.nanojson.JsonArray) JsonParserException(com.grack.nanojson.JsonParserException) JsonObject(com.grack.nanojson.JsonObject) JsonObject(com.grack.nanojson.JsonObject) IOException(java.io.IOException) ReCaptchaException(org.schabi.newpipe.extractor.exceptions.ReCaptchaException)

Example 8 with JsonParserException

use of com.grack.nanojson.JsonParserException in project NewPipeExtractor by TeamNewPipe.

the class PeertubeStreamExtractor method getStreamsFromApi.

private void getStreamsFromApi(final StreamInfoItemsCollector collector, final String apiUrl) throws ReCaptchaException, IOException, ParsingException {
    final Response response = getDownloader().get(apiUrl);
    JsonObject relatedVideosJson = null;
    if (response != null && !Utils.isBlank(response.responseBody())) {
        try {
            relatedVideosJson = JsonParser.object().from(response.responseBody());
        } catch (JsonParserException e) {
            throw new ParsingException("Could not parse json data for related videos", e);
        }
    }
    if (relatedVideosJson != null) {
        collectStreamsFrom(collector, relatedVideosJson);
    }
}
Also used : Response(org.schabi.newpipe.extractor.downloader.Response) JsonParserException(com.grack.nanojson.JsonParserException) ParsingException(org.schabi.newpipe.extractor.exceptions.ParsingException) JsonObject(com.grack.nanojson.JsonObject)

Example 9 with JsonParserException

use of com.grack.nanojson.JsonParserException in project NewPipeExtractor by TeamNewPipe.

the class SoundcloudParsingHelper method getStreamsFromApi.

/**
 * Fetch the streams from the given API and commit each of them to the collector.
 *
 * @return the next streams url, empty if don't have
 */
public static String getStreamsFromApi(final StreamInfoItemsCollector collector, final String apiUrl, final boolean charts) throws IOException, ReCaptchaException, ParsingException {
    final Response response = NewPipe.getDownloader().get(apiUrl, SoundCloud.getLocalization());
    if (response.responseCode() >= 400) {
        throw new IOException("Could not get streams from API, HTTP " + response.responseCode());
    }
    final JsonObject responseObject;
    try {
        responseObject = JsonParser.object().from(response.responseBody());
    } catch (final JsonParserException e) {
        throw new ParsingException("Could not parse json response", e);
    }
    final JsonArray responseCollection = responseObject.getArray("collection");
    for (final Object o : responseCollection) {
        if (o instanceof JsonObject) {
            final JsonObject object = (JsonObject) o;
            collector.commit(new SoundcloudStreamInfoItemExtractor(charts ? object.getObject("track") : object));
        }
    }
    String nextPageUrl;
    try {
        nextPageUrl = responseObject.getString("next_href");
        if (!nextPageUrl.contains("client_id="))
            nextPageUrl += "&client_id=" + SoundcloudParsingHelper.clientId();
    } catch (final Exception ignored) {
        nextPageUrl = "";
    }
    return nextPageUrl;
}
Also used : Response(org.schabi.newpipe.extractor.downloader.Response) JsonArray(com.grack.nanojson.JsonArray) SoundcloudStreamInfoItemExtractor(org.schabi.newpipe.extractor.services.soundcloud.extractors.SoundcloudStreamInfoItemExtractor) JsonParserException(com.grack.nanojson.JsonParserException) ParsingException(org.schabi.newpipe.extractor.exceptions.ParsingException) JsonObject(com.grack.nanojson.JsonObject) JsonObject(com.grack.nanojson.JsonObject) IOException(java.io.IOException) ReCaptchaException(org.schabi.newpipe.extractor.exceptions.ReCaptchaException) JsonParserException(com.grack.nanojson.JsonParserException) ParsingException(org.schabi.newpipe.extractor.exceptions.ParsingException) ExtractionException(org.schabi.newpipe.extractor.exceptions.ExtractionException) MalformedURLException(java.net.MalformedURLException) IOException(java.io.IOException) DateTimeParseException(java.time.format.DateTimeParseException) RegexException(org.schabi.newpipe.extractor.utils.Parser.RegexException)

Example 10 with JsonParserException

use of com.grack.nanojson.JsonParserException in project NewPipeExtractor by TeamNewPipe.

the class MediaCCCParsingHelper method getLiveStreams.

/**
 * Get currently available live streams from
 * <a href="https://streaming.media.ccc.de/streams/v2.json">https://streaming.media.ccc.de/streams/v2.json</a>.
 * Use this method to cache requests, because they can get quite big.
 * TODO: implement better caching policy (max-age: 3 min)
 * @param downloader The downloader to use for making the request
 * @param localization The localization to be used. Will most likely be ignored.
 * @return {@link JsonArray} containing current conferences and info about their rooms and streams.
 * @throws ExtractionException if the data could not be fetched or the retrieved data could not be parsed to a {@link JsonArray}
 */
public static JsonArray getLiveStreams(final Downloader downloader, final Localization localization) throws ExtractionException {
    if (liveStreams == null) {
        try {
            final String site = downloader.get("https://streaming.media.ccc.de/streams/v2.json", localization).responseBody();
            liveStreams = JsonParser.array().from(site);
        } catch (IOException | ReCaptchaException e) {
            throw new ExtractionException("Could not get live stream JSON.", e);
        } catch (JsonParserException e) {
            throw new ExtractionException("Could not parse JSON.", e);
        }
    }
    return liveStreams;
}
Also used : ExtractionException(org.schabi.newpipe.extractor.exceptions.ExtractionException) JsonParserException(com.grack.nanojson.JsonParserException) IOException(java.io.IOException) ReCaptchaException(org.schabi.newpipe.extractor.exceptions.ReCaptchaException)

Aggregations

JsonParserException (com.grack.nanojson.JsonParserException)30 JsonObject (com.grack.nanojson.JsonObject)20 ParsingException (org.schabi.newpipe.extractor.exceptions.ParsingException)18 JsonArray (com.grack.nanojson.JsonArray)11 Downloader (org.schabi.newpipe.extractor.downloader.Downloader)11 IOException (java.io.IOException)10 ReCaptchaException (org.schabi.newpipe.extractor.exceptions.ReCaptchaException)9 Response (org.schabi.newpipe.extractor.downloader.Response)8 ExtractionException (org.schabi.newpipe.extractor.exceptions.ExtractionException)8 ArrayList (java.util.ArrayList)5 MalformedURLException (java.net.MalformedURLException)4 Nonnull (javax.annotation.Nonnull)4 Page (org.schabi.newpipe.extractor.Page)3 SharedPreferences (android.content.SharedPreferences)2 URL (java.net.URL)2 DateTimeParseException (java.time.format.DateTimeParseException)2 CommentsInfoItemsCollector (org.schabi.newpipe.extractor.comments.CommentsInfoItemsCollector)2 PeertubeInstance (org.schabi.newpipe.extractor.services.peertube.PeertubeInstance)2 RegexException (org.schabi.newpipe.extractor.utils.Parser.RegexException)2 Caffeine (com.github.benmanes.caffeine.cache.Caffeine)1