Search in sources :

Example 1 with StringDelimitedProcessor

use of com.twitter.hbc.core.processor.StringDelimitedProcessor in project nifi by apache.

the class GetTwitter method onScheduled.

@OnScheduled
public void onScheduled(final ProcessContext context) throws MalformedURLException {
    final String endpointName = context.getProperty(ENDPOINT).getValue();
    final Authentication oauth = new OAuth1(context.getProperty(CONSUMER_KEY).getValue(), context.getProperty(CONSUMER_SECRET).getValue(), context.getProperty(ACCESS_TOKEN).getValue(), context.getProperty(ACCESS_TOKEN_SECRET).getValue());
    final ClientBuilder clientBuilder = new ClientBuilder();
    clientBuilder.name("GetTwitter[id=" + getIdentifier() + "]").authentication(oauth).eventMessageQueue(eventQueue).processor(new StringDelimitedProcessor(messageQueue));
    final String languageString = context.getProperty(LANGUAGES).getValue();
    final List<String> languages;
    if (languageString == null) {
        languages = null;
    } else {
        languages = new ArrayList<>();
        for (final String language : context.getProperty(LANGUAGES).getValue().split(",")) {
            languages.add(language.trim());
        }
    }
    final String host;
    final StreamingEndpoint streamingEndpoint;
    if (ENDPOINT_SAMPLE.getValue().equals(endpointName)) {
        host = Constants.STREAM_HOST;
        final StatusesSampleEndpoint sse = new StatusesSampleEndpoint();
        streamingEndpoint = sse;
        if (languages != null) {
            sse.languages(languages);
        }
    } else if (ENDPOINT_FIREHOSE.getValue().equals(endpointName)) {
        host = Constants.STREAM_HOST;
        final StatusesFirehoseEndpoint firehoseEndpoint = new StatusesFirehoseEndpoint();
        streamingEndpoint = firehoseEndpoint;
        if (languages != null) {
            firehoseEndpoint.languages(languages);
        }
    } else if (ENDPOINT_FILTER.getValue().equals(endpointName)) {
        host = Constants.STREAM_HOST;
        final StatusesFilterEndpoint filterEndpoint = new StatusesFilterEndpoint();
        final String followingString = context.getProperty(FOLLOWING).getValue();
        final List<Long> followingIds;
        if (followingString == null) {
            followingIds = Collections.emptyList();
        } else {
            followingIds = new ArrayList<>();
            for (final String split : followingString.split(",")) {
                final Long id = Long.parseLong(split.trim());
                followingIds.add(id);
            }
        }
        final String termString = context.getProperty(TERMS).getValue();
        final List<String> terms;
        if (termString == null) {
            terms = Collections.emptyList();
        } else {
            terms = new ArrayList<>();
            for (final String split : termString.split(",")) {
                terms.add(split.trim());
            }
        }
        if (!terms.isEmpty()) {
            filterEndpoint.trackTerms(terms);
        }
        if (!followingIds.isEmpty()) {
            filterEndpoint.followings(followingIds);
        }
        if (languages != null) {
            filterEndpoint.languages(languages);
        }
        final String locationString = context.getProperty(LOCATIONS).getValue();
        final List<Location> locations;
        if (locationString == null) {
            locations = Collections.emptyList();
        } else {
            locations = LocationUtil.parseLocations(locationString);
        }
        if (!locations.isEmpty()) {
            filterEndpoint.locations(locations);
        }
        streamingEndpoint = filterEndpoint;
    } else {
        throw new AssertionError("Endpoint was invalid value: " + endpointName);
    }
    clientBuilder.hosts(host).endpoint(streamingEndpoint);
    client = clientBuilder.build();
    client.connect();
}
Also used : OAuth1(com.twitter.hbc.httpclient.auth.OAuth1) StreamingEndpoint(com.twitter.hbc.core.endpoint.StreamingEndpoint) Authentication(com.twitter.hbc.httpclient.auth.Authentication) StatusesSampleEndpoint(com.twitter.hbc.core.endpoint.StatusesSampleEndpoint) StatusesFirehoseEndpoint(com.twitter.hbc.core.endpoint.StatusesFirehoseEndpoint) StringDelimitedProcessor(com.twitter.hbc.core.processor.StringDelimitedProcessor) StatusesFilterEndpoint(com.twitter.hbc.core.endpoint.StatusesFilterEndpoint) ClientBuilder(com.twitter.hbc.ClientBuilder) Location(com.twitter.hbc.core.endpoint.Location) OnScheduled(org.apache.nifi.annotation.lifecycle.OnScheduled)

Example 2 with StringDelimitedProcessor

use of com.twitter.hbc.core.processor.StringDelimitedProcessor in project ignite by apache.

the class TwitterStreamer method buildClient.

/**
 * @param tweetQueue tweet Queue.
 * @param hosts Hostes.
 * @param endpoint Endpoint.
 * @return Client.
 */
protected Client buildClient(BlockingQueue<String> tweetQueue, HttpHosts hosts, StreamingEndpoint endpoint) {
    Authentication authentication = new OAuth1(oAuthSettings.getConsumerKey(), oAuthSettings.getConsumerSecret(), oAuthSettings.getAccessToken(), oAuthSettings.getAccessTokenSecret());
    ClientBuilder builder = new ClientBuilder().name("Ignite-Twitter-Client").hosts(hosts).authentication(authentication).endpoint(endpoint).processor(new StringDelimitedProcessor(tweetQueue));
    return builder.build();
}
Also used : OAuth1(com.twitter.hbc.httpclient.auth.OAuth1) Authentication(com.twitter.hbc.httpclient.auth.Authentication) StringDelimitedProcessor(com.twitter.hbc.core.processor.StringDelimitedProcessor) ClientBuilder(com.twitter.hbc.ClientBuilder)

Aggregations

ClientBuilder (com.twitter.hbc.ClientBuilder)2 StringDelimitedProcessor (com.twitter.hbc.core.processor.StringDelimitedProcessor)2 Authentication (com.twitter.hbc.httpclient.auth.Authentication)2 OAuth1 (com.twitter.hbc.httpclient.auth.OAuth1)2 Location (com.twitter.hbc.core.endpoint.Location)1 StatusesFilterEndpoint (com.twitter.hbc.core.endpoint.StatusesFilterEndpoint)1 StatusesFirehoseEndpoint (com.twitter.hbc.core.endpoint.StatusesFirehoseEndpoint)1 StatusesSampleEndpoint (com.twitter.hbc.core.endpoint.StatusesSampleEndpoint)1 StreamingEndpoint (com.twitter.hbc.core.endpoint.StreamingEndpoint)1 OnScheduled (org.apache.nifi.annotation.lifecycle.OnScheduled)1