Search in sources :

Example 1 with UnixLocalDownloadServiceFetcher

use of com.hubspot.singularity.executor.task.UnixLocalDownloadServiceFetcher in project Singularity by HubSpot.

the class SingularityExecutorModule method provideDownloadFetcher.

@Provides
@Singleton
public LocalDownloadServiceFetcher provideDownloadFetcher(SingularityS3Configuration s3Configuration, SingularityExecutorConfiguration executorConfiguration, ObjectMapper objectMapper) {
    if (s3Configuration.getLocalDownloadSocket().isPresent()) {
        HttpClient httpClient = new HttpClient(new HttpClientTransportOverUnixSockets(s3Configuration.getLocalDownloadSocket().get()), null);
        return new UnixLocalDownloadServiceFetcher(httpClient, objectMapper, executorConfiguration, s3Configuration);
    } else {
        AsyncHttpClientConfig.Builder configBldr = new AsyncHttpClientConfig.Builder();
        configBldr.setRequestTimeout((int) executorConfiguration.getLocalDownloadServiceTimeoutMillis());
        configBldr.setPooledConnectionIdleTimeout((int) executorConfiguration.getLocalDownloadServiceTimeoutMillis());
        configBldr.addRequestFilter(new ThrottleRequestFilter(executorConfiguration.getLocalDownloadServiceMaxConnections()));
        return new HttpLocalDownloadServiceFetcher(new AsyncHttpClient(configBldr.build()), objectMapper, executorConfiguration, s3Configuration);
    }
}
Also used : ThrottleRequestFilter(com.ning.http.client.extra.ThrottleRequestFilter) HttpLocalDownloadServiceFetcher(com.hubspot.singularity.executor.task.HttpLocalDownloadServiceFetcher) UnixLocalDownloadServiceFetcher(com.hubspot.singularity.executor.task.UnixLocalDownloadServiceFetcher) HttpClient(org.eclipse.jetty.client.HttpClient) AsyncHttpClient(com.ning.http.client.AsyncHttpClient) Builder(com.spotify.docker.client.DefaultDockerClient.Builder) AsyncHttpClientConfig(com.ning.http.client.AsyncHttpClientConfig) HttpClientTransportOverUnixSockets(org.eclipse.jetty.unixsocket.client.HttpClientTransportOverUnixSockets) AsyncHttpClient(com.ning.http.client.AsyncHttpClient) Singleton(com.google.inject.Singleton) Provides(com.google.inject.Provides)

Aggregations

Provides (com.google.inject.Provides)1 Singleton (com.google.inject.Singleton)1 HttpLocalDownloadServiceFetcher (com.hubspot.singularity.executor.task.HttpLocalDownloadServiceFetcher)1 UnixLocalDownloadServiceFetcher (com.hubspot.singularity.executor.task.UnixLocalDownloadServiceFetcher)1 AsyncHttpClient (com.ning.http.client.AsyncHttpClient)1 AsyncHttpClientConfig (com.ning.http.client.AsyncHttpClientConfig)1 ThrottleRequestFilter (com.ning.http.client.extra.ThrottleRequestFilter)1 Builder (com.spotify.docker.client.DefaultDockerClient.Builder)1 HttpClient (org.eclipse.jetty.client.HttpClient)1 HttpClientTransportOverUnixSockets (org.eclipse.jetty.unixsocket.client.HttpClientTransportOverUnixSockets)1