Search in sources :

Example 41 with Response

use of org.apache.apex.shaded.ning19.com.ning.http.client.Response in project Singularity by HubSpot.

the class HttpLocalDownloadServiceFetcher method downloadFiles.

@Override
public void downloadFiles(List<? extends S3Artifact> s3Artifacts, SingularityExecutorTask task) throws InterruptedException {
    final List<FutureHolder> futures = Lists.newArrayListWithCapacity(s3Artifacts.size());
    for (S3Artifact s3Artifact : s3Artifacts) {
        String destination = task.getArtifactPath(s3Artifact, task.getTaskDefinition().getTaskDirectoryPath()).toString();
        ArtifactDownloadRequest artifactDownloadRequest = new ArtifactDownloadRequest(destination, s3Artifact, Optional.of(executorConfiguration.getLocalDownloadServiceTimeoutMillis()));
        task.getLog().debug("Requesting {} from {}", artifactDownloadRequest, localDownloadUri);
        BoundRequestBuilder postRequestBldr = httpClient.preparePost(localDownloadUri);
        try {
            postRequestBldr.setBody(objectMapper.writeValueAsBytes(artifactDownloadRequest));
        } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
        }
        try {
            ListenableFuture<Response> future = httpClient.executeRequest(postRequestBldr.build());
            futures.add(new FutureHolder(future, System.currentTimeMillis(), s3Artifact));
        } catch (Throwable t) {
            throw new RuntimeException(t);
        }
    }
    for (FutureHolder future : futures) {
        Response response = future.getResponse();
        task.getLog().debug("Future for {} got status code {} after {}", future.s3Artifact.getName(), response.getStatusCode(), JavaUtils.duration(future.start));
        if (response.getStatusCode() != 200) {
            throw new IllegalStateException("Got status code:" + response.getStatusCode());
        }
    }
}
Also used : Response(com.ning.http.client.Response) BoundRequestBuilder(com.ning.http.client.AsyncHttpClient.BoundRequestBuilder) ArtifactDownloadRequest(com.hubspot.singularity.s3.base.ArtifactDownloadRequest) S3Artifact(com.hubspot.deploy.S3Artifact) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Example 42 with Response

use of org.apache.apex.shaded.ning19.com.ning.http.client.Response in project Singularity by HubSpot.

the class SandboxManager method read.

@SuppressWarnings("deprecation")
public Optional<MesosFileChunkObject> read(String hostname, String fullPath, Optional<Long> offset, Optional<Long> length) throws AgentNotFoundException {
    try {
        final AsyncHttpClient.BoundRequestBuilder builder = asyncHttpClient.prepareGet(String.format("http://%s:5051/files/read", hostname)).addQueryParam("path", fullPath);
        builder.setRequestTimeout((int) configuration.getSandboxHttpTimeoutMillis());
        if (offset.isPresent()) {
            builder.addQueryParam("offset", offset.get().toString());
        }
        if (length.isPresent()) {
            builder.addQueryParam("length", length.get().toString());
        }
        final Response response = builder.execute().get();
        if (response.getStatusCode() == 404) {
            return Optional.empty();
        }
        if (response.getStatusCode() != 200) {
            throw new RuntimeException(String.format("Got HTTP %s from Mesos agent", response.getStatusCode()));
        }
        return Optional.of(parseResponseBody(response));
    } catch (Exception e) {
        if (Throwables.getCausalChain(e).stream().anyMatch(t -> t instanceof UnknownHostException || t instanceof ConnectException)) {
            throw new AgentNotFoundException(e);
        } else {
            throw new RuntimeException(e);
        }
    }
}
Also used : Response(com.ning.http.client.Response) Inject(com.google.inject.Inject) Collection(java.util.Collection) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) Throwables(com.google.common.base.Throwables) Singularity(com.hubspot.singularity.Singularity) IOException(java.io.IOException) Reader(java.io.Reader) CharsetDecoder(java.nio.charset.CharsetDecoder) UnknownHostException(java.net.UnknownHostException) ByteBuffer(java.nio.ByteBuffer) StandardCharsets(java.nio.charset.StandardCharsets) MesosFileObject(com.hubspot.mesos.json.MesosFileObject) CodingErrorAction(java.nio.charset.CodingErrorAction) AsyncHttpClient(com.ning.http.client.AsyncHttpClient) Optional(java.util.Optional) ConnectException(java.net.ConnectException) VisibleForTesting(com.google.common.annotations.VisibleForTesting) CharSource(com.google.common.io.CharSource) MesosFileChunkObject(com.hubspot.mesos.json.MesosFileChunkObject) TypeReference(com.fasterxml.jackson.core.type.TypeReference) Collections(java.util.Collections) Singleton(com.google.inject.Singleton) SingularityConfiguration(com.hubspot.singularity.config.SingularityConfiguration) Response(com.ning.http.client.Response) UnknownHostException(java.net.UnknownHostException) IOException(java.io.IOException) UnknownHostException(java.net.UnknownHostException) ConnectException(java.net.ConnectException) AsyncHttpClient(com.ning.http.client.AsyncHttpClient) ConnectException(java.net.ConnectException)

Example 43 with Response

use of org.apache.apex.shaded.ning19.com.ning.http.client.Response in project Singularity by HubSpot.

the class SingularityWebhookSender method executeWebhookAsync.

// TODO handle retries, errors.
private <T> CompletableFuture<Response> executeWebhookAsync(String uri, Object payload, AbstractSingularityWebhookAsyncHandler<T> handler) {
    LOG.trace("Sending {} to {}", payload, uri);
    BoundRequestBuilder postRequest = http.preparePost(uri);
    postRequest.setHeader(HttpHeaders.CONTENT_TYPE, "application/json");
    try {
        postRequest.setBody(objectMapper.writeValueAsBytes(payload));
    } catch (JsonProcessingException e) {
        throw new RuntimeException(e);
    }
    CompletableFuture<Response> webhookFuture = new CompletableFuture<>();
    try {
        handler.setCompletableFuture(webhookFuture);
        postRequest.execute(handler);
    } catch (Throwable t) {
        LOG.warn("Couldn't execute webhook to {}", uri, t);
        if (handler.shouldDeleteUpdateDueToQueueAboveCapacity()) {
            handler.deleteWebhookUpdate();
        }
        webhookFuture.completeExceptionally(t);
    }
    return webhookFuture;
}
Also used : Response(com.ning.http.client.Response) BoundRequestBuilder(com.ning.http.client.AsyncHttpClient.BoundRequestBuilder) CompletableFuture(java.util.concurrent.CompletableFuture) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Example 44 with Response

use of org.apache.apex.shaded.ning19.com.ning.http.client.Response in project Singularity by HubSpot.

the class AbstractLeaderAwareResource method maybeProxyToLeader.

protected <T, Q> T maybeProxyToLeader(HttpServletRequest request, Class<T> clazz, Q body, Supplier<T> runnable) {
    if (leaderLatch.hasLeadership()) {
        return runnable.get();
    }
    String leaderUri;
    try {
        leaderUri = leaderLatch.getLeader().getId();
    } catch (Exception e) {
        throw new RuntimeException("Could not get leader uri to proxy request");
    }
    if (leaderUri.equals(leaderLatch.getId())) {
        LOG.warn("Got own leader id when not the leader! There is likely no leader, will not proxy");
        return runnable.get();
    }
    String url = "http://" + leaderUri + request.getContextPath() + request.getPathInfo();
    LOG.debug("Not the leader, proxying request to {}", url);
    BoundRequestBuilder requestBuilder;
    switch(request.getMethod().toUpperCase()) {
        case "POST":
            requestBuilder = httpClient.preparePost(url);
            // necessary to ensure POST requests without bodies work (will hit 30 second timeout otherwise)
            requestBuilder.setBody("");
            break;
        case "PUT":
            requestBuilder = httpClient.preparePut(url);
            break;
        case "DELETE":
            requestBuilder = httpClient.prepareDelete(url);
            break;
        default:
            throw new WebApplicationException(String.format("Not meant to proxy request of method %s", request.getMethod()), 400);
    }
    try {
        if (body != null) {
            requestBuilder.setBody(objectMapper.writeValueAsBytes(body));
            LOG.trace("Added body {} to reqeust", body);
        }
    } catch (JsonProcessingException jpe) {
        LOG.error("Could not write body from object {}", body);
        throw new WebApplicationException(jpe, 500);
    }
    copyHeadersAndParams(requestBuilder, request);
    Request httpRequest = requestBuilder.build();
    Response response;
    try {
        LOG.trace("Sending request to leader: {}", httpRequest);
        response = httpClient.executeRequest(httpRequest).get();
    } catch (ExecutionException | InterruptedException e) {
        LOG.error("Could not proxy request {} to leader", e);
        throw new WebApplicationException(e, 500);
    }
    try {
        if (clazz.isAssignableFrom(javax.ws.rs.core.Response.class)) {
            return (T) javax.ws.rs.core.Response.status(response.getStatusCode()).entity(response.getResponseBody()).build();
        }
        if (response.getStatusCode() > 399) {
            throw new WebApplicationException(response.getResponseBody(Charsets.UTF_8.toString()), response.getStatusCode());
        } else {
            return objectMapper.readValue(response.getResponseBodyAsStream(), clazz);
        }
    } catch (IOException ioe) {
        String message = String.format("Request to leader succeeded with status %s, but could not interpret response", response.getStatusCode());
        LOG.error(message, ioe);
        throw new WebApplicationException(message, ioe, 500);
    }
}
Also used : WebApplicationException(javax.ws.rs.WebApplicationException) Request(com.ning.http.client.Request) HttpServletRequest(javax.servlet.http.HttpServletRequest) IOException(java.io.IOException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) IOException(java.io.IOException) ExecutionException(java.util.concurrent.ExecutionException) WebApplicationException(javax.ws.rs.WebApplicationException) Response(com.ning.http.client.Response) BoundRequestBuilder(com.ning.http.client.AsyncHttpClient.BoundRequestBuilder) ExecutionException(java.util.concurrent.ExecutionException) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException)

Aggregations

Response (com.ning.http.client.Response)42 IOException (java.io.IOException)14 Test (org.junit.Test)11 AsyncHttpClient (com.ning.http.client.AsyncHttpClient)9 JsonProcessingException (com.fasterxml.jackson.core.JsonProcessingException)6 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)6 BoundRequestBuilder (com.ning.http.client.AsyncHttpClient.BoundRequestBuilder)6 Map (java.util.Map)6 CompletableFuture (java.util.concurrent.CompletableFuture)6 Request (com.ning.http.client.Request)5 ExecutionException (java.util.concurrent.ExecutionException)5 AsyncHttpClientConfig (com.ning.http.client.AsyncHttpClientConfig)4 RequestBuilder (com.ning.http.client.RequestBuilder)4 DataProvider (com.tngtech.java.junit.dataprovider.DataProvider)4 HttpMethod (io.netty.handler.codec.http.HttpMethod)4 TimeoutException (java.util.concurrent.TimeoutException)4 NodesInfoResponse (org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse)4 Span (com.nike.wingtips.Span)3 AsyncCompletionHandler (com.ning.http.client.AsyncCompletionHandler)3 NettyAsyncHttpProvider (com.ning.http.client.providers.netty.NettyAsyncHttpProvider)3