Search in sources :

Example 81 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project flink by apache.

the class RestClient method sendRequest.

public <M extends MessageHeaders<R, P, U>, U extends MessageParameters, R extends RequestBody, P extends ResponseBody> CompletableFuture<P> sendRequest(String targetAddress, int targetPort, M messageHeaders, U messageParameters, R request, Collection<FileUpload> fileUploads, RestAPIVersion apiVersion) throws IOException {
    Preconditions.checkNotNull(targetAddress);
    Preconditions.checkArgument(NetUtils.isValidHostPort(targetPort), "The target port " + targetPort + " is not in the range [0, 65535].");
    Preconditions.checkNotNull(messageHeaders);
    Preconditions.checkNotNull(request);
    Preconditions.checkNotNull(messageParameters);
    Preconditions.checkNotNull(fileUploads);
    Preconditions.checkState(messageParameters.isResolved(), "Message parameters were not resolved.");
    if (!messageHeaders.getSupportedAPIVersions().contains(apiVersion)) {
        throw new IllegalArgumentException(String.format("The requested version %s is not supported by the request (method=%s URL=%s). Supported versions are: %s.", apiVersion, messageHeaders.getHttpMethod(), messageHeaders.getTargetRestEndpointURL(), messageHeaders.getSupportedAPIVersions().stream().map(RestAPIVersion::getURLVersionPrefix).collect(Collectors.joining(","))));
    }
    String versionedHandlerURL = "/" + apiVersion.getURLVersionPrefix() + messageHeaders.getTargetRestEndpointURL();
    String targetUrl = MessageParameters.resolveUrl(versionedHandlerURL, messageParameters);
    LOG.debug("Sending request of class {} to {}:{}{}", request.getClass(), targetAddress, targetPort, targetUrl);
    // serialize payload
    StringWriter sw = new StringWriter();
    objectMapper.writeValue(sw, request);
    ByteBuf payload = Unpooled.wrappedBuffer(sw.toString().getBytes(ConfigConstants.DEFAULT_CHARSET));
    Request httpRequest = createRequest(targetAddress + ':' + targetPort, targetUrl, messageHeaders.getHttpMethod().getNettyHttpMethod(), payload, fileUploads);
    final JavaType responseType;
    final Collection<Class<?>> typeParameters = messageHeaders.getResponseTypeParameters();
    if (typeParameters.isEmpty()) {
        responseType = objectMapper.constructType(messageHeaders.getResponseClass());
    } else {
        responseType = objectMapper.getTypeFactory().constructParametricType(messageHeaders.getResponseClass(), typeParameters.toArray(new Class<?>[typeParameters.size()]));
    }
    return submitRequest(targetAddress, targetPort, httpRequest, responseType);
}
Also used : JavaType(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JavaType) StringWriter(java.io.StringWriter) HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) ByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf)

Example 82 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project flink by apache.

the class RestClient method submitRequest.

private <P extends ResponseBody> CompletableFuture<P> submitRequest(String targetAddress, int targetPort, Request httpRequest, JavaType responseType) {
    final ChannelFuture connectFuture = bootstrap.connect(targetAddress, targetPort);
    final CompletableFuture<Channel> channelFuture = new CompletableFuture<>();
    connectFuture.addListener((ChannelFuture future) -> {
        if (future.isSuccess()) {
            channelFuture.complete(future.channel());
        } else {
            channelFuture.completeExceptionally(future.cause());
        }
    });
    return channelFuture.thenComposeAsync(channel -> {
        ClientHandler handler = channel.pipeline().get(ClientHandler.class);
        CompletableFuture<JsonResponse> future;
        boolean success = false;
        try {
            if (handler == null) {
                throw new IOException("Netty pipeline was not properly initialized.");
            } else {
                httpRequest.writeTo(channel);
                future = handler.getJsonFuture();
                success = true;
            }
        } catch (IOException e) {
            future = FutureUtils.completedExceptionally(new ConnectionException("Could not write request.", e));
        } finally {
            if (!success) {
                channel.close();
            }
        }
        return future;
    }, executor).thenComposeAsync((JsonResponse rawResponse) -> parseResponse(rawResponse, responseType), executor);
}
Also used : ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) SimpleChannelInboundHandler(org.apache.flink.shaded.netty4.io.netty.channel.SimpleChannelInboundHandler) MessageParameters(org.apache.flink.runtime.rest.messages.MessageParameters) FullHttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.FullHttpResponse) ChunkedWriteHandler(org.apache.flink.shaded.netty4.io.netty.handler.stream.ChunkedWriteHandler) OutboundChannelHandlerFactory(org.apache.flink.runtime.io.network.netty.OutboundChannelHandlerFactory) MemoryAttribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.MemoryAttribute) LoggerFactory(org.slf4j.LoggerFactory) ExceptionUtils(org.apache.flink.util.ExceptionUtils) JsonParser(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser) JsonNode(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JsonNode) NetUtils(org.apache.flink.util.NetUtils) EmptyMessageParameters(org.apache.flink.runtime.rest.messages.EmptyMessageParameters) EmptyRequestBody(org.apache.flink.runtime.rest.messages.EmptyRequestBody) HttpResponse(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponse) RequestBody(org.apache.flink.runtime.rest.messages.RequestBody) HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) RestClientException(org.apache.flink.runtime.rest.util.RestClientException) Path(java.nio.file.Path) ChannelHandlerContext(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) Attribute(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.Attribute) ExecutorThreadFactory(org.apache.flink.util.concurrent.ExecutorThreadFactory) RestConstants(org.apache.flink.runtime.rest.util.RestConstants) Collection(java.util.Collection) NioEventLoopGroup(org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoopGroup) HttpVersion(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpVersion) HttpObjectAggregator(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpObjectAggregator) IdleStateHandler(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateHandler) ServiceLoader(java.util.ServiceLoader) JavaType(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.JavaType) Preconditions(org.apache.flink.util.Preconditions) ByteBuf(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBuf) Collectors(java.util.stream.Collectors) List(java.util.List) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) HttpClientCodec(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpClientCodec) DefaultHttpDataFactory(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.DefaultHttpDataFactory) Optional(java.util.Optional) IdleStateEvent(org.apache.flink.shaded.netty4.io.netty.handler.timeout.IdleStateEvent) Time(org.apache.flink.api.common.time.Time) RestMapperUtils(org.apache.flink.runtime.rest.util.RestMapperUtils) Bootstrap(org.apache.flink.shaded.netty4.io.netty.bootstrap.Bootstrap) ChannelInitializer(org.apache.flink.shaded.netty4.io.netty.channel.ChannelInitializer) ConfigurationException(org.apache.flink.util.ConfigurationException) AtomicBoolean(java.util.concurrent.atomic.AtomicBoolean) SSLHandlerFactory(org.apache.flink.runtime.io.network.netty.SSLHandlerFactory) CompletableFuture(java.util.concurrent.CompletableFuture) ChannelHandler(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandler) MessageHeaders(org.apache.flink.runtime.rest.messages.MessageHeaders) HttpResponseStatus(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus) ArrayList(java.util.ArrayList) FutureUtils(org.apache.flink.util.concurrent.FutureUtils) ByteBufInputStream(org.apache.flink.shaded.netty4.io.netty.buffer.ByteBufInputStream) HttpHeaders(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpHeaders) REQUEST_ENTITY_TOO_LARGE(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpResponseStatus.REQUEST_ENTITY_TOO_LARGE) ObjectMapper(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper) ConfigConstants(org.apache.flink.configuration.ConfigConstants) RestOptions(org.apache.flink.configuration.RestOptions) RestAPIVersion(org.apache.flink.runtime.rest.versioning.RestAPIVersion) ChannelFuture(org.apache.flink.shaded.netty4.io.netty.channel.ChannelFuture) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody) ChannelOption(org.apache.flink.shaded.netty4.io.netty.channel.ChannelOption) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) AutoCloseableAsync(org.apache.flink.util.AutoCloseableAsync) Files(java.nio.file.Files) Executor(java.util.concurrent.Executor) StringWriter(java.io.StringWriter) Configuration(org.apache.flink.configuration.Configuration) Unpooled(org.apache.flink.shaded.netty4.io.netty.buffer.Unpooled) IOException(java.io.IOException) JsonProcessingException(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException) File(java.io.File) VisibleForTesting(org.apache.flink.annotation.VisibleForTesting) TimeUnit(java.util.concurrent.TimeUnit) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) ResponseBody(org.apache.flink.runtime.rest.messages.ResponseBody) TooLongFrameException(org.apache.flink.shaded.netty4.io.netty.handler.codec.TooLongFrameException) HttpPostRequestEncoder(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.multipart.HttpPostRequestEncoder) HttpMethod(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpMethod) Comparator(java.util.Comparator) Collections(java.util.Collections) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) InputStream(java.io.InputStream) CompletableFuture(java.util.concurrent.CompletableFuture) SocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.SocketChannel) NioSocketChannel(org.apache.flink.shaded.netty4.io.netty.channel.socket.nio.NioSocketChannel) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) IOException(java.io.IOException)

Example 83 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project flink by apache.

the class FileUploadHandler method handleError.

private void handleError(ChannelHandlerContext ctx, String errorMessage, HttpResponseStatus responseStatus, @Nullable Throwable e) {
    HttpRequest tmpRequest = currentHttpRequest;
    deleteUploadedFiles();
    reset();
    LOG.warn(errorMessage, e);
    HandlerUtils.sendErrorResponse(ctx, tmpRequest, new ErrorResponseBody(errorMessage), responseStatus, Collections.emptyMap());
    ReferenceCountUtil.release(tmpRequest);
}
Also used : HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) ErrorResponseBody(org.apache.flink.runtime.rest.messages.ErrorResponseBody)

Example 84 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project flink by apache.

the class Prio0InboundChannelHandlerFactory method createHandler.

@Override
public Optional<ChannelHandler> createHandler(Configuration configuration, Map<String, String> responseHeaders) throws ConfigurationException {
    String redirectFromUrl = configuration.getString(REDIRECT_FROM_URL);
    String redirectToUrl = configuration.getString(REDIRECT_TO_URL);
    if (!redirectFromUrl.isEmpty() && !redirectToUrl.isEmpty()) {
        return Optional.of(new ChannelInboundHandlerAdapter() {

            @Override
            public void channelRead(ChannelHandlerContext ctx, Object msg) {
                if (msg instanceof HttpRequest) {
                    HttpRequest httpRequest = (HttpRequest) msg;
                    if (httpRequest.uri().equals(redirectFromUrl)) {
                        httpRequest.setUri(redirectToUrl);
                    }
                }
                ctx.fireChannelRead(ReferenceCountUtil.retain(msg));
            }
        });
    }
    return Optional.empty();
}
Also used : HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) ChannelHandlerContext(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext) ChannelInboundHandlerAdapter(org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter)

Example 85 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project flink by apache.

the class AbstractHandlerTest method testFileCleanup.

@Test
public void testFileCleanup() throws Exception {
    final Path dir = temporaryFolder.newFolder().toPath();
    final Path file = dir.resolve("file");
    Files.createFile(file);
    RestfulGateway mockRestfulGateway = new TestingRestfulGateway.Builder().build();
    final GatewayRetriever<RestfulGateway> mockGatewayRetriever = () -> CompletableFuture.completedFuture(mockRestfulGateway);
    CompletableFuture<Void> requestProcessingCompleteFuture = new CompletableFuture<>();
    TestHandler handler = new TestHandler(requestProcessingCompleteFuture, mockGatewayRetriever);
    RouteResult<?> routeResult = new RouteResult<>("", "", Collections.emptyMap(), Collections.emptyMap(), "");
    HttpRequest request = new DefaultFullHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.GET, TestHandler.TestHeaders.INSTANCE.getTargetRestEndpointURL(), Unpooled.wrappedBuffer(new byte[0]));
    RoutedRequest<?> routerRequest = new RoutedRequest<>(routeResult, request);
    Attribute<FileUploads> attribute = new SimpleAttribute();
    attribute.set(new FileUploads(dir));
    Channel channel = mock(Channel.class);
    when(channel.attr(any(AttributeKey.class))).thenReturn(attribute);
    ChannelHandlerContext context = mock(ChannelHandlerContext.class);
    when(context.channel()).thenReturn(channel);
    handler.respondAsLeader(context, routerRequest, mockRestfulGateway);
    // the (asynchronous) request processing is not yet complete so the files should still exist
    Assert.assertTrue(Files.exists(file));
    requestProcessingCompleteFuture.complete(null);
    Assert.assertFalse(Files.exists(file));
}
Also used : Path(java.nio.file.Path) HttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) DefaultFullHttpRequest(org.apache.flink.shaded.netty4.io.netty.handler.codec.http.DefaultFullHttpRequest) Channel(org.apache.flink.shaded.netty4.io.netty.channel.Channel) ChannelHandlerContext(org.apache.flink.shaded.netty4.io.netty.channel.ChannelHandlerContext) AttributeKey(org.apache.flink.shaded.netty4.io.netty.util.AttributeKey) CompletableFuture(java.util.concurrent.CompletableFuture) RouteResult(org.apache.flink.runtime.rest.handler.router.RouteResult) RoutedRequest(org.apache.flink.runtime.rest.handler.router.RoutedRequest) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) RestfulGateway(org.apache.flink.runtime.webmonitor.RestfulGateway) TestingRestfulGateway(org.apache.flink.runtime.webmonitor.TestingRestfulGateway) Test(org.junit.Test)

Aggregations

HttpRequest (io.netty.handler.codec.http.HttpRequest)332 Test (org.junit.Test)115 DefaultHttpRequest (io.netty.handler.codec.http.DefaultHttpRequest)111 DefaultFullHttpRequest (io.netty.handler.codec.http.DefaultFullHttpRequest)102 FullHttpRequest (io.netty.handler.codec.http.FullHttpRequest)71 EmbeddedChannel (io.netty.channel.embedded.EmbeddedChannel)67 HttpResponse (io.netty.handler.codec.http.HttpResponse)65 FullHttpResponse (io.netty.handler.codec.http.FullHttpResponse)51 HttpHeaders (io.netty.handler.codec.http.HttpHeaders)45 LastHttpContent (io.netty.handler.codec.http.LastHttpContent)36 ByteBuf (io.netty.buffer.ByteBuf)35 HttpContent (io.netty.handler.codec.http.HttpContent)34 Test (org.junit.jupiter.api.Test)34 Channel (io.netty.channel.Channel)31 HttpMethod (io.netty.handler.codec.http.HttpMethod)31 URI (java.net.URI)30 DefaultHttpHeaders (io.netty.handler.codec.http.DefaultHttpHeaders)28 DefaultFullHttpResponse (io.netty.handler.codec.http.DefaultFullHttpResponse)26 IOException (java.io.IOException)25 DefaultLastHttpContent (io.netty.handler.codec.http.DefaultLastHttpContent)22