Search in sources :

Example 51 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project cdap by caskdata.

the class DefaultStreamManager method send.

@Override
public void send(File file, String contentType) throws Exception {
    String path = String.format("/v3/namespaces/%s/streams/%s/batch", streamId.getNamespaceId(), streamId.getId());
    HttpRequest request = new DefaultHttpRequest(HttpVersion.HTTP_1_1, HttpMethod.POST, path);
    request.headers().set(HttpHeaderNames.CONTENT_TYPE, contentType);
    HttpUtil.setTransferEncodingChunked(request, true);
    final MockResponder responder = new MockResponder();
    final BodyConsumer bodyConsumer = streamHandler.batch(request, responder, streamId.getNamespaceId(), streamId.getId());
    Preconditions.checkNotNull(bodyConsumer, "BodyConsumer from stream batch load call should not be null");
    ByteStreams.readBytes(Files.newInputStreamSupplier(file), new ByteProcessor<BodyConsumer>() {

        @Override
        public boolean processBytes(byte[] buf, int off, int len) throws IOException {
            bodyConsumer.chunk(Unpooled.wrappedBuffer(buf, off, len), responder);
            return true;
        }

        @Override
        public BodyConsumer getResult() {
            bodyConsumer.finished(responder);
            return bodyConsumer;
        }
    });
    Preconditions.checkState(HttpResponseStatus.OK.equals(responder.getStatus()), "Failed to load events to stream %s in batch", streamId);
}
Also used : DefaultFullHttpRequest(io.netty.handler.codec.http.DefaultFullHttpRequest) DefaultHttpRequest(io.netty.handler.codec.http.DefaultHttpRequest) HttpRequest(io.netty.handler.codec.http.HttpRequest) FullHttpRequest(io.netty.handler.codec.http.FullHttpRequest) DefaultHttpRequest(io.netty.handler.codec.http.DefaultHttpRequest) MockResponder(co.cask.cdap.internal.MockResponder) BodyConsumer(co.cask.http.BodyConsumer) IOException(java.io.IOException)

Example 52 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project cdap by caskdata.

the class AuditLogHandler method channelRead.

@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
    // When a request is forwarded to the internal CDAP service
    if (msg instanceof HttpRequest) {
        HttpRequest request = (HttpRequest) msg;
        // Extra configurations for audit log
        AuditLogConfig logConfig = AUDIT_LOG_LOOKUP_METHOD.contains(request.method()) ? RouterAuditLookUp.getInstance().getAuditLogContent(request.uri(), request.method()) : null;
        if (logConfig == null) {
            logEntry = new AuditLogEntry(request, Networks.getIP(ctx.channel().remoteAddress()));
        } else {
            logEntry = new AuditLogEntry(request, Networks.getIP(ctx.channel().remoteAddress()), logConfig.getHeaderNames());
            logRequestBody = logConfig.isLogRequestBody();
            logResponseBody = logConfig.isLogResponseBody();
        }
    } else if (msg instanceof HttpContent && logEntry != null) {
        ByteBuf content = ((HttpContent) msg).content();
        if (logRequestBody && content.isReadable()) {
            logEntry.appendRequestBody(content.toString(StandardCharsets.UTF_8));
        }
    }
    ctx.fireChannelRead(msg);
}
Also used : HttpRequest(io.netty.handler.codec.http.HttpRequest) AuditLogEntry(co.cask.cdap.common.logging.AuditLogEntry) ByteBuf(io.netty.buffer.ByteBuf) AuditLogConfig(co.cask.cdap.common.logging.AuditLogConfig) HttpContent(io.netty.handler.codec.http.HttpContent) LastHttpContent(io.netty.handler.codec.http.LastHttpContent)

Example 53 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project cdap by caskdata.

the class HttpRequestRouter method channelRead.

@Override
public void channelRead(final ChannelHandlerContext ctx, final Object msg) throws Exception {
    try {
        final Channel inboundChannel = ctx.channel();
        ChannelFutureListener writeCompletedListener = getFailureResponseListener(inboundChannel);
        if (msg instanceof HttpRequest) {
            inflightRequests++;
            if (inflightRequests != 1) {
                // At the end of the first response, we'll respond to all the other requests as well
                return;
            }
            // Disable read until sending of this request object is completed successfully
            // This is for handling the initial connection delay
            inboundChannel.config().setAutoRead(false);
            writeCompletedListener = new ChannelFutureListener() {

                @Override
                public void operationComplete(ChannelFuture future) throws Exception {
                    if (future.isSuccess()) {
                        inboundChannel.config().setAutoRead(true);
                    } else {
                        getFailureResponseListener(inboundChannel).operationComplete(future);
                    }
                }
            };
            HttpRequest request = (HttpRequest) msg;
            currentMessageSender = getMessageSender(inboundChannel, getDiscoverable(request, (InetSocketAddress) inboundChannel.localAddress()));
        }
        if (inflightRequests == 1 && currentMessageSender != null) {
            ReferenceCountUtil.retain(msg);
            currentMessageSender.send(msg, writeCompletedListener);
        }
    } finally {
        ReferenceCountUtil.release(msg);
    }
}
Also used : HttpRequest(io.netty.handler.codec.http.HttpRequest) ChannelFuture(io.netty.channel.ChannelFuture) SocketChannel(io.netty.channel.socket.SocketChannel) NioSocketChannel(io.netty.channel.socket.nio.NioSocketChannel) Channel(io.netty.channel.Channel) ChannelFutureListener(io.netty.channel.ChannelFutureListener) SSLException(javax.net.ssl.SSLException) HandlerException(co.cask.cdap.common.HandlerException) ClosedChannelException(java.nio.channels.ClosedChannelException)

Example 54 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project sidewinder by srotya.

the class HTTPDataPointDecoder method channelRead0.

@Override
protected void channelRead0(ChannelHandlerContext ctx, Object msg) throws Exception {
    try {
        if (ResourceMonitor.getInstance().isReject()) {
            logger.warning("Write rejected, insufficient memory");
            if (writeResponse(request, ctx)) {
                ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(ChannelFutureListener.CLOSE);
            }
            return;
        }
        if (msg instanceof HttpRequest) {
            HttpRequest request = this.request = (HttpRequest) msg;
            if (HttpUtil.is100ContinueExpected(request)) {
                send100Continue(ctx);
            }
            QueryStringDecoder queryStringDecoder = new QueryStringDecoder(request.uri());
            path = queryStringDecoder.path();
            Map<String, List<String>> params = queryStringDecoder.parameters();
            if (!params.isEmpty()) {
                for (Entry<String, List<String>> p : params.entrySet()) {
                    String key = p.getKey();
                    if (key.equalsIgnoreCase("db")) {
                        dbName = p.getValue().get(0);
                    }
                }
            }
            if (path != null && path.contains("query")) {
                Gson gson = new Gson();
                JsonObject obj = new JsonObject();
                JsonArray ary = new JsonArray();
                ary.add(new JsonObject());
                obj.add("results", ary);
                responseString.append(gson.toJson(obj));
            }
        }
        if (msg instanceof HttpContent) {
            HttpContent httpContent = (HttpContent) msg;
            ByteBuf byteBuf = httpContent.content();
            if (byteBuf.isReadable()) {
                requestBuffer.append(byteBuf.toString(CharsetUtil.UTF_8));
            }
            if (msg instanceof LastHttpContent) {
                if (dbName == null) {
                    responseString.append("Invalid database null");
                    logger.severe("Invalid database null");
                } else {
                    String payload = requestBuffer.toString();
                    logger.fine("Request:" + payload);
                    List<Point> dps = InfluxDecoder.pointsFromString(dbName, payload);
                    meter.inc(dps.size());
                    for (Point dp : dps) {
                        try {
                            engine.writeDataPoint(dp);
                            logger.fine("Accepted:" + dp + "\t" + new Date(dp.getTimestamp()));
                        } catch (IOException e) {
                            logger.fine("Dropped:" + dp + "\t" + e.getMessage());
                            responseString.append("Dropped:" + dp);
                        }
                    }
                }
                if (writeResponse(request, ctx)) {
                    ctx.writeAndFlush(Unpooled.EMPTY_BUFFER).addListener(ChannelFutureListener.CLOSE);
                }
            }
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
}
Also used : HttpRequest(io.netty.handler.codec.http.HttpRequest) Gson(com.google.gson.Gson) JsonObject(com.google.gson.JsonObject) Point(com.srotya.sidewinder.core.rpc.Point) IOException(java.io.IOException) ByteBuf(io.netty.buffer.ByteBuf) LastHttpContent(io.netty.handler.codec.http.LastHttpContent) Date(java.util.Date) IOException(java.io.IOException) QueryStringDecoder(io.netty.handler.codec.http.QueryStringDecoder) JsonArray(com.google.gson.JsonArray) List(java.util.List) LastHttpContent(io.netty.handler.codec.http.LastHttpContent) HttpContent(io.netty.handler.codec.http.HttpContent)

Example 55 with HttpRequest

use of org.apache.flink.shaded.netty4.io.netty.handler.codec.http.HttpRequest in project selenium_java by sergueik.

the class UtilsTest method isChangingRequest.

@Test
public void isChangingRequest() throws URISyntaxException {
    URI uri = new URI("http://tenniskafe.com");
    HttpHeaders httpHeaders = mock(HttpHeaders.class);
    when(httpHeaders.get("Accept")).thenReturn("application/html");
    HttpRequest httpRequest = mock(HttpRequest.class);
    when(httpRequest.getUri()).thenReturn("http://google.com");
    when(httpRequest.headers()).thenReturn(httpHeaders);
    assertTrue(Utils.isUrlChangingRequest(uri, httpRequest));
}
Also used : HttpRequest(io.netty.handler.codec.http.HttpRequest) HttpHeaders(io.netty.handler.codec.http.HttpHeaders) URI(java.net.URI) Test(org.junit.Test)

Aggregations

HttpRequest (io.netty.handler.codec.http.HttpRequest)332 Test (org.junit.Test)115 DefaultHttpRequest (io.netty.handler.codec.http.DefaultHttpRequest)111 DefaultFullHttpRequest (io.netty.handler.codec.http.DefaultFullHttpRequest)102 FullHttpRequest (io.netty.handler.codec.http.FullHttpRequest)71 EmbeddedChannel (io.netty.channel.embedded.EmbeddedChannel)67 HttpResponse (io.netty.handler.codec.http.HttpResponse)65 FullHttpResponse (io.netty.handler.codec.http.FullHttpResponse)51 HttpHeaders (io.netty.handler.codec.http.HttpHeaders)45 LastHttpContent (io.netty.handler.codec.http.LastHttpContent)36 ByteBuf (io.netty.buffer.ByteBuf)35 HttpContent (io.netty.handler.codec.http.HttpContent)34 Test (org.junit.jupiter.api.Test)34 Channel (io.netty.channel.Channel)31 HttpMethod (io.netty.handler.codec.http.HttpMethod)31 URI (java.net.URI)30 DefaultHttpHeaders (io.netty.handler.codec.http.DefaultHttpHeaders)28 DefaultFullHttpResponse (io.netty.handler.codec.http.DefaultFullHttpResponse)26 IOException (java.io.IOException)25 DefaultLastHttpContent (io.netty.handler.codec.http.DefaultLastHttpContent)22