Search in sources :

Example 36 with XFuture

use of org.webpieces.util.futures.XFuture in project webpieces by deanhiller.

the class Layer2Http11Handler method processWithBackpressure.

public XFuture<Void> processWithBackpressure(FrontendSocketImpl socket, int newDataSize, int numBytesRead) {
    Memento state = socket.getHttp11ParseState();
    List<HttpPayload> parsed = state.getParsedMessages();
    WebSession session = (WebSession) socket.getSession().get(WEB_SESSION_KEY);
    if (session == null) {
        session = new WebSession();
        socket.getSession().put(WEB_SESSION_KEY, session);
    }
    // ALL of the below MUST happen AFTER the previous processing happened
    // which may not have finished so chain the below with the previous future
    XFuture<Void> future = session.getProcessFuture();
    for (HttpPayload payload : parsed) {
        // VERY IMPORTANT: Writing the code like this would slam through calling process N times
        // BUT it doesn't give the clients a chance to seet a flag between packets
        // Mainly done for exceptions and streaming so you can log exc, set a boolean so you
        // don't get 100 exceptions while something is happening like socket disconnect
        // In these 2 lines of code, processCorrectly is CALLED N times RIGHT NOW
        // The code below this only calls them right now IF AND ONLY IF the client returns
        // a completed future each time!!!
        // XFuture<Void> fut = processCorrectly(socket, payload);
        // future = future.thenCompose(s -> fut);
        future = future.thenCompose(s -> processCorrectly(socket, payload));
    }
    // replace with new future so it blocks any future pieces if these are not processed
    session.setProcessFuture(future);
    return future;
}
Also used : Http2Msg(com.webpieces.http2.api.dto.lowlevel.lib.Http2Msg) Http11ToHttp2(org.webpieces.http2translations.api.Http11ToHttp2) LoggerFactory(org.slf4j.LoggerFactory) Http2Request(com.webpieces.http2.api.dto.highlevel.Http2Request) HttpPayload(org.webpieces.httpparser.api.dto.HttpPayload) ByteBuffer(java.nio.ByteBuffer) HttpStream(org.webpieces.frontend2.api.HttpStream) HttpRequest(org.webpieces.httpparser.api.dto.HttpRequest) AtomicInteger(java.util.concurrent.atomic.AtomicInteger) DataWrapper(org.webpieces.data.api.DataWrapper) HttpData(org.webpieces.httpparser.api.dto.HttpData) HttpMessageType(org.webpieces.httpparser.api.dto.HttpMessageType) DataWrapperGeneratorFactory(org.webpieces.data.api.DataWrapperGeneratorFactory) PermitQueue(org.webpieces.util.locking.PermitQueue) Logger(org.slf4j.Logger) MarshalState(org.webpieces.httpparser.api.MarshalState) HttpParser(org.webpieces.httpparser.api.HttpParser) StreamListener(org.webpieces.frontend2.api.StreamListener) FutureHelper(org.webpieces.util.futures.FutureHelper) Memento(org.webpieces.httpparser.api.Memento) Consumer(java.util.function.Consumer) List(java.util.List) DataFrame(com.webpieces.http2.api.dto.lowlevel.DataFrame) XFuture(org.webpieces.util.futures.XFuture) StreamRef(com.webpieces.http2.api.streaming.StreamRef) MDC(org.slf4j.MDC) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) DataWrapperGenerator(org.webpieces.data.api.DataWrapperGenerator) MDCUtil(org.webpieces.nio.impl.cm.basic.MDCUtil) Memento(org.webpieces.httpparser.api.Memento) HttpPayload(org.webpieces.httpparser.api.dto.HttpPayload)

Example 37 with XFuture

use of org.webpieces.util.futures.XFuture in project webpieces by deanhiller.

the class Http11StreamImpl method process.

@Override
public XFuture<StreamWriter> process(Http2Response headers) {
    closeCheck(headers);
    HttpResponse response = Http2ToHttp11.translateResponse(headers);
    if (http2Request.getKnownMethod() == Http2Method.CONNECT) {
        // bytes so we don't care about parsing anymore(ie. SSL or http)..
        return write(response).thenApply(c -> new Http11ChunkedWriter(http1Req, http2Request));
    } else if (headers.isEndOfStream()) {
        validateHeader(response);
        remove(headers);
        return write(response).thenApply(w -> {
            permitQueue.releasePermit();
            return new NoWritesWriter();
        });
    } else if (contentLengthGreaterThanZero(headers)) {
        return write(response).thenApply(w -> new ContentLengthResponseWriter(headers));
    }
    return write(response).thenApply(c -> new Http11ChunkedWriter(http1Req, http2Request));
}
Also used : Http2Msg(com.webpieces.http2.api.dto.lowlevel.lib.Http2Msg) HttpChunk(org.webpieces.httpparser.api.dto.HttpChunk) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) Http2Request(com.webpieces.http2.api.dto.highlevel.Http2Request) AtomicReference(java.util.concurrent.atomic.AtomicReference) HttpPayload(org.webpieces.httpparser.api.dto.HttpPayload) ByteBuffer(java.nio.ByteBuffer) HttpResponse(org.webpieces.httpparser.api.dto.HttpResponse) HttpStream(org.webpieces.frontend2.api.HttpStream) HttpRequest(org.webpieces.httpparser.api.dto.HttpRequest) StreamMsg(com.webpieces.http2.api.dto.lowlevel.lib.StreamMsg) Map(java.util.Map) DataWrapper(org.webpieces.data.api.DataWrapper) Http2Method(com.webpieces.http2.api.dto.lowlevel.Http2Method) HttpData(org.webpieces.httpparser.api.dto.HttpData) FrontendSocket(org.webpieces.frontend2.api.FrontendSocket) PermitQueue(org.webpieces.util.locking.PermitQueue) Header(org.webpieces.httpparser.api.common.Header) Logger(org.slf4j.Logger) HttpParser(org.webpieces.httpparser.api.HttpParser) CancelReason(com.webpieces.http2.api.dto.lowlevel.CancelReason) KnownHeaderName(org.webpieces.httpparser.api.common.KnownHeaderName) HttpLastChunk(org.webpieces.httpparser.api.dto.HttpLastChunk) Http2ToHttp11(org.webpieces.http2translations.api.Http2ToHttp11) DataFrame(com.webpieces.http2.api.dto.lowlevel.DataFrame) XFuture(org.webpieces.util.futures.XFuture) StreamRef(com.webpieces.http2.api.streaming.StreamRef) Http2Header(com.webpieces.http2.api.dto.lowlevel.lib.Http2Header) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) Http2HeaderName(com.webpieces.http2.api.dto.lowlevel.lib.Http2HeaderName) PushStreamHandle(com.webpieces.http2.api.streaming.PushStreamHandle) ResponseStream(org.webpieces.frontend2.api.ResponseStream) Http2Response(com.webpieces.http2.api.dto.highlevel.Http2Response) HttpResponse(org.webpieces.httpparser.api.dto.HttpResponse)

Example 38 with XFuture

use of org.webpieces.util.futures.XFuture in project webpieces by deanhiller.

the class ProxyHttpStream method incomingRequest.

@Override
public StreamRef incomingRequest(Http2Request request, ResponseStream stream) {
    String expect = request.getSingleHeaderValue("Expect");
    XFuture<StreamWriter> future = XFuture.completedFuture(null);
    if (expect != null && "100-continue".equals(expect.toLowerCase())) {
        Http2Response continueResponse = new Http2Response();
        continueResponse.setEndOfStream(false);
        continueResponse.addHeader(new Http2Header(Http2HeaderName.STATUS, "100"));
        future = stream.process(continueResponse);
    }
    // This is only for streaming to backpressure clients IF we responded OR cancelled so we don't
    // waste CPU on a client stream coming in
    XFuture<ProxyWriter> futureWriter = new XFuture<>();
    ProxyResponseStream proxy = new ProxyResponseStream(request, stream, futureWriter);
    StreamRef streamRef = openStream.incomingRequest(request, proxy);
    XFuture<StreamWriter> writer = future.thenCompose(w -> {
        return createProxy(streamRef.getWriter(), futureWriter);
    });
    return new ProxyStreamRef(writer, streamRef);
}
Also used : Http2Response(com.webpieces.http2.api.dto.highlevel.Http2Response) StreamRef(com.webpieces.http2.api.streaming.StreamRef) XFuture(org.webpieces.util.futures.XFuture) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) Http2Header(com.webpieces.http2.api.dto.lowlevel.lib.Http2Header)

Example 39 with XFuture

use of org.webpieces.util.futures.XFuture in project webpieces by deanhiller.

the class StreamProxy method openStream.

@Override
public RouterStreamRef openStream(MethodMeta meta, ProxyStreamHandle handle) {
    RequestContext requestCtx = meta.getCtx();
    LoadedController loadedController = meta.getLoadedController();
    Object instance = loadedController.getControllerInstance();
    Method controllerMethod = loadedController.getControllerMethod();
    Parameter[] parameters = loadedController.getParameters();
    if (parameters.length != 1)
        throw new IllegalArgumentException("Your method='" + controllerMethod + "' MUST one parameter and does not.  It needs to take a RouterStreamHandler");
    else if (!ResponseStreamHandle.class.equals(parameters[0].getType()))
        throw new IllegalArgumentException("The single parameter must be RouterStreamHandle and was not for this method='" + controllerMethod + "'");
    else if (!StreamRef.class.equals(controllerMethod.getReturnType()))
        throw new IllegalArgumentException("The return value must be a subclass of StreamRef and was not for this method='" + controllerMethod + "'");
    StreamRef streamRef = invokeStream(meta, controllerMethod, instance, requestCtx, handle);
    XFuture<StreamWriter> writer = streamRef.getWriter();
    XFuture<StreamWriter> newFuture = futureUtil.catchBlockWrap(() -> writer, (t) -> convert(loadedController, t));
    Function<CancelReason, XFuture<Void>> cancelFunc = (reason) -> streamRef.cancel(reason);
    return new RouterStreamRef("streamProxy", newFuture, cancelFunc);
}
Also used : LoadedController(org.webpieces.router.impl.loader.LoadedController) CancelReason(com.webpieces.http2.api.dto.lowlevel.CancelReason) Function(java.util.function.Function) FutureHelper(org.webpieces.util.futures.FutureHelper) ResponseStreamHandle(com.webpieces.http2.api.streaming.ResponseStreamHandle) ProxyStreamHandle(org.webpieces.router.impl.proxyout.ProxyStreamHandle) ControllerException(org.webpieces.router.api.exceptions.ControllerException) MethodMeta(org.webpieces.router.api.routes.MethodMeta) XFuture(org.webpieces.util.futures.XFuture) RequestContext(org.webpieces.ctx.api.RequestContext) StreamRef(com.webpieces.http2.api.streaming.StreamRef) Parameter(java.lang.reflect.Parameter) ServiceInvoker(org.webpieces.router.impl.routeinvoker.ServiceInvoker) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) StreamService(org.webpieces.router.api.streams.StreamService) Method(java.lang.reflect.Method) RouterStreamRef(org.webpieces.router.impl.routeinvoker.RouterStreamRef) WebpiecesException(org.webpieces.util.exceptions.WebpiecesException) LoadedController(org.webpieces.router.impl.loader.LoadedController) XFuture(org.webpieces.util.futures.XFuture) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) Method(java.lang.reflect.Method) RouterStreamRef(org.webpieces.router.impl.routeinvoker.RouterStreamRef) StreamRef(com.webpieces.http2.api.streaming.StreamRef) RouterStreamRef(org.webpieces.router.impl.routeinvoker.RouterStreamRef) CancelReason(com.webpieces.http2.api.dto.lowlevel.CancelReason) Parameter(java.lang.reflect.Parameter) RequestContext(org.webpieces.ctx.api.RequestContext)

Example 40 with XFuture

use of org.webpieces.util.futures.XFuture in project webpieces by deanhiller.

the class XFileReader method runFileRead.

public XFuture<Void> runFileRead(RequestInfo info, RenderStaticResponse renderStatic, ProxyStreamHandle handle) throws IOException {
    VirtualFile fullFilePath = renderStatic.getFilePath();
    if (!fullFilePath.exists()) {
        throw new NotFoundException("File not found=" + fullFilePath);
    } else if (fullFilePath.isDirectory()) {
        throw new NotFoundException("File not found (it was a directory that can't be rendered)=" + fullFilePath);
    }
    String fileName = getNameToUse(fullFilePath);
    String extension = null;
    int lastDot = fileName.lastIndexOf(".");
    if (lastDot > 0) {
        extension = fileName.substring(lastDot + 1);
    }
    ResponseCreator.ResponseEncodingTuple tuple = responseCreator.createResponse(info.getRequest(), StatusCode.HTTP_200_OK, extension, "application/octet-stream", false);
    Http2Response response = tuple.response;
    // On startup, we protect developers from breaking clients.  In http, all files that change
    // must also change the hash automatically and the %%{ }%% tag generates those hashes so the
    // files loaded are always the latest
    Long timeSeconds = config.getStaticFileCacheTimeSeconds();
    if (timeSeconds != null)
        response.addHeader(new Http2Header(Http2HeaderName.CACHE_CONTROL, "max-age=" + timeSeconds));
    ChunkReader reader = createFileReader(response, renderStatic, fileName, fullFilePath, info, extension, tuple, handle);
    if (log.isDebugEnabled())
        log.debug("sending chunked file via async read=" + reader);
    ProxyStreamHandle stream = info.getResponseSender();
    return futureUtil.finallyBlock(() -> stream.process(response).thenCompose(s -> readLoop(s, info.getPool(), reader, 0)), () -> handleClose(info, reader));
}
Also used : VirtualFile(org.webpieces.util.file.VirtualFile) BufferPool(org.webpieces.data.api.BufferPool) StatusCode(org.webpieces.http.StatusCode) Logger(org.slf4j.Logger) LoggerFactory(org.slf4j.LoggerFactory) RouterConfig(org.webpieces.router.api.RouterConfig) IOException(java.io.IOException) ByteBuffer(java.nio.ByteBuffer) FutureHelper(org.webpieces.util.futures.FutureHelper) NotFoundException(org.webpieces.http.exception.NotFoundException) VirtualFile(org.webpieces.util.file.VirtualFile) ProxyStreamHandle(org.webpieces.router.impl.proxyout.ProxyStreamHandle) ResponseCreator(org.webpieces.router.impl.proxyout.ResponseCreator) DataFrame(com.webpieces.http2.api.dto.lowlevel.DataFrame) XFuture(org.webpieces.util.futures.XFuture) Http2Header(com.webpieces.http2.api.dto.lowlevel.lib.Http2Header) StreamWriter(com.webpieces.http2.api.streaming.StreamWriter) DataWrapper(org.webpieces.data.api.DataWrapper) DataWrapperGenerator(org.webpieces.data.api.DataWrapperGenerator) Http2HeaderName(com.webpieces.http2.api.dto.lowlevel.lib.Http2HeaderName) RenderStaticResponse(org.webpieces.router.impl.dto.RenderStaticResponse) DataWrapperGeneratorFactory(org.webpieces.data.api.DataWrapperGeneratorFactory) Http2Response(com.webpieces.http2.api.dto.highlevel.Http2Response) Http2Response(com.webpieces.http2.api.dto.highlevel.Http2Response) ProxyStreamHandle(org.webpieces.router.impl.proxyout.ProxyStreamHandle) Http2Header(com.webpieces.http2.api.dto.lowlevel.lib.Http2Header) NotFoundException(org.webpieces.http.exception.NotFoundException) ResponseCreator(org.webpieces.router.impl.proxyout.ResponseCreator)

Aggregations

XFuture (org.webpieces.util.futures.XFuture)71 Test (org.junit.Test)21 StreamWriter (com.webpieces.http2.api.streaming.StreamWriter)20 ByteBuffer (java.nio.ByteBuffer)16 Logger (org.slf4j.Logger)15 LoggerFactory (org.slf4j.LoggerFactory)15 ArrayList (java.util.ArrayList)14 List (java.util.List)13 Map (java.util.Map)12 DataWrapper (org.webpieces.data.api.DataWrapper)12 HttpFullRequest (org.webpieces.httpclient11.api.HttpFullRequest)12 HttpFullResponse (org.webpieces.httpclient11.api.HttpFullResponse)12 NotFoundException (org.webpieces.http.exception.NotFoundException)11 AbstractWebpiecesTest (org.webpieces.webserver.test.AbstractWebpiecesTest)11 ResponseWrapper (org.webpieces.webserver.test.ResponseWrapper)11 Http2Request (com.webpieces.http2.api.dto.highlevel.Http2Request)10 PrivateWebserverForTest (org.webpieces.webserver.PrivateWebserverForTest)10 StreamRef (com.webpieces.http2.api.streaming.StreamRef)9 RequestContext (org.webpieces.ctx.api.RequestContext)9 Http2Response (com.webpieces.http2.api.dto.highlevel.Http2Response)8