Examples with OperationStatus com.yahoo.vespa.http.client.core.OperationStatus used on opensource projects

Example 11 with OperationStatus

use of com.yahoo.vespa.http.client.core.OperationStatus in project vespa by vespa-engine.

the class V3MockParsingRequestHandler method respondFailedWithTransitiveErrorSeenFromClient.

private void respondFailedWithTransitiveErrorSeenFromClient(PrintWriter responseWriter, String docId) {
    final OperationStatus operationStatus = new OperationStatus("NETWORK_ERROR", docId, ErrorCode.ERROR, false, "trace");
    writeResponse(responseWriter, operationStatus);
}

Example 12 with OperationStatus

use of com.yahoo.vespa.http.client.core.OperationStatus in project vespa by vespa-engine.

the class V3MockParsingRequestHandler method respondFailed.

private void respondFailed(PrintWriter responseWriter, String docId) {
    final OperationStatus operationStatus = new OperationStatus("mbus returned boom", docId, ErrorCode.ERROR, false, "trace");
    writeResponse(responseWriter, operationStatus);
}

Example 13 with OperationStatus

use of com.yahoo.vespa.http.client.core.OperationStatus in project vespa by vespa-engine.

the class ClientFeederV3 method handleRequest.

public HttpResponse handleRequest(HttpRequest request) throws IOException {
    threadsAvailableForFeeding.decrementAndGet();
    ongoingRequests.incrementAndGet();
    try {
        FeederSettings feederSettings = new FeederSettings(request);
        /**
         * The gateway handle overload from clients in different ways.
         *
         * If the backend is overloaded, but not the gateway, it will fill the backend, messagebus throttler
         * will start to block new documents and finally all threadsAvailableForFeeding will be blocking.
         * However, as more threads are added, the gateway will not block on messagebus but return
         * transitive errors on the documents that can not be processed. These errors will cause the client(s) to
         * back off a bit.
         *
         * However, we can also have the case that the gateway becomes the bottleneck (e.g. CPU). In this case
         * we need to stop processing of new messages as early as possible and reject the request. This
         * will cause the client(s) to back off for a while. We want some slack before we enter this mode.
         * If we can simply transitively fail each document, it is nicer. Therefor we allow some threads to be
         * busy processing requests with transitive errors before entering this mode. Since we already
         * have flooded the backend, have several threads hanging and waiting for capacity, the number should
         * not be very large. Too much slack can lead to too many threads handling feed and impacting query traffic.
         * We try 10 for now. This should only kick in with very massive feeding to few gateway nodes.
         */
        if (feederSettings.denyIfBusy && threadsAvailableForFeeding.get() < -10) {
            return new ErrorHttpResponse(getOverloadReturnCode(request), "Gateway overloaded");
        }
        InputStream inputStream = StreamReaderV3.unzipStreamIfNeeded(request);
        final BlockingQueue<OperationStatus> replies = new LinkedBlockingQueue<>();
        try {
            feed(feederSettings, inputStream, replies, threadsAvailableForFeeding);
            synchronized (monitor) {
                // handshakes as it won't be processed by the client.
                if (request.getJDiscRequest().headers().get(Headers.DATA_FORMAT) != null) {
                    transferPreviousRepliesToResponse(replies);
                }
            }
        } catch (InterruptedException e) {
        // NOP, just terminate
        } catch (Throwable e) {
            log.log(LogLevel.WARNING, "Unhandled exception while feeding: " + Exceptions.toMessageString(e), e);
        } finally {
            try {
                replies.add(createOperationStatus("-", "-", ErrorCode.END_OF_FEED, false, null));
            } catch (InterruptedException e) {
            // NOP, we are already exiting the thread
            }
        }
        return new FeedResponse(200, replies, 3, /* protocol version */
        clientId, outstandingOperations.get(), hostName);
    } finally {
        ongoingRequests.decrementAndGet();
        threadsAvailableForFeeding.incrementAndGet();
    }
}

Example 14 with OperationStatus

use of com.yahoo.vespa.http.client.core.OperationStatus in project vespa by vespa-engine.

the class FeedReplyReader method enqueue.

private void enqueue(ReplyContext context, String message, ErrorCode status, boolean isConditionNotMet, Trace trace) {
    try {
        String traceMessage = (trace != null && trace.getLevel() > 0) ? trace.toString() : "";
        context.feedReplies.put(new OperationStatus(message, context.docId, status, isConditionNotMet, traceMessage));
    } catch (InterruptedException e) {
        log.log(LogLevel.WARNING, "Interrupted while enqueueing result from putting document with id: " + context.docId);
        Thread.currentThread().interrupt();
    }
}

Example 15 with OperationStatus

use of com.yahoo.vespa.http.client.core.OperationStatus in project vespa by vespa-engine.

the class Feeder method drain.

void drain() throws InterruptedException {
    if (settings.drain) {
        while (numPending > 0) {
            OperationStatus o = feedReplies.take();
            decreasePending(o);
        }
    }
}