Search in sources :

Example 1 with Result

use of com.yahoo.messagebus.Result in project vespa by vespa-engine.

the class FeedTesterV3 method setupFeederHandler.

FeedHandlerV3 setupFeederHandler() throws Exception {
    Executor threadPool = Executors.newCachedThreadPool();
    DocumentmanagerConfig docMan = new DocumentmanagerConfig(new DocumentmanagerConfig.Builder().enablecompression(true));
    FeedHandlerV3 feedHandlerV3 = new FeedHandlerV3(new FeedHandlerV3.Context(threadPool, AccessLog.voidAccessLog(), new NullFeedMetric()), docMan, null, /* session cache */
    null, /* thread pool config */
    new DocumentApiMetrics(MetricReceiver.nullImplementation, "test")) {

        @Override
        protected ReferencedResource<SharedSourceSession> retainSource(SessionCache sessionCache, SourceSessionParams sessionParams) {
            SharedSourceSession sharedSourceSession = mock(SharedSourceSession.class);
            try {
                Mockito.stub(sharedSourceSession.sendMessageBlocking(anyObject())).toAnswer((Answer) invocation -> {
                    Object[] args = invocation.getArguments();
                    PutDocumentMessage putDocumentMessage = (PutDocumentMessage) args[0];
                    ReplyContext replyContext = (ReplyContext) putDocumentMessage.getContext();
                    replyContext.feedReplies.add(new OperationStatus("message", replyContext.docId, ErrorCode.OK, false, "trace"));
                    Result result = mock(Result.class);
                    when(result.isAccepted()).thenReturn(true);
                    return result;
                });
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            Result result = mock(Result.class);
            when(result.isAccepted()).thenReturn(true);
            ReferencedResource<SharedSourceSession> refSharedSessopn = new ReferencedResource<>(sharedSourceSession, () -> {
            });
            return refSharedSessopn;
        }
    };
    feedHandlerV3.injectDocumentManangerForTests(createDoctypeManager());
    return feedHandlerV3;
}
Also used : CoreMatchers.is(org.hamcrest.CoreMatchers.is) ByteArrayOutputStream(java.io.ByteArrayOutputStream) DocumentmanagerConfig(com.yahoo.document.config.DocumentmanagerConfig) DataType(com.yahoo.document.DataType) ReplyContext(com.yahoo.vespa.http.server.ReplyContext) DocumentType(com.yahoo.document.DocumentType) OperationStatus(com.yahoo.vespa.http.client.core.OperationStatus) Assert.assertThat(org.junit.Assert.assertThat) Answer(org.mockito.stubbing.Answer) ByteArrayInputStream(java.io.ByteArrayInputStream) SharedSourceSession(com.yahoo.messagebus.shared.SharedSourceSession) ErrorCode(com.yahoo.vespa.http.client.core.ErrorCode) Matchers.anyObject(org.mockito.Matchers.anyObject) SessionCache(com.yahoo.container.jdisc.messagebus.SessionCache) PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) MetricReceiver(com.yahoo.metrics.simple.MetricReceiver) Splitter(com.google.common.base.Splitter) FeedHandlerV3(com.yahoo.vespa.http.server.FeedHandlerV3) Executor(java.util.concurrent.Executor) Utf8(com.yahoo.text.Utf8) NullFeedMetric(com.yahoo.feedhandler.NullFeedMetric) AccessLog(com.yahoo.container.logging.AccessLog) HttpRequest(com.yahoo.container.jdisc.HttpRequest) FeedParams(com.yahoo.vespa.http.client.config.FeedParams) Test(org.junit.Test) Mockito.when(org.mockito.Mockito.when) ReferencedResource(com.yahoo.jdisc.ReferencedResource) Executors(java.util.concurrent.Executors) Result(com.yahoo.messagebus.Result) DocumentApiMetrics(com.yahoo.documentapi.metrics.DocumentApiMetrics) Mockito(org.mockito.Mockito) Headers(com.yahoo.vespa.http.client.core.Headers) DocumentTypeManager(com.yahoo.document.DocumentTypeManager) SourceSessionParams(com.yahoo.messagebus.SourceSessionParams) HttpResponse(com.yahoo.container.jdisc.HttpResponse) InputStream(java.io.InputStream) Mockito.mock(org.mockito.Mockito.mock) ReferencedResource(com.yahoo.jdisc.ReferencedResource) SourceSessionParams(com.yahoo.messagebus.SourceSessionParams) SharedSourceSession(com.yahoo.messagebus.shared.SharedSourceSession) Result(com.yahoo.messagebus.Result) FeedHandlerV3(com.yahoo.vespa.http.server.FeedHandlerV3) PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) Executor(java.util.concurrent.Executor) DocumentmanagerConfig(com.yahoo.document.config.DocumentmanagerConfig) DocumentApiMetrics(com.yahoo.documentapi.metrics.DocumentApiMetrics) OperationStatus(com.yahoo.vespa.http.client.core.OperationStatus) ReplyContext(com.yahoo.vespa.http.server.ReplyContext) SessionCache(com.yahoo.container.jdisc.messagebus.SessionCache) NullFeedMetric(com.yahoo.feedhandler.NullFeedMetric)

Example 2 with Result

use of com.yahoo.messagebus.Result in project vespa by vespa-engine.

the class ClientFeederV3 method feed.

private void feed(FeederSettings settings, InputStream requestInputStream, BlockingQueue<OperationStatus> repliesFromOldMessages, AtomicInteger threadsAvailableForFeeding) throws InterruptedException {
    while (true) {
        Optional<DocumentOperationMessageV3> msg = pullMessageFromRequest(settings, requestInputStream, repliesFromOldMessages);
        if (!msg.isPresent()) {
            break;
        }
        setMessageParameters(msg.get(), settings);
        final Result result;
        try {
            result = sendMessage(settings, msg.get(), threadsAvailableForFeeding);
        } catch (RuntimeException e) {
            repliesFromOldMessages.add(createOperationStatus(msg.get().getOperationId(), Exceptions.toMessageString(e), ErrorCode.ERROR, false, msg.get().getMessage()));
            continue;
        }
        if (result.isAccepted()) {
            outstandingOperations.incrementAndGet();
            updateOpsPerSec();
            log(LogLevel.DEBUG, "Sent message successfully, document id: ", msg.get().getOperationId());
        } else if (!result.getError().isFatal()) {
            repliesFromOldMessages.add(createOperationStatus(msg.get().getOperationId(), result.getError().getMessage(), ErrorCode.TRANSIENT_ERROR, false, msg.get().getMessage()));
            continue;
        } else {
            // should probably not happen, but everybody knows stuff that
            // shouldn't happen, happens all the time
            boolean isConditionNotMet = result.getError().getCode() == DocumentProtocol.ERROR_TEST_AND_SET_CONDITION_FAILED;
            repliesFromOldMessages.add(createOperationStatus(msg.get().getOperationId(), result.getError().getMessage(), ErrorCode.ERROR, isConditionNotMet, msg.get().getMessage()));
            continue;
        }
    }
}
Also used : Result(com.yahoo.messagebus.Result)

Example 3 with Result

use of com.yahoo.messagebus.Result in project vespa by vespa-engine.

the class Feeder method feed.

void feed() throws InterruptedException {
    while (true) {
        Result result;
        String operationId;
        try {
            operationId = getNextOperationId();
        } catch (IOException ioe) {
            if (log.isLoggable(LogLevel.DEBUG)) {
                log.log(LogLevel.DEBUG, Exceptions.toMessageString(ioe), ioe);
            }
            break;
        }
        // noinspection StringEquality
        if (operationId == EOF) {
            break;
        }
        Tuple2<String, Message> msg;
        try {
            msg = getNextMessage(operationId);
            setRoute(msg);
        } catch (Exception e) {
            if (log.isLoggable(LogLevel.DEBUG)) {
                log.log(LogLevel.DEBUG, Exceptions.toMessageString(e), e);
            }
            // noinspection StringEquality
            if (operationId != null) {
                // v1 always returns null, all others return something useful, or throw an exception above
                msg = newErrorMessage(operationId, e);
            } else {
                break;
            }
        }
        if (msg == null) {
            break;
        }
        setMessageParameters(msg);
        while (true) {
            try {
                msg.second.pushHandler(feedReplyHandler);
                if (settings.denyIfBusy) {
                    result = sourceSession.getResource().sendMessage(msg.second);
                } else {
                    result = sourceSession.getResource().sendMessageBlocking(msg.second);
                }
            } catch (RuntimeException e) {
                enqueue(msg.first, Exceptions.toMessageString(e), ErrorCode.ERROR, false, msg.second);
                return;
            }
            if (result.isAccepted() || result.getError().getCode() != SEND_QUEUE_FULL) {
                break;
            }
            if (settings.denyIfBusy) {
                break;
            } else {
                // This will never happen
                Thread.sleep(100);
            }
        }
        if (result.isAccepted()) {
            ++numPending;
            updateMetrics(msg.second);
            updateOpsPerSec();
            log(LogLevel.DEBUG, "Sent message successfully, document id: ", msg.first);
        } else if (!result.getError().isFatal()) {
            enqueue(msg.first, result.getError().getMessage(), ErrorCode.TRANSIENT_ERROR, false, msg.second);
            break;
        } else {
            // should probably not happen, but everybody knows stuff that
            // shouldn't happen, happens all the time
            boolean isConditionNotMet = result.getError().getCode() == DocumentProtocol.ERROR_TEST_AND_SET_CONDITION_FAILED;
            enqueue(msg.first, result.getError().getMessage(), ErrorCode.ERROR, isConditionNotMet, msg.second);
            break;
        }
    }
}
Also used : PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) UpdateDocumentMessage(com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage) Message(com.yahoo.messagebus.Message) DocumentMessage(com.yahoo.documentapi.messagebus.protocol.DocumentMessage) RemoveDocumentMessage(com.yahoo.documentapi.messagebus.protocol.RemoveDocumentMessage) Utf8String(com.yahoo.text.Utf8String) IOException(java.io.IOException) IOException(java.io.IOException) Result(com.yahoo.messagebus.Result)

Example 4 with Result

use of com.yahoo.messagebus.Result in project vespa by vespa-engine.

the class ClientFeederV3 method sendMessage.

private Result sendMessage(FeederSettings settings, DocumentOperationMessageV3 msg, AtomicInteger threadsAvailableForFeeding) throws InterruptedException {
    Result result = null;
    while (result == null || result.getError().getCode() == SEND_QUEUE_FULL) {
        msg.getMessage().pushHandler(feedReplyHandler);
        if (settings.denyIfBusy && threadsAvailableForFeeding.get() < 1) {
            return sourceSession.getResource().sendMessage(msg.getMessage());
        } else {
            result = sourceSession.getResource().sendMessageBlocking(msg.getMessage());
        }
        if (result.isAccepted()) {
            return result;
        }
        Thread.sleep(100);
    }
    return result;
}
Also used : Result(com.yahoo.messagebus.Result)

Aggregations

Result (com.yahoo.messagebus.Result)4 PutDocumentMessage (com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage)2 Splitter (com.google.common.base.Splitter)1 HttpRequest (com.yahoo.container.jdisc.HttpRequest)1 HttpResponse (com.yahoo.container.jdisc.HttpResponse)1 SessionCache (com.yahoo.container.jdisc.messagebus.SessionCache)1 AccessLog (com.yahoo.container.logging.AccessLog)1 DataType (com.yahoo.document.DataType)1 DocumentType (com.yahoo.document.DocumentType)1 DocumentTypeManager (com.yahoo.document.DocumentTypeManager)1 DocumentmanagerConfig (com.yahoo.document.config.DocumentmanagerConfig)1 DocumentMessage (com.yahoo.documentapi.messagebus.protocol.DocumentMessage)1 RemoveDocumentMessage (com.yahoo.documentapi.messagebus.protocol.RemoveDocumentMessage)1 UpdateDocumentMessage (com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage)1 DocumentApiMetrics (com.yahoo.documentapi.metrics.DocumentApiMetrics)1 NullFeedMetric (com.yahoo.feedhandler.NullFeedMetric)1 ReferencedResource (com.yahoo.jdisc.ReferencedResource)1 Message (com.yahoo.messagebus.Message)1 SourceSessionParams (com.yahoo.messagebus.SourceSessionParams)1 SharedSourceSession (com.yahoo.messagebus.shared.SharedSourceSession)1