Search in sources :

Example 6 with Tuple2

use of com.yahoo.collections.Tuple2 in project vespa by vespa-engine.

the class Feeder method newUpdateMessage.

private Tuple2<String, Message> newUpdateMessage(Operation op, String operationId) {
    DocumentUpdate update = op.getDocumentUpdate();
    update.setCondition(op.getCondition());
    Message msg = new UpdateDocumentMessage(update);
    String id = (operationId == null) ? update.getId().toString() : operationId;
    return new Tuple2<>(id, msg);
}
Also used : DocumentUpdate(com.yahoo.document.DocumentUpdate) PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) UpdateDocumentMessage(com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage) Message(com.yahoo.messagebus.Message) DocumentMessage(com.yahoo.documentapi.messagebus.protocol.DocumentMessage) RemoveDocumentMessage(com.yahoo.documentapi.messagebus.protocol.RemoveDocumentMessage) Tuple2(com.yahoo.collections.Tuple2) Utf8String(com.yahoo.text.Utf8String) UpdateDocumentMessage(com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage)

Example 7 with Tuple2

use of com.yahoo.collections.Tuple2 in project vespa by vespa-engine.

the class Feeder method newPutMessage.

private Tuple2<String, Message> newPutMessage(Operation op, String operationId) {
    DocumentPut put = new DocumentPut(op.getDocument());
    put.setCondition(op.getCondition());
    Message msg = new PutDocumentMessage(put);
    String id = (operationId == null) ? put.getId().toString() : operationId;
    return new Tuple2<>(id, msg);
}
Also used : PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) UpdateDocumentMessage(com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage) Message(com.yahoo.messagebus.Message) DocumentMessage(com.yahoo.documentapi.messagebus.protocol.DocumentMessage) RemoveDocumentMessage(com.yahoo.documentapi.messagebus.protocol.RemoveDocumentMessage) Tuple2(com.yahoo.collections.Tuple2) DocumentPut(com.yahoo.document.DocumentPut) Utf8String(com.yahoo.text.Utf8String)

Example 8 with Tuple2

use of com.yahoo.collections.Tuple2 in project vespa by vespa-engine.

the class MinimalQueryInserterTestCase method testStringReprBasicSanity.

@Test
public void testStringReprBasicSanity() {
    String yql = "select%20ignoredfield%20from%20ignoredsource%20where%20title%20contains%20%22madonna%22%20order%20by%20something%2C%20shoesize%20desc%20limit%20300%20timeout%203%3B";
    Query query = new Query("search/?yql=" + yql);
    execution.search(query);
    assertEquals("select ignoredfield from ignoredsource where [{\"segmenter\": {\"version\": \"1.9\", \"backend\": \"YqlUnitTest\"}}](title contains \"madonna\") order by something, shoesize desc limit 300 timeout 3;", query.yqlRepresentation(new Tuple2<>("YqlUnitTest", new Version(1, 9)), true));
}
Also used : Query(com.yahoo.search.Query) Version(com.yahoo.component.Version) Tuple2(com.yahoo.collections.Tuple2) Test(org.junit.Test)

Example 9 with Tuple2

use of com.yahoo.collections.Tuple2 in project vespa by vespa-engine.

the class JsonReaderTestCase method testUpdateMatch.

@Test
public final void testUpdateMatch() throws IOException {
    DocumentUpdate doc = parseUpdate("{\"update\": \"id:unittest:testset::whee\"," + " \"fields\": { " + "\"actualset\": {" + " \"match\": {" + " \"element\": \"person\"," + " \"increment\": 13}}}}");
    Map<String, Tuple2<Number, String>> matches = new HashMap<>();
    FieldUpdate x = doc.getFieldUpdate("actualset");
    for (ValueUpdate<?> v : x.getValueUpdates()) {
        MapValueUpdate adder = (MapValueUpdate) v;
        final String key = ((StringFieldValue) adder.getValue()).getString();
        String op = ((ArithmeticValueUpdate) adder.getUpdate()).getOperator().toString();
        Number n = ((ArithmeticValueUpdate) adder.getUpdate()).getOperand();
        matches.put(key, new Tuple2<>(n, op));
    }
    assertEquals(1, matches.size());
    final String o = "person";
    assertEquals("ADD", matches.get(o).second);
    assertEquals(Double.valueOf(13), matches.get(o).first);
}
Also used : MapValueUpdate(com.yahoo.document.update.MapValueUpdate) DocumentUpdate(com.yahoo.document.DocumentUpdate) HashMap(java.util.HashMap) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) Tuple2(com.yahoo.collections.Tuple2) FieldUpdate(com.yahoo.document.update.FieldUpdate) ArithmeticValueUpdate(com.yahoo.document.update.ArithmeticValueUpdate) Test(org.junit.Test)

Example 10 with Tuple2

use of com.yahoo.collections.Tuple2 in project vespa by vespa-engine.

the class DocumentProcessingTask method process.

/**
 * Processes a single Processing, and fails the message if this processing fails.
 *
 * @param executor the DocprocService to use for processing
 */
private DocumentProcessor.Progress process(DocprocExecutor executor) {
    Iterator<Processing> iterator = processings.iterator();
    List<Tuple2<DocumentProcessor.Progress, Processing>> later = new ArrayList<>();
    while (iterator.hasNext()) {
        Processing processing = iterator.next();
        iterator.remove();
        if (requestContext.hasExpired()) {
            DocumentProcessor.Progress progress = DocumentProcessor.Progress.FAILED;
            final String location;
            if (processing != null) {
                final CallStack callStack = processing.callStack();
                if (callStack != null) {
                    final Call lastPopped = callStack.getLastPopped();
                    if (lastPopped != null) {
                        location = lastPopped.toString();
                    } else {
                        location = "empty call stack or no processors popped";
                    }
                } else {
                    location = "no call stack";
                }
            } else {
                location = "no processing instance";
            }
            String errorMsg = processing + " failed, " + location;
            log.log(Level.FINE, "Time is up for '" + errorMsg + "'.");
            requestContext.processingFailed(RequestContext.ErrorCode.ERROR_PROCESSING_FAILURE, "Time is up.");
            return progress;
        }
        DocumentProcessor.Progress progress = DocumentProcessor.Progress.FAILED;
        try {
            progress = executor.process(processing);
        } catch (Exception e) {
            logProcessingFailure(processing, e);
            requestContext.processingFailed(e);
            return progress;
        }
        if (DocumentProcessor.Progress.LATER.equals(progress)) {
            later.add(new Tuple2<>(progress, processing));
        } else if (DocumentProcessor.Progress.DONE.equals(progress)) {
            processingsDone.add(processing);
        } else if (DocumentProcessor.Progress.FAILED.equals(progress)) {
            logProcessingFailure(processing, null);
            requestContext.processingFailed(RequestContext.ErrorCode.ERROR_PROCESSING_FAILURE, progress.getReason().orElse("Document processing failed."));
            return progress;
        } else if (DocumentProcessor.Progress.PERMANENT_FAILURE.equals(progress)) {
            logProcessingFailure(processing, null);
            requestContext.processingFailed(RequestContext.ErrorCode.ERROR_PROCESSING_FAILURE, progress.getReason().orElse("Document processing failed."));
            return progress;
        }
    }
    if (!later.isEmpty()) {
        // Outdated comment:
        // "if this was a multioperationmessage and more than one of the processings returned LATER,
        // return the one with the lowest timeout:"
        // As multioperation is removed this can probably be simplified?
        DocumentProcessor.LaterProgress shortestDelay = (DocumentProcessor.LaterProgress) later.get(0).first;
        for (Tuple2<DocumentProcessor.Progress, Processing> tuple : later) {
            // re-add the LATER one to processings
            processings.add(tuple.second);
            // check to see if this one had a lower timeout than the previous one:
            if (((DocumentProcessor.LaterProgress) tuple.first).getDelay() < shortestDelay.getDelay()) {
                shortestDelay = (DocumentProcessor.LaterProgress) tuple.first;
            }
        }
        return shortestDelay;
    } else {
        requestContext.processingDone(processingsDone);
        return DocumentProcessor.Progress.DONE;
    }
}
Also used : Call(com.yahoo.docproc.Call) DocumentProcessor(com.yahoo.docproc.DocumentProcessor) CallStack(com.yahoo.docproc.CallStack) ArrayList(java.util.ArrayList) HandledProcessingException(com.yahoo.docproc.HandledProcessingException) TimeoutException(java.util.concurrent.TimeoutException) Processing(com.yahoo.docproc.Processing) Tuple2(com.yahoo.collections.Tuple2)

Aggregations

Tuple2 (com.yahoo.collections.Tuple2)11 DocumentMessage (com.yahoo.documentapi.messagebus.protocol.DocumentMessage)4 PutDocumentMessage (com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage)4 RemoveDocumentMessage (com.yahoo.documentapi.messagebus.protocol.RemoveDocumentMessage)4 UpdateDocumentMessage (com.yahoo.documentapi.messagebus.protocol.UpdateDocumentMessage)4 Message (com.yahoo.messagebus.Message)4 Utf8String (com.yahoo.text.Utf8String)4 DocumentUpdate (com.yahoo.document.DocumentUpdate)3 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)3 Test (org.junit.Test)3 ArithmeticValueUpdate (com.yahoo.document.update.ArithmeticValueUpdate)2 FieldUpdate (com.yahoo.document.update.FieldUpdate)2 MapValueUpdate (com.yahoo.document.update.MapValueUpdate)2 ArrayList (java.util.ArrayList)2 Version (com.yahoo.component.Version)1 CompressionType (com.yahoo.compress.CompressionType)1 Call (com.yahoo.docproc.Call)1 CallStack (com.yahoo.docproc.CallStack)1 DocumentProcessor (com.yahoo.docproc.DocumentProcessor)1 HandledProcessingException (com.yahoo.docproc.HandledProcessingException)1