Search in sources :

Example 1 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class MockDispatchDocproc method process.

@Override
public Progress process(Processing processing) {
    for (DocumentOperation op : processing.getDocumentOperations()) {
        PutDocumentMessage message = new PutDocumentMessage((DocumentPut) op);
        ListenableFuture<Response> future = createRequest(message).dispatch();
        try {
            responses.add(future.get());
        } catch (ExecutionException | InterruptedException e) {
            throw new RuntimeException(e);
        }
    }
    return Progress.DONE;
}
Also used : PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) Response(com.yahoo.jdisc.Response) DocumentOperation(com.yahoo.document.DocumentOperation) ExecutionException(java.util.concurrent.ExecutionException)

Example 2 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class Rot13DocumentProcessor method process.

@Override
public Progress process(Processing processing) {
    int oldVal = counter.getAndIncrement();
    if ((oldVal % 3) != 0) {
        return Progress.LATER;
    }
    for (DocumentOperation op : processing.getDocumentOperations()) {
        if (op instanceof DocumentPut) {
            Document document = ((DocumentPut) op).getDocument();
            StringFieldValue oldTitle = (StringFieldValue) document.getFieldValue(FIELD_NAME);
            if (oldTitle != null) {
                document.setFieldValue(FIELD_NAME, rot13(oldTitle.getString()));
            }
        }
    }
    return Progress.DONE;
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document)

Example 3 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class JsonReader method next.

public DocumentOperation next() {
    switch(state) {
        case AT_START:
            JsonToken t = nextToken(parser);
            expectArrayStart(t);
            state = ReaderState.READING;
            break;
        case END_OF_FEED:
            return null;
        case READING:
            break;
    }
    Optional<DocumentParseInfo> documentParseInfo;
    try {
        documentParseInfo = parseDocument();
    } catch (IOException r) {
        // Jackson is not able to recover from structural parse errors
        state = END_OF_FEED;
        throw new RuntimeException(r);
    }
    if (!documentParseInfo.isPresent()) {
        state = END_OF_FEED;
        return null;
    }
    VespaJsonDocumentReader vespaJsonDocumentReader = new VespaJsonDocumentReader();
    DocumentOperation operation = vespaJsonDocumentReader.createDocumentOperation(getDocumentTypeFromString(documentParseInfo.get().documentId.getDocType(), typeManager), documentParseInfo.get());
    operation.setCondition(TestAndSetCondition.fromConditionString(documentParseInfo.get().condition));
    return operation;
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) JsonToken(com.fasterxml.jackson.core.JsonToken) IOException(java.io.IOException) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader)

Example 4 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class JsonFeedReader method read.

@Override
public void read(Operation operation) throws Exception {
    DocumentOperation documentOperation = reader.next();
    if (documentOperation == null) {
        stream.close();
        operation.setInvalid();
        return;
    }
    if (documentOperation instanceof DocumentUpdate) {
        operation.setDocumentUpdate((DocumentUpdate) documentOperation);
    } else if (documentOperation instanceof DocumentRemove) {
        operation.setRemove(documentOperation.getId());
    } else if (documentOperation instanceof DocumentPut) {
        operation.setDocument(((DocumentPut) documentOperation).getDocument());
    } else {
        throw new IllegalStateException("Got unknown class from JSON reader: " + documentOperation.getClass().getName());
    }
    operation.setCondition(documentOperation.getCondition());
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentUpdate(com.yahoo.document.DocumentUpdate) DocumentRemove(com.yahoo.document.DocumentRemove) DocumentPut(com.yahoo.document.DocumentPut)

Example 5 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class IndexingProcessorTestCase method requireThatIndexerForwardsDocumentsOfUnknownType.

@Test
public void requireThatIndexerForwardsDocumentsOfUnknownType() {
    Document input = new Document(new DocumentType("unknown"), "doc:scheme:");
    DocumentOperation output = process(new DocumentPut(input));
    assertTrue(output instanceof DocumentPut);
    assertSame(input, ((DocumentPut) output).getDocument());
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document) Test(org.junit.Test)

Aggregations

DocumentOperation (com.yahoo.document.DocumentOperation)23 DocumentPut (com.yahoo.document.DocumentPut)13 Document (com.yahoo.document.Document)10 DocumentUpdate (com.yahoo.document.DocumentUpdate)8 Test (org.junit.Test)7 DocumentRemove (com.yahoo.document.DocumentRemove)5 DocumentType (com.yahoo.document.DocumentType)5 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)4 ByteArrayInputStream (java.io.ByteArrayInputStream)4 DocumentId (com.yahoo.document.DocumentId)3 IOException (java.io.IOException)3 InputStream (java.io.InputStream)3 Processing (com.yahoo.docproc.Processing)2 DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)2 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)2 FieldUpdate (com.yahoo.document.update.FieldUpdate)2 JsonToken (com.fasterxml.jackson.core.JsonToken)1 SplitterDocumentProcessor.doProcessOuterDocument (com.yahoo.docproc.util.SplitterDocumentProcessor.doProcessOuterDocument)1 DocumentTypeManager (com.yahoo.document.DocumentTypeManager)1 Field (com.yahoo.document.Field)1