Search in sources :

Example 16 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class IndexingProcessorTestCase method requireThatIndexerProcessesUpdates.

@Test
public void requireThatIndexerProcessesUpdates() {
    DocumentType inputType = indexer.getDocumentTypeManager().getDocumentType("music");
    DocumentUpdate input = new DocumentUpdate(inputType, "doc:scheme:");
    input.addFieldUpdate(FieldUpdate.createAssign(inputType.getField("isbn"), new StringFieldValue("isbnmarker")));
    input.addFieldUpdate(FieldUpdate.createAssign(inputType.getField("artist"), new StringFieldValue("69")));
    DocumentOperation output = process(input);
    assertTrue(output instanceof DocumentUpdate);
    DocumentUpdate docUpdate = (DocumentUpdate) output;
    assertEquals(3, docUpdate.getFieldUpdates().size());
    {
        FieldUpdate fieldUpdate = docUpdate.getFieldUpdate(0);
        assertEquals("song", fieldUpdate.getField().getName());
        assertEquals(1, fieldUpdate.getValueUpdates().size());
        ValueUpdate<?> valueUpdate = fieldUpdate.getValueUpdate(0);
        assertTrue(valueUpdate instanceof AssignValueUpdate);
        assertEquals(new StringFieldValue("isbnmarker"), valueUpdate.getValue());
        fieldUpdate = docUpdate.getFieldUpdate(1);
        assertEquals("title", fieldUpdate.getField().getName());
        assertEquals(1, fieldUpdate.getValueUpdates().size());
        valueUpdate = fieldUpdate.getValueUpdate(0);
        assertTrue(valueUpdate instanceof AssignValueUpdate);
        assertEquals(new StringFieldValue("69"), valueUpdate.getValue());
    }
    {
        FieldUpdate fieldUpdate = docUpdate.getFieldUpdate(1);
        ValueUpdate<?> valueUpdate = fieldUpdate.getValueUpdate(0);
        assertEquals("title", fieldUpdate.getField().getName());
        assertTrue(valueUpdate instanceof AssignValueUpdate);
        assertEquals(new StringFieldValue("69"), valueUpdate.getValue());
    }
    {
        FieldUpdate fieldUpdate = docUpdate.getFieldUpdate(2);
        ValueUpdate<?> valueUpdate = fieldUpdate.getValueUpdate(0);
        assertEquals("isbn", fieldUpdate.getField().getName());
        assertTrue(valueUpdate instanceof AssignValueUpdate);
        assertEquals(new StringFieldValue("isbnmarker"), valueUpdate.getValue());
    }
}
Also used : ValueUpdate(com.yahoo.document.update.ValueUpdate) AssignValueUpdate(com.yahoo.document.update.AssignValueUpdate) DocumentOperation(com.yahoo.document.DocumentOperation) DocumentUpdate(com.yahoo.document.DocumentUpdate) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) FieldUpdate(com.yahoo.document.update.FieldUpdate) DocumentType(com.yahoo.document.DocumentType) AssignValueUpdate(com.yahoo.document.update.AssignValueUpdate) Test(org.junit.Test)

Example 17 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class SplitterDocumentProcessor method doProcessOuterDocument.

static boolean doProcessOuterDocument(Object o, String documentTypeName) {
    if (!(o instanceof DocumentOperation)) {
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, o + " is not a DocumentOperation.");
        }
        return false;
    }
    DocumentOperation outerDocOp = (DocumentOperation) o;
    if (!(outerDocOp instanceof DocumentPut)) {
        // this is not a put, return
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, "Given DocumentOperation is not a DocumentPut, returning. (Was given " + outerDocOp + ").");
        }
        return false;
    }
    Document outerDoc = ((DocumentPut) outerDocOp).getDocument();
    DocumentType type = outerDoc.getDataType();
    if (!type.getName().equalsIgnoreCase(documentTypeName)) {
        // this is not the right document type
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, "Given Document is of wrong type, returning. (Was given " + outerDoc + ").");
        }
        return false;
    }
    return true;
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document)

Example 18 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class JoinerDocumentProcessor method process.

@Override
public Progress process(Processing processing) {
    if (!doProcessOuterDocument(processing.getVariable(contextFieldName), documentTypeName)) {
        return Progress.DONE;
    }
    DocumentPut outerDoc = (DocumentPut) processing.getVariable(contextFieldName);
    @SuppressWarnings("unchecked") Array<Document> innerDocuments = (Array<Document>) outerDoc.getDocument().getFieldValue(arrayFieldName);
    if (innerDocuments == null) {
        @SuppressWarnings("unchecked") Array<Document> empty = (Array<Document>) outerDoc.getDocument().getDataType().getField(arrayFieldName).getDataType().createFieldValue();
        innerDocuments = empty;
    }
    for (DocumentOperation op : processing.getDocumentOperations()) {
        if (op instanceof DocumentPut) {
            innerDocuments.add(((DocumentPut) op).getDocument());
        } else {
            log.log(LogLevel.DEBUG, "Skipping: " + op);
        }
    }
    processing.getDocumentOperations().clear();
    processing.getDocumentOperations().add(outerDoc);
    processing.removeVariable(contextFieldName);
    return Progress.DONE;
}
Also used : Array(com.yahoo.document.datatypes.Array) DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document) SplitterDocumentProcessor.doProcessOuterDocument(com.yahoo.docproc.util.SplitterDocumentProcessor.doProcessOuterDocument)

Example 19 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class JsonReaderTestCase method controlBasicFeed.

protected void controlBasicFeed(JsonReader r) {
    DocumentOperation d = r.next();
    Document doc = ((DocumentPut) d).getDocument();
    smokeTestDoc(doc);
    d = r.next();
    DocumentUpdate update = (DocumentUpdate) d;
    checkSimpleArrayAdd(update);
    d = r.next();
    DocumentRemove remove = (DocumentRemove) d;
    assertEquals("smoke", remove.getId().getDocType());
    assertNull(r.next());
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentUpdate(com.yahoo.document.DocumentUpdate) DocumentRemove(com.yahoo.document.DocumentRemove) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document)

Example 20 with DocumentOperation

use of com.yahoo.document.DocumentOperation in project vespa by vespa-engine.

the class Call method unwrapSchemaMapping.

private void unwrapSchemaMapping(Processing processing) {
    final List<DocumentOperation> documentOperations = processing.getDocumentOperations();
    for (int i = 0; i < documentOperations.size(); i++) {
        DocumentOperation documentOperation = documentOperations.get(i);
        if (documentOperation instanceof DocumentPut) {
            DocumentPut putOperation = (DocumentPut) documentOperation;
            if (putOperation.getDocument() instanceof DocumentOperationWrapper) {
                DocumentOperationWrapper proxy = (DocumentOperationWrapper) putOperation.getDocument();
                documentOperations.set(i, new DocumentPut(putOperation, ((DocumentPut) proxy.getWrappedDocumentOperation()).getDocument()));
            }
        }
    }
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut)

Aggregations

DocumentOperation (com.yahoo.document.DocumentOperation)23 DocumentPut (com.yahoo.document.DocumentPut)13 Document (com.yahoo.document.Document)10 DocumentUpdate (com.yahoo.document.DocumentUpdate)8 Test (org.junit.Test)7 DocumentRemove (com.yahoo.document.DocumentRemove)5 DocumentType (com.yahoo.document.DocumentType)5 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)4 ByteArrayInputStream (java.io.ByteArrayInputStream)4 DocumentId (com.yahoo.document.DocumentId)3 IOException (java.io.IOException)3 InputStream (java.io.InputStream)3 Processing (com.yahoo.docproc.Processing)2 DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)2 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)2 FieldUpdate (com.yahoo.document.update.FieldUpdate)2 JsonToken (com.fasterxml.jackson.core.JsonToken)1 SplitterDocumentProcessor.doProcessOuterDocument (com.yahoo.docproc.util.SplitterDocumentProcessor.doProcessOuterDocument)1 DocumentTypeManager (com.yahoo.document.DocumentTypeManager)1 Field (com.yahoo.document.Field)1