Search in sources :

Example 61 with Document

use of com.yahoo.document.Document in project vespa by vespa-engine.

the class IndexingProcessorTestCase method requireThatIndexerProcessesDocuments.

@Test
public void requireThatIndexerProcessesDocuments() {
    Document input = new Document(indexer.getDocumentTypeManager().getDocumentType("music"), "doc:scheme:");
    input.setFieldValue("artist", new StringFieldValue("69"));
    DocumentOperation op = process(new DocumentPut(input));
    assertTrue(op instanceof DocumentPut);
    Document output = ((DocumentPut) op).getDocument();
    assertEquals(new StringFieldValue("69"), output.getFieldValue("title"));
    assertEquals("music", output.getDataType().getName());
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document) Test(org.junit.Test)

Example 62 with Document

use of com.yahoo.document.Document in project vespa by vespa-engine.

the class DocumentProcessingHandlerBasicTestCase method testPut.

@Test
public void testPut() throws InterruptedException {
    Document document = new Document(getType(), "doc:yalla:balla");
    document.setFieldValue("blahblah", new StringFieldValue("This is a test."));
    PutDocumentMessage message = new PutDocumentMessage(new DocumentPut(document));
    assertTrue(sendMessage("foobar", message));
    Message msg = remoteServer.awaitMessage(60, TimeUnit.SECONDS);
    assertNotNull(msg);
    remoteServer.ackMessage(msg);
    Reply reply = driver.client().awaitReply(60, TimeUnit.SECONDS);
    assertNotNull(reply);
    assertThat((msg instanceof PutDocumentMessage), is(true));
    PutDocumentMessage put = (PutDocumentMessage) msg;
    Document outDoc = put.getDocumentPut().getDocument();
    assertThat(document, equalTo(outDoc));
    assertFalse(reply.hasErrors());
}
Also used : PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) Message(com.yahoo.messagebus.Message) PutDocumentMessage(com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentPut(com.yahoo.document.DocumentPut) Reply(com.yahoo.messagebus.Reply) Document(com.yahoo.document.Document) Test(org.junit.Test)

Example 63 with Document

use of com.yahoo.document.Document in project vespa by vespa-engine.

the class SplitterDocumentProcessor method doProcessOuterDocument.

static boolean doProcessOuterDocument(Object o, String documentTypeName) {
    if (!(o instanceof DocumentOperation)) {
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, o + " is not a DocumentOperation.");
        }
        return false;
    }
    DocumentOperation outerDocOp = (DocumentOperation) o;
    if (!(outerDocOp instanceof DocumentPut)) {
        // this is not a put, return
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, "Given DocumentOperation is not a DocumentPut, returning. (Was given " + outerDocOp + ").");
        }
        return false;
    }
    Document outerDoc = ((DocumentPut) outerDocOp).getDocument();
    DocumentType type = outerDoc.getDataType();
    if (!type.getName().equalsIgnoreCase(documentTypeName)) {
        // this is not the right document type
        if (log.isLoggable(LogLevel.DEBUG)) {
            log.log(LogLevel.DEBUG, "Given Document is of wrong type, returning. (Was given " + outerDoc + ").");
        }
        return false;
    }
    return true;
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document)

Example 64 with Document

use of com.yahoo.document.Document in project vespa by vespa-engine.

the class AccessesAnnotationTestCase method requireThatFieldsAreRestricted.

@Test
public void requireThatFieldsAreRestricted() {
    DocumentType type = new DocumentType("album");
    type.addField("title", DataType.STRING);
    type.addField("artist", DataType.STRING);
    type.addField("year", DataType.INT);
    Document doc = new Document(type, new DocumentId("doc:map:test:1"));
    MyDocProc docProc = new MyDocProc();
    DocumentPut put = new DocumentPut(doc);
    Document proxy = new Call(docProc).configDoc(docProc, put).getDocument();
    proxy.setFieldValue("title", new StringFieldValue("foo"));
    try {
        proxy.setFieldValue("year", new IntegerFieldValue(69));
        fail("Should have failed");
    } catch (Exception e) {
        System.out.println(e.getMessage());
        assertTrue(e.getMessage().matches(".*not allowed.*"));
    }
    proxy.getFieldValue("title");
    try {
        proxy.getFieldValue("year");
        fail("Should have failed");
    } catch (Exception e) {
        System.out.println(e.getMessage());
        assertTrue(e.getMessage().matches(".*not allowed.*"));
    }
}
Also used : StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentId(com.yahoo.document.DocumentId) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) IntegerFieldValue(com.yahoo.document.datatypes.IntegerFieldValue) Document(com.yahoo.document.Document) Test(org.junit.Test)

Example 65 with Document

use of com.yahoo.document.Document in project vespa by vespa-engine.

the class JoinerDocumentProcessor method process.

@Override
public Progress process(Processing processing) {
    if (!doProcessOuterDocument(processing.getVariable(contextFieldName), documentTypeName)) {
        return Progress.DONE;
    }
    DocumentPut outerDoc = (DocumentPut) processing.getVariable(contextFieldName);
    @SuppressWarnings("unchecked") Array<Document> innerDocuments = (Array<Document>) outerDoc.getDocument().getFieldValue(arrayFieldName);
    if (innerDocuments == null) {
        @SuppressWarnings("unchecked") Array<Document> empty = (Array<Document>) outerDoc.getDocument().getDataType().getField(arrayFieldName).getDataType().createFieldValue();
        innerDocuments = empty;
    }
    for (DocumentOperation op : processing.getDocumentOperations()) {
        if (op instanceof DocumentPut) {
            innerDocuments.add(((DocumentPut) op).getDocument());
        } else {
            log.log(LogLevel.DEBUG, "Skipping: " + op);
        }
    }
    processing.getDocumentOperations().clear();
    processing.getDocumentOperations().add(outerDoc);
    processing.removeVariable(contextFieldName);
    return Progress.DONE;
}
Also used : Array(com.yahoo.document.datatypes.Array) DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document) SplitterDocumentProcessor.doProcessOuterDocument(com.yahoo.docproc.util.SplitterDocumentProcessor.doProcessOuterDocument)

Aggregations

Document (com.yahoo.document.Document)109 Test (org.junit.Test)63 DocumentType (com.yahoo.document.DocumentType)41 DocumentPut (com.yahoo.document.DocumentPut)40 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)33 ByteArrayInputStream (java.io.ByteArrayInputStream)23 InputStream (java.io.InputStream)18 DocumentId (com.yahoo.document.DocumentId)16 DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)16 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)16 IntegerFieldValue (com.yahoo.document.datatypes.IntegerFieldValue)13 TensorFieldValue (com.yahoo.document.datatypes.TensorFieldValue)13 HashMap (java.util.HashMap)13 DocumentTypeManager (com.yahoo.document.DocumentTypeManager)12 FieldValue (com.yahoo.document.datatypes.FieldValue)12 MapFieldValue (com.yahoo.document.datatypes.MapFieldValue)12 DocumentOperation (com.yahoo.document.DocumentOperation)10 GrowableByteBuffer (com.yahoo.io.GrowableByteBuffer)10 DocumentUpdate (com.yahoo.document.DocumentUpdate)8 Array (com.yahoo.document.datatypes.Array)8