Search in sources :

Example 46 with DocumentPut

use of com.yahoo.document.DocumentPut in project vespa by vespa-engine.

the class JoinerDocumentProcessor method process.

@Override
public Progress process(Processing processing) {
    if (!doProcessOuterDocument(processing.getVariable(contextFieldName), documentTypeName)) {
        return Progress.DONE;
    }
    DocumentPut outerDoc = (DocumentPut) processing.getVariable(contextFieldName);
    @SuppressWarnings("unchecked") Array<Document> innerDocuments = (Array<Document>) outerDoc.getDocument().getFieldValue(arrayFieldName);
    if (innerDocuments == null) {
        @SuppressWarnings("unchecked") Array<Document> empty = (Array<Document>) outerDoc.getDocument().getDataType().getField(arrayFieldName).getDataType().createFieldValue();
        innerDocuments = empty;
    }
    for (DocumentOperation op : processing.getDocumentOperations()) {
        if (op instanceof DocumentPut) {
            innerDocuments.add(((DocumentPut) op).getDocument());
        } else {
            log.log(LogLevel.DEBUG, "Skipping: " + op);
        }
    }
    processing.getDocumentOperations().clear();
    processing.getDocumentOperations().add(outerDoc);
    processing.removeVariable(contextFieldName);
    return Progress.DONE;
}
Also used : Array(com.yahoo.document.datatypes.Array) DocumentOperation(com.yahoo.document.DocumentOperation) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document) SplitterDocumentProcessor.doProcessOuterDocument(com.yahoo.docproc.util.SplitterDocumentProcessor.doProcessOuterDocument)

Example 47 with DocumentPut

use of com.yahoo.document.DocumentPut in project vespa by vespa-engine.

the class JsonReaderTestCase method createPutWithoutTensor.

private DocumentPut createPutWithoutTensor() {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("[ { \"put\": \"" + TENSOR_DOC_ID + "\", \"fields\": { } } ]"));
    JsonReader reader = new JsonReader(types, rawDoc, parserFactory);
    return (DocumentPut) reader.next();
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut)

Example 48 with DocumentPut

use of com.yahoo.document.DocumentPut in project vespa by vespa-engine.

the class JsonReaderTestCase method controlBasicFeed.

protected void controlBasicFeed(JsonReader r) {
    DocumentOperation d = r.next();
    Document doc = ((DocumentPut) d).getDocument();
    smokeTestDoc(doc);
    d = r.next();
    DocumentUpdate update = (DocumentUpdate) d;
    checkSimpleArrayAdd(update);
    d = r.next();
    DocumentRemove remove = (DocumentRemove) d;
    assertEquals("smoke", remove.getId().getDocType());
    assertNull(r.next());
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) DocumentUpdate(com.yahoo.document.DocumentUpdate) DocumentRemove(com.yahoo.document.DocumentRemove) DocumentPut(com.yahoo.document.DocumentPut) Document(com.yahoo.document.Document)

Example 49 with DocumentPut

use of com.yahoo.document.DocumentPut in project vespa by vespa-engine.

the class JsonReaderTestCase method smokeTest.

@Test
public final void smokeTest() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{\"put\": \"id:unittest:smoke::whee\"," + " \"fields\": { \"something\": \"smoketest\"," + " \"nalle\": \"bamse\"}}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    smokeTestDoc(put.getDocument());
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Example 50 with DocumentPut

use of com.yahoo.document.DocumentPut in project vespa by vespa-engine.

the class JsonReaderTestCase method docIdLookaheadTest.

@Test
public final void docIdLookaheadTest() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{" + " \"fields\": { \"something\": \"smoketest\"," + " \"nalle\": \"bamse\"}," + "\"put\": \"id:unittest:smoke::whee\"" + "}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    smokeTestDoc(put.getDocument());
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Aggregations

DocumentPut (com.yahoo.document.DocumentPut)67 Document (com.yahoo.document.Document)40 DocumentType (com.yahoo.document.DocumentType)35 Test (org.junit.Test)30 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)23 ByteArrayInputStream (java.io.ByteArrayInputStream)21 InputStream (java.io.InputStream)21 DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)16 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)16 DocumentOperation (com.yahoo.document.DocumentOperation)13 DocumentId (com.yahoo.document.DocumentId)12 IntegerFieldValue (com.yahoo.document.datatypes.IntegerFieldValue)11 FieldValue (com.yahoo.document.datatypes.FieldValue)10 MapFieldValue (com.yahoo.document.datatypes.MapFieldValue)10 TensorFieldValue (com.yahoo.document.datatypes.TensorFieldValue)10 DocumentRemove (com.yahoo.document.DocumentRemove)8 DocumentUpdate (com.yahoo.document.DocumentUpdate)7 Array (com.yahoo.document.datatypes.Array)7 DocumentProcessor (com.yahoo.docproc.DocumentProcessor)6 PutDocumentMessage (com.yahoo.documentapi.messagebus.protocol.PutDocumentMessage)5