Search in sources :

Example 1 with DocumentParseInfo

use of com.yahoo.document.json.readers.DocumentParseInfo in project vespa by vespa-engine.

the class JsonReader method next.

public DocumentOperation next() {
    switch(state) {
        case AT_START:
            JsonToken t = nextToken(parser);
            expectArrayStart(t);
            state = ReaderState.READING;
            break;
        case END_OF_FEED:
            return null;
        case READING:
            break;
    }
    Optional<DocumentParseInfo> documentParseInfo;
    try {
        documentParseInfo = parseDocument();
    } catch (IOException r) {
        // Jackson is not able to recover from structural parse errors
        state = END_OF_FEED;
        throw new RuntimeException(r);
    }
    if (!documentParseInfo.isPresent()) {
        state = END_OF_FEED;
        return null;
    }
    VespaJsonDocumentReader vespaJsonDocumentReader = new VespaJsonDocumentReader();
    DocumentOperation operation = vespaJsonDocumentReader.createDocumentOperation(getDocumentTypeFromString(documentParseInfo.get().documentId.getDocType(), typeManager), documentParseInfo.get());
    operation.setCondition(TestAndSetCondition.fromConditionString(documentParseInfo.get().condition));
    return operation;
}
Also used : DocumentOperation(com.yahoo.document.DocumentOperation) JsonToken(com.fasterxml.jackson.core.JsonToken) IOException(java.io.IOException) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader)

Example 2 with DocumentParseInfo

use of com.yahoo.document.json.readers.DocumentParseInfo in project vespa by vespa-engine.

the class DocumentParser method parse.

public Optional<DocumentParseInfo> parse(Optional<DocumentId> documentIdArg) throws IOException {
    indentLevel = 0;
    DocumentParseInfo documentParseInfo = new DocumentParseInfo();
    documentIdArg.ifPresent(documentId -> documentParseInfo.documentId = documentId);
    do {
        parseOneItem(documentParseInfo, documentIdArg.isPresent());
    } while (indentLevel > 0L);
    if (documentParseInfo.documentId != null) {
        return Optional.of(documentParseInfo);
    }
    return Optional.empty();
}
Also used : DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo)

Example 3 with DocumentParseInfo

use of com.yahoo.document.json.readers.DocumentParseInfo in project vespa by vespa-engine.

the class JsonReaderTestCase method parseUpdate.

private DocumentUpdate parseUpdate(String json) throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes(json));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentUpdate update = new DocumentUpdate(docType, parseInfo.documentId);
    new VespaJsonDocumentReader().readUpdate(parseInfo.fieldsBuffer, update);
    return update;
}
Also used : DocumentUpdate(com.yahoo.document.DocumentUpdate) ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentType(com.yahoo.document.DocumentType) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader)

Example 4 with DocumentParseInfo

use of com.yahoo.document.json.readers.DocumentParseInfo in project vespa by vespa-engine.

the class JsonReaderTestCase method testMap.

@Test
public final void testMap() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{\"put\": \"id:unittest:testmap::whee\"," + " \"fields\": { \"actualmap\": {" + " \"nalle\": \"kalle\", \"tralle\": \"skalle\"}}}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    Document doc = put.getDocument();
    FieldValue f = doc.getFieldValue(doc.getField("actualmap"));
    assertSame(MapFieldValue.class, f.getClass());
    MapFieldValue<?, ?> m = (MapFieldValue<?, ?>) f;
    assertEquals(2, m.size());
    assertEquals(new StringFieldValue("kalle"), m.get(new StringFieldValue("nalle")));
    assertEquals(new StringFieldValue("skalle"), m.get(new StringFieldValue("tralle")));
}
Also used : MapFieldValue(com.yahoo.document.datatypes.MapFieldValue) ByteArrayInputStream(java.io.ByteArrayInputStream) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) FieldValue(com.yahoo.document.datatypes.FieldValue) IntegerFieldValue(com.yahoo.document.datatypes.IntegerFieldValue) TensorFieldValue(com.yahoo.document.datatypes.TensorFieldValue) MapFieldValue(com.yahoo.document.datatypes.MapFieldValue) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Example 5 with DocumentParseInfo

use of com.yahoo.document.json.readers.DocumentParseInfo in project vespa by vespa-engine.

the class JsonReaderTestCase method emptyDocTest.

@Test
public final void emptyDocTest() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{\"put\": \"id:unittest:smoke::whee\"," + " \"fields\": {}}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    assertEquals("id:unittest:smoke::whee", parseInfo.documentId.toString());
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Aggregations

DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)20 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)19 DocumentType (com.yahoo.document.DocumentType)17 ByteArrayInputStream (java.io.ByteArrayInputStream)17 InputStream (java.io.InputStream)17 Document (com.yahoo.document.Document)16 DocumentPut (com.yahoo.document.DocumentPut)16 Test (org.junit.Test)15 FieldValue (com.yahoo.document.datatypes.FieldValue)10 IntegerFieldValue (com.yahoo.document.datatypes.IntegerFieldValue)10 MapFieldValue (com.yahoo.document.datatypes.MapFieldValue)10 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)10 TensorFieldValue (com.yahoo.document.datatypes.TensorFieldValue)10 Array (com.yahoo.document.datatypes.Array)3 DocumentOperation (com.yahoo.document.DocumentOperation)2 IOException (java.io.IOException)2 JsonToken (com.fasterxml.jackson.core.JsonToken)1 JsonStringEncoder (com.fasterxml.jackson.core.io.JsonStringEncoder)1 DocumentId (com.yahoo.document.DocumentId)1 DocumentUpdate (com.yahoo.document.DocumentUpdate)1