Search in sources :

Example 26 with DocumentType

use of com.yahoo.document.DocumentType in project vespa by vespa-engine.

the class FailingDocumentProcessingTestCase method assertProcessingWorks.

protected void assertProcessingWorks(DocprocService service) {
    // Create documents
    DocumentType type = new DocumentType("test");
    type.addField("test", DataType.STRING);
    DocumentPut put1 = new DocumentPut(type, new DocumentId("doc:failing:test:1"));
    DocumentPut put2 = new DocumentPut(type, new DocumentId("doc:failing:test:2"));
    DocumentPut put3 = new DocumentPut(type, new DocumentId("doc:failing:test:3"));
    // Process them
    service.process(put1);
    service.process(put2);
    service.process(put3);
    while (service.doWork()) {
    }
    // Verify
    assertEquals(new StringFieldValue("done 3"), put1.getDocument().getFieldValue("test"));
    // Due to exception in 2
    assertEquals(new StringFieldValue("done 2"), put2.getDocument().getFieldValue("test"));
    assertEquals(new StringFieldValue("done 3"), put3.getDocument().getFieldValue("test"));
}
Also used : StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentPut(com.yahoo.document.DocumentPut) DocumentId(com.yahoo.document.DocumentId) DocumentType(com.yahoo.document.DocumentType)

Example 27 with DocumentType

use of com.yahoo.document.DocumentType in project vespa by vespa-engine.

the class FailingPermanentlyDocumentProcessingTestCase method assertProcessingWorks.

protected void assertProcessingWorks(DocprocService service) {
    // Create documents
    DocumentType type = new DocumentType("test");
    type.addField("test", DataType.STRING);
    DocumentPut put1 = new DocumentPut(type, new DocumentId("doc:permanentfailure:test:1"));
    DocumentPut put2 = new DocumentPut(type, new DocumentId("doc:permanentfailure:test:2"));
    DocumentPut put3 = new DocumentPut(type, new DocumentId("doc:permanentfailure:test:3"));
    // Process them
    service.process(put1);
    service.process(put2);
    service.process(put3);
    while (service.doWork()) {
    }
    // Verify
    assertEquals(new StringFieldValue("done 3"), put1.getDocument().getFieldValue("test"));
    // Due to PERMANENT_FAILURE in 2
    assertEquals(new StringFieldValue("done 2"), put2.getDocument().getFieldValue("test"));
    // service is disabled now
    assertNull(put3.getDocument().getFieldValue("test"));
    assertFalse(service.isInService());
    service.setInService(true);
    while (service.doWork()) {
    }
    assertEquals(new StringFieldValue("done 3"), put3.getDocument().getFieldValue("test"));
    assertTrue(service.isInService());
}
Also used : StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) DocumentPut(com.yahoo.document.DocumentPut) DocumentId(com.yahoo.document.DocumentId) DocumentType(com.yahoo.document.DocumentType)

Example 28 with DocumentType

use of com.yahoo.document.DocumentType in project vespa by vespa-engine.

the class JsonReaderTestCase method parseUpdate.

private DocumentUpdate parseUpdate(String json) throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes(json));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentUpdate update = new DocumentUpdate(docType, parseInfo.documentId);
    new VespaJsonDocumentReader().readUpdate(parseInfo.fieldsBuffer, update);
    return update;
}
Also used : DocumentUpdate(com.yahoo.document.DocumentUpdate) ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentType(com.yahoo.document.DocumentType) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader)

Example 29 with DocumentType

use of com.yahoo.document.DocumentType in project vespa by vespa-engine.

the class JsonReaderTestCase method testMap.

@Test
public final void testMap() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{\"put\": \"id:unittest:testmap::whee\"," + " \"fields\": { \"actualmap\": {" + " \"nalle\": \"kalle\", \"tralle\": \"skalle\"}}}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    Document doc = put.getDocument();
    FieldValue f = doc.getFieldValue(doc.getField("actualmap"));
    assertSame(MapFieldValue.class, f.getClass());
    MapFieldValue<?, ?> m = (MapFieldValue<?, ?>) f;
    assertEquals(2, m.size());
    assertEquals(new StringFieldValue("kalle"), m.get(new StringFieldValue("nalle")));
    assertEquals(new StringFieldValue("skalle"), m.get(new StringFieldValue("tralle")));
}
Also used : MapFieldValue(com.yahoo.document.datatypes.MapFieldValue) ByteArrayInputStream(java.io.ByteArrayInputStream) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) StringFieldValue(com.yahoo.document.datatypes.StringFieldValue) FieldValue(com.yahoo.document.datatypes.FieldValue) IntegerFieldValue(com.yahoo.document.datatypes.IntegerFieldValue) TensorFieldValue(com.yahoo.document.datatypes.TensorFieldValue) MapFieldValue(com.yahoo.document.datatypes.MapFieldValue) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Example 30 with DocumentType

use of com.yahoo.document.DocumentType in project vespa by vespa-engine.

the class JsonReaderTestCase method emptyDocTest.

@Test
public final void emptyDocTest() throws IOException {
    InputStream rawDoc = new ByteArrayInputStream(Utf8.toBytes("{\"put\": \"id:unittest:smoke::whee\"," + " \"fields\": {}}"));
    JsonReader r = new JsonReader(types, rawDoc, parserFactory);
    DocumentParseInfo parseInfo = r.parseDocument().get();
    DocumentType docType = r.readDocumentType(parseInfo.documentId);
    DocumentPut put = new DocumentPut(new Document(docType, parseInfo.documentId));
    new VespaJsonDocumentReader().readPut(parseInfo.fieldsBuffer, put);
    assertEquals("id:unittest:smoke::whee", parseInfo.documentId.toString());
}
Also used : ByteArrayInputStream(java.io.ByteArrayInputStream) ByteArrayInputStream(java.io.ByteArrayInputStream) InputStream(java.io.InputStream) DocumentPut(com.yahoo.document.DocumentPut) DocumentType(com.yahoo.document.DocumentType) Document(com.yahoo.document.Document) DocumentParseInfo(com.yahoo.document.json.readers.DocumentParseInfo) VespaJsonDocumentReader(com.yahoo.document.json.readers.VespaJsonDocumentReader) Test(org.junit.Test)

Aggregations

DocumentType (com.yahoo.document.DocumentType)98 Test (org.junit.Test)45 Document (com.yahoo.document.Document)41 DocumentPut (com.yahoo.document.DocumentPut)35 Field (com.yahoo.document.Field)24 StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)24 DocumentId (com.yahoo.document.DocumentId)20 ByteArrayInputStream (java.io.ByteArrayInputStream)19 InputStream (java.io.InputStream)19 DocumentTypeManager (com.yahoo.document.DocumentTypeManager)17 DocumentParseInfo (com.yahoo.document.json.readers.DocumentParseInfo)17 VespaJsonDocumentReader (com.yahoo.document.json.readers.VespaJsonDocumentReader)17 DocumentUpdate (com.yahoo.document.DocumentUpdate)15 IntegerFieldValue (com.yahoo.document.datatypes.IntegerFieldValue)15 StructDataType (com.yahoo.document.StructDataType)14 TensorDataType (com.yahoo.document.TensorDataType)12 TensorFieldValue (com.yahoo.document.datatypes.TensorFieldValue)11 ArrayDataType (com.yahoo.document.ArrayDataType)10 MapDataType (com.yahoo.document.MapDataType)10 ReferenceDataType (com.yahoo.document.ReferenceDataType)10