Example 96 with StringFieldValue
use of com.yahoo.document.datatypes.StringFieldValue in project vespa by vespa-engine.
the class SerializeAnnotationsTestCase method testSerializeSimpleTree.
public void testSerializeSimpleTree() throws IOException {
SpanList root = new SpanList();
root.add(new Span(0, 19)).add(new Span(19, 5)).add(new Span(24, 21)).add(new Span(45, 23)).add(new Span(68, 14));
SpanTree tree = new SpanTree("html", root);
StringFieldValue value = new StringFieldValue("lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj " + "lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj l jlkj lkj lkj " + "lkjoijoij oij oij oij oij oij oijoijoij oij oij oij oij oij " + "oijoijoijoijoijoijoijoijoijoijoijoijoij oij oij oij oij " + "oijaosdifjoai fdoais jdoasi jai os oafoai ai dfojsfoa dfoi dsf" + "aosifjasofija sodfij oasdifj aosdiosifjsi ooai oais osi");
value.setSpanTree(tree);
/*
Important note! The iteration order of annotations in SpanTree.iterator() is non-deterministic, meaning
that the order which annotations are serialized will differ between test runs. Thus, we cannot assert
that a serialized buffer is equal to a buffer written earlier. We can, however, assert that the size stays
the same, and the deserialized values from the buffers should be equal.
*/
// important! call readFile() before writeFile()!
ByteBuffer serializedFromFile = readFile("test_data_serialized_simple");
ByteBuffer serialized = writeFile(value, "test_data_serialized_simple");
assertEquals(serialized.limit(), serializedFromFile.limit());
StringFieldValue valueFromFile = new StringFieldValue();
DocumentDeserializer deserializer = DocumentDeserializerFactory.create42(docMan, new GrowableByteBuffer(serializedFromFile));
deserializer.read(null, valueFromFile);
assertEquals(value, valueFromFile);
}
Also used :
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
GrowableByteBuffer(com.yahoo.io.GrowableByteBuffer)
ByteBuffer(java.nio.ByteBuffer)
GrowableByteBuffer(com.yahoo.io.GrowableByteBuffer)
Example 97 with StringFieldValue
use of com.yahoo.document.datatypes.StringFieldValue in project vespa by vespa-engine.
the class ValueUpdateTestCase method testUpdateSimple.
public void testUpdateSimple() {
/**
* We cannot test much on this level anyway, most stuff in ValueUpdate is package
* private. Better tests exist in FieldUpdateTestCase.
*/
AssignValueUpdate upd = (AssignValueUpdate) ValueUpdate.createAssign(new StringFieldValue("newvalue"));
assertEquals(ValueUpdate.ValueUpdateClassID.ASSIGN, upd.getValueUpdateClassID());
FieldValue newValue = upd.getValue();
assertEquals(new StringFieldValue("newvalue"), newValue);
}
Also used :
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
FieldValue(com.yahoo.document.datatypes.FieldValue)
Example 98 with StringFieldValue
use of com.yahoo.document.datatypes.StringFieldValue in project vespa by vespa-engine.
the class SpanListTestCase method serializeAndAssert.
private void serializeAndAssert(SpanList spanList) {
GrowableByteBuffer buffer;
{
buffer = new GrowableByteBuffer(1024);
DocumentSerializer serializer = DocumentSerializerFactory.create42(buffer);
StringFieldValue value = new StringFieldValue("lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lkj lk");
SpanTree tree = new SpanTree("bababa", spanList);
value.setSpanTree(tree);
serializer.write(null, value);
buffer.flip();
}
SpanList spanList2;
{
DocumentDeserializer deserializer = DocumentDeserializerFactory.create42(man, buffer);
StringFieldValue value = new StringFieldValue();
deserializer.read(null, value);
spanList2 = (SpanList) value.getSpanTree("bababa").getRoot();
}
assertEquals(spanList, spanList2);
assertNotSame(spanList, spanList2);
}
Also used :
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
GrowableByteBuffer(com.yahoo.io.GrowableByteBuffer)
Example 99 with StringFieldValue
use of com.yahoo.document.datatypes.StringFieldValue in project vespa by vespa-engine.
the class DocTestCase method simple7.
public void simple7() {
StringFieldValue text = new StringFieldValue("<html><head><title>Diary</title></head><body>I live in San Francisco</body></html>");
// 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
SpanTree tree = text.getSpanTree("html");
Iterator<Annotation> annotationIt = tree.iterator();
while (annotationIt.hasNext()) {
Annotation annotation = annotationIt.next();
if (annotation.getType().getName().equals("markup")) {
// we have an annotation of type markup, remove it:
annotationIt.remove();
}
}
}
Also used :
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
Example 100 with StringFieldValue
use of com.yahoo.document.datatypes.StringFieldValue in project vespa by vespa-engine.
the class DocTestCase method simple4.
public void simple4() {
// the following line works inside process(Document, Arguments, Processing) in a DocumentProcessor
AnnotationTypeRegistry atr = processing.getService().getDocumentTypeManager().getAnnotationTypeRegistry();
StringFieldValue text = new StringFieldValue("<html><head><title>Diary</title></head><body>I live in San Francisco</body></html>");
// 012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
SpanList root = new SpanList();
SpanTree tree = new SpanTree("html", root);
AnnotationType textType = atr.getType("text");
AnnotationType beginTag = atr.getType("begintag");
AnnotationType endTag = atr.getType("endtag");
AnnotationType bodyType = atr.getType("body");
AnnotationType headerType = atr.getType("header");
AnnotationType cityType = atr.getType("city");
Struct position = (Struct) cityType.getDataType().createFieldValue();
position.setFieldValue("latitude", 37.774929);
position.setFieldValue("longitude", -122.419415);
Annotation city = new Annotation(cityType, position);
SpanList header = new SpanList();
{
Span span1 = new Span(6, 6);
Span span2 = new Span(12, 7);
Span span3 = new Span(19, 5);
Span span4 = new Span(24, 8);
Span span5 = new Span(32, 7);
header.add(span1).add(span2).add(span3).add(span4).add(span5);
tree.annotate(span1, beginTag).annotate(span2, beginTag).annotate(span3, textType).annotate(span4, endTag).annotate(span4, endTag).annotate(header, headerType);
}
SpanList textNode = new SpanList();
{
Span span1 = new Span(45, 10);
Span span2 = new Span(55, 13);
textNode.add(span1).add(span2);
tree.annotate(span2, city).annotate(textNode, textType);
}
SpanList body = new SpanList();
{
Span span1 = new Span(39, 6);
Span span2 = new Span(68, 7);
body.add(span1).add(textNode).add(span2);
tree.annotate(span1, beginTag).annotate(span2, endTag).annotate(body, bodyType);
}
{
Span span1 = new Span(0, 6);
Span span2 = new Span(75, 7);
root.add(span1).add(header).add(body).add(span2);
tree.annotate(span1, beginTag).annotate(span2, endTag);
}
text.setSpanTree(tree);
}
Also used :
StringFieldValue(com.yahoo.document.datatypes.StringFieldValue)
Struct(com.yahoo.document.datatypes.Struct)
Aggregations
StringFieldValue (com.yahoo.document.datatypes.StringFieldValue)210
Test (org.junit.Test)136
FieldValue (com.yahoo.document.datatypes.FieldValue)49
SimpleTestAdapter (com.yahoo.vespa.indexinglanguage.SimpleTestAdapter)40
IntegerFieldValue (com.yahoo.document.datatypes.IntegerFieldValue)37
Document (com.yahoo.document.Document)30
Array (com.yahoo.document.datatypes.Array)25
DocumentType (com.yahoo.document.DocumentType)21
LongFieldValue (com.yahoo.document.datatypes.LongFieldValue)21
Struct (com.yahoo.document.datatypes.Struct)21
DocumentPut (com.yahoo.document.DocumentPut)20
GrowableByteBuffer (com.yahoo.io.GrowableByteBuffer)20
MapFieldValue (com.yahoo.document.datatypes.MapFieldValue)18
SpanTree (com.yahoo.document.annotation.SpanTree)15
FieldUpdate (com.yahoo.document.update.FieldUpdate)15
DocumentUpdate (com.yahoo.document.DocumentUpdate)12
Field (com.yahoo.document.Field)12
Annotation (com.yahoo.document.annotation.Annotation)12
FloatFieldValue (com.yahoo.document.datatypes.FloatFieldValue)12
HashMap (java.util.HashMap)12
