Search in sources :

Example 66 with IField

use of edu.uci.ics.texera.api.field.IField in project textdb by TextDB.

the class TestConstantsRegexSplit method constructSamplePeopleTuples.

public static List<Tuple> constructSamplePeopleTuples() {
    IField[] fields1 = { new StringField("bruce"), new StringField("john Lee"), new IntegerField(46), new DoubleField(5.50), new DateTimeField(LocalDateTime.parse("1970-01-01T11:11:11")), new TextField("banana") };
    IField[] fields2 = { new StringField("tom hanks"), new StringField("cruise"), new IntegerField(45), new DoubleField(5.95), new DateTimeField(LocalDateTime.parse("1980-01-02T13:14:15")), new TextField("mississippi") };
    Tuple tuple1 = new Tuple(SCHEMA_PEOPLE, fields1);
    Tuple tuple2 = new Tuple(SCHEMA_PEOPLE, fields2);
    return Arrays.asList(tuple1, tuple2);
}
Also used : StringField(edu.uci.ics.texera.api.field.StringField) TextField(edu.uci.ics.texera.api.field.TextField) IntegerField(edu.uci.ics.texera.api.field.IntegerField) DateTimeField(edu.uci.ics.texera.api.field.DateTimeField) IField(edu.uci.ics.texera.api.field.IField) DoubleField(edu.uci.ics.texera.api.field.DoubleField) Tuple(edu.uci.ics.texera.api.tuple.Tuple)

Example 67 with IField

use of edu.uci.ics.texera.api.field.IField in project textdb by TextDB.

the class TupleJsonDeserializer method deserialize.

@Override
public Tuple deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
    JsonNode node = p.getCodec().readTree(p);
    JsonNode schemaNode = node.get(JsonConstants.SCHEMA);
    JsonNode fieldsNode = node.get(JsonConstants.FIELDS);
    Schema schema = new ObjectMapper().treeToValue(schemaNode, Schema.class);
    ArrayList<IField> fields = new ArrayList<>();
    for (int i = 0; i < schema.getAttributes().size(); i++) {
        AttributeType attributeType = schema.getAttributes().get(i).getType();
        JsonNode fieldNode = fieldsNode.get(i);
        IField field = new ObjectMapper().treeToValue(fieldNode, attributeType.getFieldClass());
        fields.add(field);
    }
    return new Tuple(schema, fields);
}
Also used : AttributeType(edu.uci.ics.texera.api.schema.AttributeType) Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) JsonNode(com.fasterxml.jackson.databind.JsonNode) IField(edu.uci.ics.texera.api.field.IField) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper)

Example 68 with IField

use of edu.uci.ics.texera.api.field.IField in project textdb by TextDB.

the class SpanTupleTest method testGetters.

@Test
public void testGetters() throws ParseException {
    // create data tuple first
    Attribute[] attributes = new Attribute[TestConstants.ATTRIBUTES_PEOPLE.length + 1];
    for (int count = 0; count < attributes.length - 1; count++) {
        attributes[count] = TestConstants.ATTRIBUTES_PEOPLE[count];
    }
    attributes[attributes.length - 1] = SchemaConstants.SPAN_LIST_ATTRIBUTE;
    List<IField> fields = new ArrayList<IField>(Arrays.asList(new IField[] { new StringField("bruce"), new StringField("lee"), new IntegerField(46), new DoubleField(5.50), new DateField(new SimpleDateFormat("MM-dd-yyyy").parse("01-14-1970")), new TextField("bruce was born in new york city and was grown up in los angeles") }));
    IField spanField = createSpanListField();
    fields.add(spanField);
    spanTuple = new Tuple(new Schema(attributes), fields.toArray(new IField[fields.size()]));
    IField spanFieldRetrieved = spanTuple.getField(SchemaConstants.SPAN_LIST);
    Assert.assertTrue(spanFieldRetrieved instanceof ListField);
    Assert.assertSame(spanField, spanFieldRetrieved);
}
Also used : Attribute(edu.uci.ics.texera.api.schema.Attribute) Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) IntegerField(edu.uci.ics.texera.api.field.IntegerField) ListField(edu.uci.ics.texera.api.field.ListField) IField(edu.uci.ics.texera.api.field.IField) StringField(edu.uci.ics.texera.api.field.StringField) TextField(edu.uci.ics.texera.api.field.TextField) DateField(edu.uci.ics.texera.api.field.DateField) SimpleDateFormat(java.text.SimpleDateFormat) DoubleField(edu.uci.ics.texera.api.field.DoubleField) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Example 69 with IField

use of edu.uci.ics.texera.api.field.IField in project textdb by TextDB.

the class SpanTupleTest method createSpanListField.

private IField createSpanListField() {
    List<Span> list = new ArrayList<Span>();
    // The key value will be:
    // For RegexMatcher : "n.*k"
    // For NamedEntityMatcher : LOCATION
    // For DictionaryMatcher: "new york" - For DictionaryMatcher the key and
    // value are same
    // For KeyWordMatcher: "new york" - the value can be "new" or "york"
    Span span1 = new Span("description", 18, 26, "LOCATION", "new york");
    Span span2 = new Span("description", 52, 63, "LOCATION", "los angeles");
    list.add(span1);
    list.add(span2);
    IField spanListField = new ListField<Span>(list);
    return spanListField;
}
Also used : ArrayList(java.util.ArrayList) ListField(edu.uci.ics.texera.api.field.ListField) IField(edu.uci.ics.texera.api.field.IField) Span(edu.uci.ics.texera.api.span.Span)

Example 70 with IField

use of edu.uci.ics.texera.api.field.IField in project textdb by TextDB.

the class LabeledRegexMatcherTest method testMultipleLabeledRegex.

@Test
public void testMultipleLabeledRegex() throws Exception {
    String query = "<lab1>.*<lab2>";
    List<Tuple> exactResults = RegexMatcherTestHelper.getQueryResults(PEOPLE_TABLE, query, "short", Arrays.asList(TestConstants.DESCRIPTION), "lab1", false, Integer.MAX_VALUE, 0, "angry", "lab2");
    List<Tuple> expectedResults = new ArrayList<>();
    // expected to match "Short angry" and "Short and lin clooney is Angry"
    List<Tuple> data = TestConstants.getSamplePeopleTuples();
    Schema spanSchema = new Schema.Builder().add(TestConstants.SCHEMA_PEOPLE).add(RESULTS, AttributeType.LIST).build();
    List<Span> spans = new ArrayList<>();
    spans.add(new Span(TestConstants.DESCRIPTION, 0, 11, query, "Short angry"));
    IField spanField = new ListField<>(new ArrayList<>(spans));
    List<IField> fields = new ArrayList<>(data.get(5).getFields());
    fields.add(spanField);
    expectedResults.add(new Tuple(spanSchema, fields.toArray(new IField[fields.size()])));
    spans.clear();
    spans.add(new Span(TestConstants.DESCRIPTION, 15, 45, query, "Short and lin clooney is Angry"));
    spanField = new ListField<>(new ArrayList<>(spans));
    fields = new ArrayList<>(data.get(3).getFields());
    fields.add(spanField);
    expectedResults.add(new Tuple(spanSchema, fields.toArray(new IField[fields.size()])));
    List<String> attributeNames = new ArrayList<>();
    attributeNames.add(RESULTS);
    Assert.assertTrue(TestUtils.attributeEquals(expectedResults, exactResults, attributeNames));
}
Also used : Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) ListField(edu.uci.ics.texera.api.field.ListField) IField(edu.uci.ics.texera.api.field.IField) Span(edu.uci.ics.texera.api.span.Span) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Aggregations

IField (edu.uci.ics.texera.api.field.IField)145 ArrayList (java.util.ArrayList)113 Tuple (edu.uci.ics.texera.api.tuple.Tuple)106 TextField (edu.uci.ics.texera.api.field.TextField)100 Span (edu.uci.ics.texera.api.span.Span)99 Schema (edu.uci.ics.texera.api.schema.Schema)92 Test (org.junit.Test)84 StringField (edu.uci.ics.texera.api.field.StringField)79 IntegerField (edu.uci.ics.texera.api.field.IntegerField)78 DoubleField (edu.uci.ics.texera.api.field.DoubleField)63 DateField (edu.uci.ics.texera.api.field.DateField)58 Attribute (edu.uci.ics.texera.api.schema.Attribute)57 SimpleDateFormat (java.text.SimpleDateFormat)56 ListField (edu.uci.ics.texera.api.field.ListField)32 Dictionary (edu.uci.ics.texera.dataflow.dictionarymatcher.Dictionary)29 JoinDistancePredicate (edu.uci.ics.texera.dataflow.join.JoinDistancePredicate)9 KeywordMatcherSourceOperator (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator)9 IOperator (edu.uci.ics.texera.api.dataflow.IOperator)7 JsonNode (com.fasterxml.jackson.databind.JsonNode)4 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)4