Search in sources :

Example 31 with Tuple

use of edu.uci.ics.texera.api.tuple.Tuple in project textdb by TextDB.

the class JoinDistanceTest method testIdsMatchFieldsMatchSpanWithinThreshold.

/*
     * This case tests for the scenario when the difference of keyword spans
     *  is within the given span threshold.
     *  e.g.
     *  [<11, 18>]
     *  [<27, 33>]
     *  threshold = 20 (within threshold)
     *  result: [<11, 33>]
     *  
     * Test result: The list contains a tuple with all the fields and a span
     * list consisting of the joined span. The joined span is made up of the
     * field name, start and stop index (computed as <min(span1 spanStartIndex,
     * span2 spanStartIndex), max(span1 spanEndIndex, span2 spanEndIndex)>)
     * key (combination of span1 key and span2 key) and value (combination of
     * span1 value and span2 value).
     * 
     */
@Test
public void testIdsMatchFieldsMatchSpanWithinThreshold() throws Exception {
    JoinTestHelper.insertToTable(BOOK_TABLE, JoinTestConstants.bookGroup1.get(0));
    KeywordMatcherSourceOperator keywordSourceOuter = JoinTestHelper.getKeywordSource(BOOK_TABLE, "special", conjunction);
    KeywordMatcherSourceOperator keywordSourceInner = JoinTestHelper.getKeywordSource(BOOK_TABLE, "writer", conjunction);
    List<Tuple> resultList = JoinTestHelper.getJoinDistanceResults(keywordSourceInner, keywordSourceOuter, new JoinDistancePredicate(JoinTestConstants.REVIEW, 20), Integer.MAX_VALUE, 0);
    Schema resultSchema = new Schema.Builder().add(JoinTestConstants.BOOK_SCHEMA).add(SchemaConstants.SPAN_LIST_ATTRIBUTE).build();
    List<Span> spanList = new ArrayList<>();
    Span span1 = new Span(JoinTestConstants.REVIEW, 11, 33, "special_writer", "special kind of " + "writer");
    spanList.add(span1);
    IField[] book1 = { new IntegerField(52), new StringField("Mary Roach"), new StringField("Grunt: The Curious Science of Humans at War"), new IntegerField(288), new TextField("It takes a special kind " + "of writer to make topics ranging from death to our " + "gastrointestinal tract interesting (sometimes " + "hilariously so), and pop science writer Mary Roach is " + "always up to the task."), new ListField<>(spanList) };
    Tuple expectedTuple = new Tuple(resultSchema, book1);
    List<Tuple> expectedResult = new ArrayList<>();
    expectedResult.add(expectedTuple);
    Assert.assertEquals(1, resultList.size());
    Assert.assertTrue(TestUtils.equals(expectedResult, resultList));
}
Also used : Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) IntegerField(edu.uci.ics.texera.api.field.IntegerField) IField(edu.uci.ics.texera.api.field.IField) JoinDistancePredicate(edu.uci.ics.texera.dataflow.join.JoinDistancePredicate) Span(edu.uci.ics.texera.api.span.Span) KeywordMatcherSourceOperator(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator) StringField(edu.uci.ics.texera.api.field.StringField) TextField(edu.uci.ics.texera.api.field.TextField) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Example 32 with Tuple

use of edu.uci.ics.texera.api.tuple.Tuple in project textdb by TextDB.

the class JoinDistanceTest method testOneSpanEncompassesOtherAndDifferenceLessThanThreshold.

// This case tests for the scenario when one of the spans to be joined encompasses the other span
// and both |(span 1 spanStartIndex) - (span 2 spanStartIndex)|,
// |(span 1 spanEndIndex) - (span 2 spanEndIndex)| are within threshold.
// e.g.
// [<11, 18>]
// [<3, 33>]
// threshold = 20 (within threshold)
// Test result: The bigger span should be returned.
// [<3, 33>]
@Test
public void testOneSpanEncompassesOtherAndDifferenceLessThanThreshold() throws Exception {
    JoinTestHelper.insertToTable(BOOK_TABLE, JoinTestConstants.bookGroup1.get(0));
    KeywordMatcherSourceOperator keywordSourceOuter = JoinTestHelper.getKeywordSource(BOOK_TABLE, "special", conjunction);
    KeywordMatcherSourceOperator keywordSourceInner = JoinTestHelper.getKeywordSource(BOOK_TABLE, "takes a special kind of writer", phrase);
    List<Tuple> resultList = JoinTestHelper.getJoinDistanceResults(keywordSourceInner, keywordSourceOuter, new JoinDistancePredicate(JoinTestConstants.REVIEW, 20), Integer.MAX_VALUE, 0);
    Schema resultSchema = new Schema.Builder().add(JoinTestConstants.BOOK_SCHEMA).add(SchemaConstants.SPAN_LIST_ATTRIBUTE).build();
    List<Span> spanList = new ArrayList<>();
    Span span1 = new Span(JoinTestConstants.REVIEW, 3, 33, "special_takes a special " + "kind of writer", "takes a special " + "kind of writer");
    spanList.add(span1);
    IField[] book1 = { new IntegerField(52), new StringField("Mary Roach"), new StringField("Grunt: The Curious Science of Humans at War"), new IntegerField(288), new TextField("It takes a special kind " + "of writer to make topics ranging from death to our " + "gastrointestinal tract interesting (sometimes " + "hilariously so), and pop science writer Mary Roach is " + "always up to the task."), new ListField<>(spanList) };
    Tuple expectedTuple = new Tuple(resultSchema, book1);
    List<Tuple> expectedResult = new ArrayList<>();
    expectedResult.add(expectedTuple);
    Assert.assertEquals(1, resultList.size());
    Assert.assertTrue(TestUtils.equals(expectedResult, resultList));
}
Also used : Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) IntegerField(edu.uci.ics.texera.api.field.IntegerField) IField(edu.uci.ics.texera.api.field.IField) JoinDistancePredicate(edu.uci.ics.texera.dataflow.join.JoinDistancePredicate) Span(edu.uci.ics.texera.api.span.Span) KeywordMatcherSourceOperator(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator) StringField(edu.uci.ics.texera.api.field.StringField) TextField(edu.uci.ics.texera.api.field.TextField) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Example 33 with Tuple

use of edu.uci.ics.texera.api.tuple.Tuple in project textdb by TextDB.

the class JoinDistanceTest method testForLimitWhenLimitIsLesserThanActualNumberOfResults.

// ---------------------<Limit and offset test cases.>---------------------
/*
     * This case tests for the scenario when limit is some integer greater than
     * 0 and less than the actual number of results and offset is 0 and join 
     * is performed.
     * Test result: A list of tuples with number of tuples equal to limit.
     */
@Test
public void testForLimitWhenLimitIsLesserThanActualNumberOfResults() throws Exception {
    List<Tuple> tuples = JoinTestConstants.bookGroup1.subList(1, 5);
    JoinTestHelper.insertToTable(BOOK_TABLE, tuples);
    KeywordMatcherSourceOperator keywordSourceOuter = JoinTestHelper.getKeywordSource(BOOK_TABLE, "typical", conjunction);
    KeywordMatcherSourceOperator keywordSourceInner = JoinTestHelper.getKeywordSource(BOOK_TABLE, "actually", conjunction);
    List<Tuple> resultList = JoinTestHelper.getJoinDistanceResults(keywordSourceInner, keywordSourceOuter, new JoinDistancePredicate(JoinTestConstants.REVIEW, 90), 3, 0);
    Schema resultSchema = new Schema.Builder().add(JoinTestConstants.BOOK_SCHEMA).add(SchemaConstants.SPAN_LIST_ATTRIBUTE).build();
    List<Span> spanList = new ArrayList<>();
    Span span1 = new Span(JoinTestConstants.REVIEW, 28, 119, "typical_actually", "typical review. " + "This is a test. A book review test. " + "A test to test queries without actually");
    spanList.add(span1);
    Span span2 = new Span(JoinTestConstants.REVIEW, 186, 234, "typical_actually", "actually a review " + "even if it is not your typical");
    spanList.add(span2);
    IField[] book1 = { new IntegerField(51), new StringField("author unknown"), new StringField("typical"), new IntegerField(300), new TextField("Review of a Book. This is a typical " + "review. This is a test. A book review " + "test. A test to test queries without " + "actually using actual review. From " + "here onwards, we can pretend this to " + "be actually a review even if it is not " + "your typical book review."), new ListField<>(spanList) };
    IField[] book2 = { new IntegerField(53), new StringField("Noah Hawley"), new StringField("Before the Fall"), new IntegerField(400), new TextField("Review of a Book. This is a typical " + "review. This is a test. A book review " + "test. A test to test queries without " + "actually using actual review. From " + "here onwards, we can pretend this to " + "be actually a review even if it is not " + "your typical book review."), new ListField<>(spanList) };
    IField[] book3 = { new IntegerField(54), new StringField("Andria Williams"), new StringField("The Longest Night: A Novel"), new IntegerField(400), new TextField("Review of a Book. This is a typical " + "review. This is a test. A book review " + "test. A test to test queries without " + "actually using actual review. From " + "here onwards, we can pretend this to " + "be actually a review even if it is not " + "your typical book review."), new ListField<>(spanList) };
    IField[] book4 = { new IntegerField(55), new StringField("Matti Friedman"), new StringField("Pumpkinflowers: A Soldier's " + "Story"), new IntegerField(256), new TextField("Review of a Book. This is a typical " + "review. This is a test. A book review " + "test. A test to test queries without " + "actually using actual review. From " + "here onwards, we can pretend this to " + "be actually a review even if it is not " + "your typical book review."), new ListField<>(spanList) };
    Tuple expectedTuple1 = new Tuple(resultSchema, book1);
    Tuple expectedTuple2 = new Tuple(resultSchema, book2);
    Tuple expectedTuple3 = new Tuple(resultSchema, book3);
    Tuple expectedTuple4 = new Tuple(resultSchema, book4);
    List<Tuple> expectedResult = new ArrayList<>(3);
    expectedResult.add(expectedTuple1);
    expectedResult.add(expectedTuple2);
    expectedResult.add(expectedTuple3);
    expectedResult.add(expectedTuple4);
    Assert.assertEquals(3, resultList.size());
    Assert.assertTrue(TestUtils.containsAll(expectedResult, resultList));
}
Also used : Schema(edu.uci.ics.texera.api.schema.Schema) ArrayList(java.util.ArrayList) IntegerField(edu.uci.ics.texera.api.field.IntegerField) IField(edu.uci.ics.texera.api.field.IField) JoinDistancePredicate(edu.uci.ics.texera.dataflow.join.JoinDistancePredicate) Span(edu.uci.ics.texera.api.span.Span) KeywordMatcherSourceOperator(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator) StringField(edu.uci.ics.texera.api.field.StringField) TextField(edu.uci.ics.texera.api.field.TextField) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Example 34 with Tuple

use of edu.uci.ics.texera.api.tuple.Tuple in project textdb by TextDB.

the class JoinDistanceTest method testOffsetGreaterThanNumberOfResults.

/*
     * This case tests for the scenario when offset is some integer greater 
     * than 0 and greater than the actual number of results and join is 
     * performed.
     * Test result: An empty list.
     */
@Test
public void testOffsetGreaterThanNumberOfResults() throws Exception {
    List<Tuple> tuples = JoinTestConstants.bookGroup1.subList(1, 5);
    JoinTestHelper.insertToTable(BOOK_TABLE, tuples);
    KeywordMatcherSourceOperator keywordSourceOuter = JoinTestHelper.getKeywordSource(BOOK_TABLE, "typical", conjunction);
    KeywordMatcherSourceOperator keywordSourceInner = JoinTestHelper.getKeywordSource(BOOK_TABLE, "actually", conjunction);
    List<Tuple> resultList = JoinTestHelper.getJoinDistanceResults(keywordSourceInner, keywordSourceOuter, new JoinDistancePredicate(JoinTestConstants.REVIEW, 90), 1, 10);
    Assert.assertEquals(0, resultList.size());
}
Also used : JoinDistancePredicate(edu.uci.ics.texera.dataflow.join.JoinDistancePredicate) Tuple(edu.uci.ics.texera.api.tuple.Tuple) KeywordMatcherSourceOperator(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator) Test(org.junit.Test)

Example 35 with Tuple

use of edu.uci.ics.texera.api.tuple.Tuple in project textdb by TextDB.

the class JoinTestHelper method insertToTable.

public static void insertToTable(String tableName, Tuple... tuples) throws StorageException {
    RelationManager relationManager = RelationManager.getInstance();
    DataWriter tableDataWriter = relationManager.getTableDataWriter(tableName);
    tableDataWriter.open();
    for (Tuple tuple : Arrays.asList(tuples)) {
        tableDataWriter.insertTuple(tuple);
    }
    tableDataWriter.close();
}
Also used : Tuple(edu.uci.ics.texera.api.tuple.Tuple) RelationManager(edu.uci.ics.texera.storage.RelationManager) DataWriter(edu.uci.ics.texera.storage.DataWriter)

Aggregations

Tuple (edu.uci.ics.texera.api.tuple.Tuple)280 ArrayList (java.util.ArrayList)167 Test (org.junit.Test)158 Schema (edu.uci.ics.texera.api.schema.Schema)108 IField (edu.uci.ics.texera.api.field.IField)106 Span (edu.uci.ics.texera.api.span.Span)99 TextField (edu.uci.ics.texera.api.field.TextField)90 StringField (edu.uci.ics.texera.api.field.StringField)83 IntegerField (edu.uci.ics.texera.api.field.IntegerField)80 Attribute (edu.uci.ics.texera.api.schema.Attribute)75 DoubleField (edu.uci.ics.texera.api.field.DoubleField)59 DateField (edu.uci.ics.texera.api.field.DateField)55 SimpleDateFormat (java.text.SimpleDateFormat)53 DataWriter (edu.uci.ics.texera.storage.DataWriter)32 Dictionary (edu.uci.ics.texera.dataflow.dictionarymatcher.Dictionary)30 ListField (edu.uci.ics.texera.api.field.ListField)27 ScanBasedSourceOperator (edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator)21 ScanSourcePredicate (edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate)21 KeywordMatcherSourceOperator (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator)20 RelationManager (edu.uci.ics.texera.storage.RelationManager)19