Search in sources :

Example 6 with ScanSourcePredicate

use of edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class JoinTestHelper method getRegexMatcher.

public static RegexMatcher getRegexMatcher(String tableName, String query, String attrName) {
    try {
        ScanBasedSourceOperator scanBasedSourceOperator = new ScanBasedSourceOperator(new ScanSourcePredicate(tableName));
        RegexMatcher regexMatcher = new RegexMatcher(new RegexPredicate(query, Arrays.asList(attrName), SchemaConstants.SPAN_LIST));
        regexMatcher.setInputOperator(scanBasedSourceOperator);
        return regexMatcher;
    } catch (DataflowException e) {
        e.printStackTrace();
        return null;
    }
}
Also used : RegexPredicate(edu.uci.ics.texera.dataflow.regexmatcher.RegexPredicate) DataflowException(edu.uci.ics.texera.api.exception.DataflowException) RegexMatcher(edu.uci.ics.texera.dataflow.regexmatcher.RegexMatcher) ScanBasedSourceOperator(edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate)

Example 7 with ScanSourcePredicate

use of edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class ProjectionOperatorTest method testProjection1.

@Test
public void testProjection1() throws Exception {
    List<String> projectionFields = Arrays.asList(TestConstants.DESCRIPTION);
    Schema projectionSchema = new Schema(TestConstants.DESCRIPTION_ATTR);
    IField[] fields1 = { new TextField("Tall Angry") };
    IField[] fields2 = { new TextField("Short Brown") };
    IField[] fields3 = { new TextField("White Angry") };
    IField[] fields4 = { new TextField("Lin Clooney is Short and lin clooney is Angry") };
    IField[] fields5 = { new TextField("Tall Fair") };
    IField[] fields6 = { new TextField("Short angry") };
    Tuple tuple1 = new Tuple(projectionSchema, fields1);
    Tuple tuple2 = new Tuple(projectionSchema, fields2);
    Tuple tuple3 = new Tuple(projectionSchema, fields3);
    Tuple tuple4 = new Tuple(projectionSchema, fields4);
    Tuple tuple5 = new Tuple(projectionSchema, fields5);
    Tuple tuple6 = new Tuple(projectionSchema, fields6);
    List<Tuple> expectedResults = Arrays.asList(tuple1, tuple2, tuple3, tuple4, tuple5, tuple6);
    List<Tuple> returnedResults = getProjectionResults(new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE)), projectionFields);
    Assert.assertTrue(TestUtils.equals(expectedResults, returnedResults));
}
Also used : Schema(edu.uci.ics.texera.api.schema.Schema) TextField(edu.uci.ics.texera.api.field.TextField) IField(edu.uci.ics.texera.api.field.IField) Tuple(edu.uci.ics.texera.api.tuple.Tuple) ScanBasedSourceOperator(edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate) Test(org.junit.Test)

Example 8 with ScanSourcePredicate

use of edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class ComparableMatcherTest method setDefaultMatcherConfig.

public void setDefaultMatcherConfig(ComparableMatcher comparableMatcher) throws TexeraException {
    // Perform the query
    ScanBasedSourceOperator sourceOperator = new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE));
    comparableMatcher.setInputOperator(sourceOperator);
    comparableMatcher.setLimit(Integer.MAX_VALUE);
    comparableMatcher.setOffset(0);
}
Also used : ScanBasedSourceOperator(edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate)

Example 9 with ScanSourcePredicate

use of edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class ComparableMatcherTest method testDateTime2.

@Test
public void testDateTime2() throws Exception {
    // Prepare the query
    String dateCompared = "1970-01-01T11:11:12";
    String attributeName = TestConstants.DATE_OF_BIRTH_ATTR.getName();
    ComparisonType matchingType = ComparisonType.LESS_THAN;
    ComparablePredicate comparablePredicate = new ComparablePredicate(attributeName, matchingType, dateCompared);
    ComparableMatcher comparableMatcher = new ComparableMatcher(comparablePredicate);
    setDefaultMatcherConfig(comparableMatcher);
    // Perform the query
    ScanBasedSourceOperator sourceOperator = new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE_2));
    comparableMatcher.setInputOperator(sourceOperator);
    comparableMatcher.open();
    comparableMatcher.setLimit(Integer.MAX_VALUE);
    comparableMatcher.setOffset(0);
    List<Tuple> returnedResults = new ArrayList<>();
    Tuple nextTuple = null;
    while ((nextTuple = comparableMatcher.getNextTuple()) != null) {
        returnedResults.add(nextTuple);
    }
    comparableMatcher.close();
    List<Tuple> expectedResults = new ArrayList<>();
    expectedResults.add(TestConstantsRegexSplit.constructSamplePeopleTuples().get(0));
    // check the results
    Assert.assertEquals(expectedResults.size(), returnedResults.size());
    Assert.assertTrue(TestUtils.equals(expectedResults, returnedResults));
}
Also used : ArrayList(java.util.ArrayList) ScanBasedSourceOperator(edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Example 10 with ScanSourcePredicate

use of edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class ComparableMatcherTest method testDateTime1.

@Test
public void testDateTime1() throws Exception {
    // Prepare the query
    String dateCompared = "1970-01-01T11:11:11";
    String attributeName = TestConstants.DATE_OF_BIRTH_ATTR.getName();
    ComparisonType matchingType = ComparisonType.EQUAL_TO;
    ComparablePredicate comparablePredicate = new ComparablePredicate(attributeName, matchingType, dateCompared);
    ComparableMatcher comparableMatcher = new ComparableMatcher(comparablePredicate);
    setDefaultMatcherConfig(comparableMatcher);
    // Perform the query
    ScanBasedSourceOperator sourceOperator = new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE_2));
    comparableMatcher.setInputOperator(sourceOperator);
    comparableMatcher.open();
    comparableMatcher.setLimit(Integer.MAX_VALUE);
    comparableMatcher.setOffset(0);
    List<Tuple> returnedResults = new ArrayList<>();
    Tuple nextTuple = null;
    while ((nextTuple = comparableMatcher.getNextTuple()) != null) {
        returnedResults.add(nextTuple);
    }
    comparableMatcher.close();
    List<Tuple> expectedResults = new ArrayList<>();
    expectedResults.add(TestConstantsRegexSplit.constructSamplePeopleTuples().get(0));
    // check the results
    Assert.assertEquals(expectedResults.size(), returnedResults.size());
    Assert.assertTrue(TestUtils.equals(expectedResults, returnedResults));
}
Also used : ArrayList(java.util.ArrayList) ScanBasedSourceOperator(edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate) Tuple(edu.uci.ics.texera.api.tuple.Tuple) Test(org.junit.Test)

Aggregations

ScanSourcePredicate (edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate)26 ScanBasedSourceOperator (edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator)25 Tuple (edu.uci.ics.texera.api.tuple.Tuple)21 ArrayList (java.util.ArrayList)15 Test (org.junit.Test)9 IField (edu.uci.ics.texera.api.field.IField)3 TextField (edu.uci.ics.texera.api.field.TextField)3 Schema (edu.uci.ics.texera.api.schema.Schema)3 IOperator (edu.uci.ics.texera.api.dataflow.IOperator)2 RelationManager (edu.uci.ics.texera.storage.RelationManager)2 ISourceOperator (edu.uci.ics.texera.api.dataflow.ISourceOperator)1 DataflowException (edu.uci.ics.texera.api.exception.DataflowException)1 StringField (edu.uci.ics.texera.api.field.StringField)1 Span (edu.uci.ics.texera.api.span.Span)1 DictionaryPredicate (edu.uci.ics.texera.dataflow.dictionarymatcher.DictionaryPredicate)1 FuzzyTokenPredicate (edu.uci.ics.texera.dataflow.fuzzytokenmatcher.FuzzyTokenPredicate)1 KeywordMatcherSourceOperator (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator)1 KeywordSourcePredicate (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordSourcePredicate)1 NlpEntityOperator (edu.uci.ics.texera.dataflow.nlp.entity.NlpEntityOperator)1 NlpEntityPredicate (edu.uci.ics.texera.dataflow.nlp.entity.NlpEntityPredicate)1