Search in sources :

Example 6 with ScanSourcePredicate

use of edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class NlpEntityTest method getQueryResults.

public List<Tuple> getQueryResults(String tableName, List<String> attributeNames, NlpEntityType nlpEntityType, int limit, int offset) throws Exception {
    ScanBasedSourceOperator scanSource = new ScanBasedSourceOperator(new ScanSourcePredicate(tableName));
    NlpEntityPredicate nlpEntityPredicate = new NlpEntityPredicate(nlpEntityType, attributeNames, RESULTS);
    NlpEntityOperator nlpEntityOperator = new NlpEntityOperator(nlpEntityPredicate);
    nlpEntityOperator.setInputOperator(scanSource);
    nlpEntityOperator.setLimit(limit);
    nlpEntityOperator.setOffset(offset);
    Tuple nextTuple = null;
    List<Tuple> results = new ArrayList<Tuple>();
    nlpEntityOperator.open();
    while ((nextTuple = nlpEntityOperator.getNextTuple()) != null) {
        results.add(nextTuple);
    }
    nlpEntityOperator.close();
    return results;
}
Also used : ArrayList(java.util.ArrayList) ScanBasedSourceOperator(edu.uci.ics.textdb.exp.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate) Tuple(edu.uci.ics.textdb.api.tuple.Tuple)

Example 7 with ScanSourcePredicate

use of edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class ProjectionOperatorTest method testProjection1.

@Test
public void testProjection1() throws Exception {
    List<String> projectionFields = Arrays.asList(TestConstants.DESCRIPTION);
    Schema projectionSchema = new Schema(TestConstants.DESCRIPTION_ATTR);
    IField[] fields1 = { new TextField("Tall Angry") };
    IField[] fields2 = { new TextField("Short Brown") };
    IField[] fields3 = { new TextField("White Angry") };
    IField[] fields4 = { new TextField("Lin Clooney is Short and lin clooney is Angry") };
    IField[] fields5 = { new TextField("Tall Fair") };
    IField[] fields6 = { new TextField("Short angry") };
    Tuple tuple1 = new Tuple(projectionSchema, fields1);
    Tuple tuple2 = new Tuple(projectionSchema, fields2);
    Tuple tuple3 = new Tuple(projectionSchema, fields3);
    Tuple tuple4 = new Tuple(projectionSchema, fields4);
    Tuple tuple5 = new Tuple(projectionSchema, fields5);
    Tuple tuple6 = new Tuple(projectionSchema, fields6);
    List<Tuple> expectedResults = Arrays.asList(tuple1, tuple2, tuple3, tuple4, tuple5, tuple6);
    List<Tuple> returnedResults = getProjectionResults(new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE)), projectionFields);
    Assert.assertTrue(TestUtils.equals(expectedResults, returnedResults));
}
Also used : Schema(edu.uci.ics.textdb.api.schema.Schema) TextField(edu.uci.ics.textdb.api.field.TextField) IField(edu.uci.ics.textdb.api.field.IField) Tuple(edu.uci.ics.textdb.api.tuple.Tuple) ScanBasedSourceOperator(edu.uci.ics.textdb.exp.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate) Test(org.junit.Test)

Example 8 with ScanSourcePredicate

use of edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class OneToNBroadcastConnectorTest method testThreeOutputsWithItself.

/*
     * This test tests if the connectors' three outputs are the same.
     */
@Test
public void testThreeOutputsWithItself() throws Exception {
    IOperator sourceOperator = new ScanBasedSourceOperator(new ScanSourcePredicate(PEOPLE_TABLE));
    OneToNBroadcastConnector connector = new OneToNBroadcastConnector(3);
    connector.setInputOperator(sourceOperator);
    IOperator output1 = connector.getOutputOperator(0);
    IOperator output2 = connector.getOutputOperator(1);
    IOperator output3 = connector.getOutputOperator(2);
    output1.open();
    output2.open();
    output3.open();
    List<Tuple> output1Results = new ArrayList<>();
    Tuple nextTuple = null;
    while ((nextTuple = output1.getNextTuple()) != null) {
        output1Results.add(nextTuple);
    }
    List<Tuple> output2Results = new ArrayList<>();
    nextTuple = null;
    while ((nextTuple = output2.getNextTuple()) != null) {
        output2Results.add(nextTuple);
    }
    List<Tuple> output3Results = new ArrayList<>();
    nextTuple = null;
    while ((nextTuple = output3.getNextTuple()) != null) {
        output3Results.add(nextTuple);
    }
    output1.close();
    output2.close();
    output3.close();
    List<Tuple> expectedResults = TestConstants.getSamplePeopleTuples();
    Assert.assertTrue(TestUtils.equals(expectedResults, output1Results));
    Assert.assertTrue(TestUtils.equals(expectedResults, output2Results));
    Assert.assertTrue(TestUtils.equals(expectedResults, output3Results));
}
Also used : IOperator(edu.uci.ics.textdb.api.dataflow.IOperator) ArrayList(java.util.ArrayList) ScanBasedSourceOperator(edu.uci.ics.textdb.exp.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate) Tuple(edu.uci.ics.textdb.api.tuple.Tuple) Test(org.junit.Test)

Example 9 with ScanSourcePredicate

use of edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class PredicateBaseTest method testScanSource.

@Test
public void testScanSource() throws Exception {
    ScanSourcePredicate scanSourcePredicate = new ScanSourcePredicate("tableName");
    testPredicate(scanSourcePredicate);
}
Also used : ScanSourcePredicate(edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate) Test(org.junit.Test)

Example 10 with ScanSourcePredicate

use of edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate in project textdb by TextDB.

the class WordCountTest method computePayLoadWordCount.

//Compute result by tuple's PayLoad.
public static HashMap<String, Integer> computePayLoadWordCount(String tableName, String attribute) throws TextDBException {
    ScanBasedSourceOperator scanSource = new ScanBasedSourceOperator(new ScanSourcePredicate(tableName));
    WordCountOperator wordCount = null;
    HashMap<String, Integer> result = new HashMap<String, Integer>();
    if (tableName.equals(COUNT_TABLE)) {
        wordCount = new WordCountOperator(new WordCountOperatorPredicate(TestConstants.DESCRIPTION, LuceneAnalyzerConstants.standardAnalyzerString()));
    } else if (tableName.equals(COUNT_CHINESE_TABLE)) {
        wordCount = new WordCountOperator(new WordCountOperatorPredicate(TestConstantsChineseWordCount.DESCRIPTION, LuceneAnalyzerConstants.chineseAnalyzerString()));
    }
    wordCount.setInputOperator(scanSource);
    wordCount.open();
    Tuple tuple;
    while ((tuple = wordCount.getNextTuple()) != null) {
        result.put((String) tuple.getField(WordCountOperator.WORD).getValue(), (Integer) tuple.getField(WordCountOperator.COUNT).getValue());
    }
    wordCount.close();
    return result;
}
Also used : HashMap(java.util.HashMap) WordCountOperator(edu.uci.ics.textdb.exp.wordcount.WordCountOperator) WordCountOperatorPredicate(edu.uci.ics.textdb.exp.wordcount.WordCountOperatorPredicate) ScanBasedSourceOperator(edu.uci.ics.textdb.exp.source.scan.ScanBasedSourceOperator) ScanSourcePredicate(edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate) Tuple(edu.uci.ics.textdb.api.tuple.Tuple)

Aggregations

ScanSourcePredicate (edu.uci.ics.textdb.exp.source.scan.ScanSourcePredicate)21 ScanBasedSourceOperator (edu.uci.ics.textdb.exp.source.scan.ScanBasedSourceOperator)20 Tuple (edu.uci.ics.textdb.api.tuple.Tuple)17 ArrayList (java.util.ArrayList)11 Test (org.junit.Test)7 IField (edu.uci.ics.textdb.api.field.IField)3 TextField (edu.uci.ics.textdb.api.field.TextField)3 Schema (edu.uci.ics.textdb.api.schema.Schema)3 IOperator (edu.uci.ics.textdb.api.dataflow.IOperator)2 DataFlowException (edu.uci.ics.textdb.api.exception.DataFlowException)2 RelationManager (edu.uci.ics.textdb.storage.RelationManager)2 ISourceOperator (edu.uci.ics.textdb.api.dataflow.ISourceOperator)1 TextDBException (edu.uci.ics.textdb.api.exception.TextDBException)1 StringField (edu.uci.ics.textdb.api.field.StringField)1 Attribute (edu.uci.ics.textdb.api.schema.Attribute)1 Span (edu.uci.ics.textdb.api.span.Span)1 DictionaryPredicate (edu.uci.ics.textdb.exp.dictionarymatcher.DictionaryPredicate)1 FuzzyTokenPredicate (edu.uci.ics.textdb.exp.fuzzytokenmatcher.FuzzyTokenPredicate)1 KeywordMatcherSourceOperator (edu.uci.ics.textdb.exp.keywordmatcher.KeywordMatcherSourceOperator)1 KeywordSourcePredicate (edu.uci.ics.textdb.exp.keywordmatcher.KeywordSourcePredicate)1