Search in sources :

Example 6 with ProjectionPredicate

use of edu.uci.ics.texera.dataflow.projection.ProjectionPredicate in project textdb by TextDB.

the class JoinDistanceTest method testOneOfTheOperatorResultContainsNoSpan.

// This case tests for the scenario when one of the operators result lists has no span.
// If one of the operators doesn't have span, then an exception will be thrown.
// Test result: DataflowException is thrown
@Test(expected = DataflowException.class)
public void testOneOfTheOperatorResultContainsNoSpan() throws Exception {
    JoinTestHelper.insertToTable(BOOK_TABLE, JoinTestConstants.bookGroup1.get(0));
    KeywordMatcherSourceOperator keywordSourceOuter = JoinTestHelper.getKeywordSource(BOOK_TABLE, "special", conjunction);
    String fuzzyTokenQuery = "this writer writes well";
    double thresholdRatio = 0.25;
    List<String> textAttributeNames = JoinTestConstants.BOOK_SCHEMA.getAttributes().stream().filter(attr -> attr.getType() != AttributeType.TEXT).map(Attribute::getName).collect(Collectors.toList());
    FuzzyTokenSourcePredicate fuzzySourcePredicateInner = new FuzzyTokenSourcePredicate(fuzzyTokenQuery, textAttributeNames, LuceneAnalyzerConstants.standardAnalyzerString(), thresholdRatio, BOOK_TABLE, SchemaConstants.SPAN_LIST);
    FuzzyTokenMatcherSourceOperator fuzzyMatcherInner = new FuzzyTokenMatcherSourceOperator(fuzzySourcePredicateInner);
    ProjectionPredicate removeSpanListPredicate = new ProjectionPredicate(JoinTestConstants.BOOK_SCHEMA.getAttributeNames());
    ProjectionOperator removeSpanListProjection = new ProjectionOperator(removeSpanListPredicate);
    removeSpanListProjection.setInputOperator(fuzzyMatcherInner);
    JoinTestHelper.getJoinDistanceResults(keywordSourceOuter, removeSpanListProjection, new JoinDistancePredicate(JoinTestConstants.REVIEW, 20), Integer.MAX_VALUE, 0);
}
Also used : FuzzyTokenMatcherSourceOperator(edu.uci.ics.texera.dataflow.fuzzytokenmatcher.FuzzyTokenMatcherSourceOperator) ProjectionOperator(edu.uci.ics.texera.dataflow.projection.ProjectionOperator) FuzzyTokenSourcePredicate(edu.uci.ics.texera.dataflow.fuzzytokenmatcher.FuzzyTokenSourcePredicate) ProjectionPredicate(edu.uci.ics.texera.dataflow.projection.ProjectionPredicate) JoinDistancePredicate(edu.uci.ics.texera.dataflow.join.JoinDistancePredicate) KeywordMatcherSourceOperator(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator) Test(org.junit.Test)

Example 7 with ProjectionPredicate

use of edu.uci.ics.texera.dataflow.projection.ProjectionPredicate in project textdb by TextDB.

the class ProjectSomeFieldsPredicate method generateOperatorBean.

/**
 * Return this operator converted to an { @code OperatorBean }.
 * @param projectOperatorId The ID of the OperatorBean to be created.
 */
public PredicateBase generateOperatorBean(String projectOperatorId) {
    ProjectionPredicate projectionPredicate = new ProjectionPredicate(this.getProjectedFields());
    projectionPredicate.setID(projectOperatorId);
    return projectionPredicate;
}
Also used : ProjectionPredicate(edu.uci.ics.texera.dataflow.projection.ProjectionPredicate)

Example 8 with ProjectionPredicate

use of edu.uci.ics.texera.dataflow.projection.ProjectionPredicate in project textdb by TextDB.

the class SelectStatementTest method testSelectStatementBeansBuilder05.

/**
 * Test the correctness of the generated beans by a SelectStatement with a
 * ProjectAllFieldsPredicate and a KeywordExtractPredicate.
 * Get a graph by calling getInternalPredicateBases() and getInternalLinkBeans()
 * methods and check if the generated path form the node getInputNodeID() to
 * the node getOutputNodeID() is correct. Also check whether getInputViews()
 * is returning the correct dependencies.
 */
@Test
public void testSelectStatementBeansBuilder05() {
    ProjectPredicate projectPredicate = new ProjectSomeFieldsPredicate(Arrays.asList("a", "b"));
    ExtractPredicate extractPredicate = new KeywordExtractPredicate(Arrays.asList("a", "b"), "x", KeywordMatchingType.SUBSTRING_SCANBASED.toString());
    SelectStatement selectStatement = new SelectStatement("_sid4", projectPredicate, extractPredicate, "from", null, null);
    List<PredicateBase> expectedGeneratedBeans = Arrays.asList(new KeywordPredicate("x", Arrays.asList("a", "b"), null, KeywordMatchingType.SUBSTRING_SCANBASED, "_sid4_e"), new ProjectionPredicate(Arrays.asList("a", "b")));
    List<String> dependencies = Arrays.asList("from");
    Assert.assertEquals(selectStatement.getInputViews(), dependencies);
    StatementTestUtils.assertGeneratedBeans(selectStatement, expectedGeneratedBeans);
}
Also used : SelectStatement(edu.uci.ics.texera.textql.statements.SelectStatement) ProjectPredicate(edu.uci.ics.texera.textql.statements.predicates.ProjectPredicate) PredicateBase(edu.uci.ics.texera.dataflow.common.PredicateBase) ExtractPredicate(edu.uci.ics.texera.textql.statements.predicates.ExtractPredicate) KeywordExtractPredicate(edu.uci.ics.texera.textql.statements.predicates.KeywordExtractPredicate) ProjectSomeFieldsPredicate(edu.uci.ics.texera.textql.statements.predicates.ProjectSomeFieldsPredicate) KeywordExtractPredicate(edu.uci.ics.texera.textql.statements.predicates.KeywordExtractPredicate) ProjectionPredicate(edu.uci.ics.texera.dataflow.projection.ProjectionPredicate) KeywordPredicate(edu.uci.ics.texera.dataflow.keywordmatcher.KeywordPredicate) Test(org.junit.Test)

Example 9 with ProjectionPredicate

use of edu.uci.ics.texera.dataflow.projection.ProjectionPredicate in project textdb by TextDB.

the class ProjectSomeFieldsPredicateTest method testGenerateOperatorBean00.

/**
 * Test the generateOperatorBean method.
 * Build a SelectSomeFieldsPredicate, invoke the generateOperatorBean and check
 * whether a ProjectionBean with the right attributes is returned.
 * An empty list is used as the list of projected fields.
 */
@Test
public void testGenerateOperatorBean00() {
    String operatorId = "xxx";
    List<String> projectedFields = Collections.emptyList();
    ProjectSomeFieldsPredicate projectSomeFieldsPredicate = new ProjectSomeFieldsPredicate(projectedFields);
    PredicateBase computedProjectionBean = projectSomeFieldsPredicate.generateOperatorBean(operatorId);
    PredicateBase expectedProjectionBean = new ProjectionPredicate(Arrays.asList());
    expectedProjectionBean.setID(operatorId);
    Assert.assertEquals(expectedProjectionBean, computedProjectionBean);
}
Also used : PredicateBase(edu.uci.ics.texera.dataflow.common.PredicateBase) ProjectionPredicate(edu.uci.ics.texera.dataflow.projection.ProjectionPredicate) Test(org.junit.Test)

Aggregations

ProjectionPredicate (edu.uci.ics.texera.dataflow.projection.ProjectionPredicate)9 Test (org.junit.Test)8 PredicateBase (edu.uci.ics.texera.dataflow.common.PredicateBase)5 ProjectionOperator (edu.uci.ics.texera.dataflow.projection.ProjectionOperator)2 SelectStatement (edu.uci.ics.texera.textql.statements.SelectStatement)2 ProjectPredicate (edu.uci.ics.texera.textql.statements.predicates.ProjectPredicate)2 ProjectSomeFieldsPredicate (edu.uci.ics.texera.textql.statements.predicates.ProjectSomeFieldsPredicate)2 IOperator (edu.uci.ics.texera.api.dataflow.IOperator)1 IField (edu.uci.ics.texera.api.field.IField)1 TextField (edu.uci.ics.texera.api.field.TextField)1 Schema (edu.uci.ics.texera.api.schema.Schema)1 Tuple (edu.uci.ics.texera.api.tuple.Tuple)1 FuzzyTokenMatcherSourceOperator (edu.uci.ics.texera.dataflow.fuzzytokenmatcher.FuzzyTokenMatcherSourceOperator)1 FuzzyTokenSourcePredicate (edu.uci.ics.texera.dataflow.fuzzytokenmatcher.FuzzyTokenSourcePredicate)1 JoinDistancePredicate (edu.uci.ics.texera.dataflow.join.JoinDistancePredicate)1 KeywordMatcherSourceOperator (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordMatcherSourceOperator)1 KeywordPredicate (edu.uci.ics.texera.dataflow.keywordmatcher.KeywordPredicate)1 ScanBasedSourceOperator (edu.uci.ics.texera.dataflow.source.scan.ScanBasedSourceOperator)1 ScanSourcePredicate (edu.uci.ics.texera.dataflow.source.scan.ScanSourcePredicate)1 ExtractPredicate (edu.uci.ics.texera.textql.statements.predicates.ExtractPredicate)1