Search in sources :

Example 21 with ASTJexlScript

use of org.apache.commons.jexl3.parser.ASTJexlScript in project datawave by NationalSecurityAgency.

the class PushDownVisitor method visit.

@Override
public Object visit(ASTJexlScript script, Object data) {
    ASTJexlScript rewrittenScript = script;
    if (log.isTraceEnabled()) {
        log.trace("Adding script " + rewrittenScript);
    }
    for (PushDownRule rule : pushDownRules) {
        rewrittenScript = (ASTJexlScript) rule.visit(rewrittenScript, this);
    }
    if (allDelayed(rewrittenScript)) {
        if (log.isTraceEnabled()) {
            log.trace("All predicates are delayed");
        }
        DelayedPredicatePushDown predicatePushDown = new DelayedPredicatePushDown();
        rewrittenScript = (ASTJexlScript) predicatePushDown.visit(rewrittenScript, this);
    }
    return rewrittenScript;
}
Also used : PushDownRule(datawave.query.planner.pushdown.rules.PushDownRule) ASTJexlScript(org.apache.commons.jexl2.parser.ASTJexlScript) DelayedPredicatePushDown(datawave.query.planner.pushdown.rules.DelayedPredicatePushDown)

Example 22 with ASTJexlScript

use of org.apache.commons.jexl3.parser.ASTJexlScript in project datawave by NationalSecurityAgency.

the class ThreadedRangeBundlerTest method whenInstantiatingViaModifiedBuilder_thenSpecifiedValuesAreSet.

@Test
public void whenInstantiatingViaModifiedBuilder_thenSpecifiedValuesAreSet() {
    QueryData original = mock(QueryData.class);
    CloseableIterable<QueryPlan> ranges = mock(CloseableIterable.class);
    Query settings = mock(Query.class);
    ASTJexlScript queryTree = mock(ASTJexlScript.class);
    Collection<Comparator<QueryPlan>> queryPlanComparators = mock(Collection.class);
    // @formatter:off
    ThreadedRangeBundler bundler = ThreadedRangeBundler.builder().setOriginal(original).setRanges(ranges).setMaxRanges(100).setSettings(settings).setQueryTree(queryTree).setDocSpecificLimitOverride(true).setDocsToCombine(10).setMaxRangeWaitMillis(1).setQueryPlanComparators(queryPlanComparators).setNumRangesToBuffer(1).setRangeBufferTimeoutMillis(10).setRangeBufferPollMillis(5).build();
    // @formatter:on
    assertEquals(original, bundler.getOriginal());
    assertEquals(ranges, bundler.getRanges());
    assertEquals(100L, bundler.getMaxRanges());
    assertEquals(settings, bundler.getSettings());
    assertEquals(queryTree, bundler.getQueryTree());
    assertTrue(bundler.isDocSpecificLimitOverride());
    assertEquals(10, bundler.getDocsToCombine());
    assertEquals(queryPlanComparators, bundler.getQueryPlanComparators());
    assertEquals(1, bundler.getNumRangesToBuffer());
    assertEquals(10L, bundler.getRangeBufferTimeoutMillis());
    assertEquals(5L, bundler.getRangeBufferPollMillis());
    assertEquals(1L, bundler.getMaxRangeWaitMillis());
}
Also used : Query(datawave.webservice.query.Query) QueryData(datawave.webservice.query.configuration.QueryData) ASTJexlScript(org.apache.commons.jexl2.parser.ASTJexlScript) Comparator(java.util.Comparator) Test(org.junit.Test)

Example 23 with ASTJexlScript

use of org.apache.commons.jexl3.parser.ASTJexlScript in project datawave by NationalSecurityAgency.

the class NodeTransformVisitorTest method testPushdown.

private void testPushdown(String original, String expected, List<NodeTransformRule> rules) throws Exception {
    // create a query tree
    ASTJexlScript originalScript = JexlASTHelper.parseJexlQuery(original);
    MockMetadataHelper helper = new MockMetadataHelper();
    // apply the visitor
    ASTJexlScript resultScript = NodeTransformVisitor.transform(originalScript, rules, new ShardQueryConfiguration(), helper);
    // Verify the script is as expected, and has a valid lineage.
    assertScriptEquality(resultScript, expected);
    assertLineage(resultScript);
    // Verify the original script was not modified, and still has a valid lineage.
    assertScriptEquality(originalScript, original);
    assertLineage(originalScript);
}
Also used : MockMetadataHelper(datawave.query.util.MockMetadataHelper) ASTJexlScript(org.apache.commons.jexl2.parser.ASTJexlScript) ShardQueryConfiguration(datawave.query.config.ShardQueryConfiguration)

Example 24 with ASTJexlScript

use of org.apache.commons.jexl3.parser.ASTJexlScript in project datawave by NationalSecurityAgency.

the class PushdownUnexecutableNodesVisitorTest method testFieldIndex.

@Test
public void testFieldIndex() throws ParseException {
    ASTJexlScript query = JexlASTHelper.parseJexlQuery(baseQuery);
    replayAll();
    // field index
    Assert.assertEquals(expectedPreFieldState, ExecutableDeterminationVisitor.getState(query, config, helper, true));
    JexlNode result = PushdownUnexecutableNodesVisitor.pushdownPredicates(query, true, config, indexedFields, indexOnlyFields, nonEventFields, helper);
    Assert.assertEquals(expectedFieldIndexPushdown, JexlStringBuildingVisitor.buildQuery(result));
    Assert.assertEquals(expectedPostFieldState, ExecutableDeterminationVisitor.getState(result, config, helper, true));
    verifyAll();
}
Also used : ASTJexlScript(org.apache.commons.jexl2.parser.ASTJexlScript) JexlNode(org.apache.commons.jexl2.parser.JexlNode) Test(org.junit.Test)

Example 25 with ASTJexlScript

use of org.apache.commons.jexl3.parser.ASTJexlScript in project datawave by NationalSecurityAgency.

the class PushdownUnexecutableNodesVisitorTest method testGlobalIndex.

@Test
public void testGlobalIndex() throws ParseException {
    ASTJexlScript query = JexlASTHelper.parseJexlQuery(baseQuery);
    replayAll();
    // global index
    Assert.assertEquals(expectedPreGlobalState, ExecutableDeterminationVisitor.getState(query, config, helper, false));
    JexlNode result = PushdownUnexecutableNodesVisitor.pushdownPredicates(query, false, config, indexedFields, indexOnlyFields, nonEventFields, helper);
    Assert.assertEquals(expectedGlobalIndexPushdown, JexlStringBuildingVisitor.buildQuery(result));
    Assert.assertEquals(expectedPostGlobalState, ExecutableDeterminationVisitor.getState(result, config, helper, false));
    verifyAll();
}
Also used : ASTJexlScript(org.apache.commons.jexl2.parser.ASTJexlScript) JexlNode(org.apache.commons.jexl2.parser.JexlNode) Test(org.junit.Test)

Aggregations

ASTJexlScript (org.apache.commons.jexl2.parser.ASTJexlScript)457 Test (org.junit.Test)385 Key (org.apache.accumulo.core.data.Key)69 JexlNode (org.apache.commons.jexl2.parser.JexlNode)61 HashSet (java.util.HashSet)50 ShardQueryConfiguration (datawave.query.config.ShardQueryConfiguration)48 MockMetadataHelper (datawave.query.util.MockMetadataHelper)40 Type (datawave.data.type.Type)38 LcNoDiacriticsType (datawave.data.type.LcNoDiacriticsType)37 ArrayList (java.util.ArrayList)37 ExpressionFilter (datawave.query.jexl.visitors.EventDataQueryExpressionVisitor.ExpressionFilter)36 ScannerFactory (datawave.query.tables.ScannerFactory)35 Date (java.util.Date)35 NoOpType (datawave.data.type.NoOpType)34 NumberType (datawave.data.type.NumberType)34 AbstractMap (java.util.AbstractMap)32 Range (org.apache.accumulo.core.data.Range)30 Value (org.apache.accumulo.core.data.Value)29 RangeFactoryForTests.makeTestRange (datawave.common.test.utils.query.RangeFactoryForTests.makeTestRange)26 QueryPlan (datawave.query.planner.QueryPlan)26