Search in sources :

Example 1 with Expression

use of com.yahoo.vespa.indexinglanguage.expressions.Expression in project vespa by vespa-engine.

the class IndexingScript method addContentInOrder.

private void addContentInOrder(IlscriptsConfig.Ilscript.Builder ilscriptBuilder) {
    ArrayList<Expression> later = new ArrayList<>();
    Set<String> touchedFields = new HashSet<String>();
    for (Expression exp : expressions) {
        FieldScanVisitor fieldFetcher = new FieldScanVisitor();
        if (modifiesSelf(exp)) {
            later.add(exp);
        } else {
            ilscriptBuilder.content(exp.toString());
        }
        fieldFetcher.visit(exp);
        touchedFields.addAll(fieldFetcher.touchedFields());
    }
    for (Expression exp : later) {
        ilscriptBuilder.content(exp.toString());
    }
    generateSyntheticStatementsForUntouchedFields(ilscriptBuilder, touchedFields);
}
Also used : StatementExpression(com.yahoo.vespa.indexinglanguage.expressions.StatementExpression) OutputExpression(com.yahoo.vespa.indexinglanguage.expressions.OutputExpression) ClearStateExpression(com.yahoo.vespa.indexinglanguage.expressions.ClearStateExpression) Expression(com.yahoo.vespa.indexinglanguage.expressions.Expression) ZCurveExpression(com.yahoo.vespa.indexinglanguage.expressions.ZCurveExpression) PassthroughExpression(com.yahoo.vespa.indexinglanguage.expressions.PassthroughExpression) GuardExpression(com.yahoo.vespa.indexinglanguage.expressions.GuardExpression) ScriptExpression(com.yahoo.vespa.indexinglanguage.expressions.ScriptExpression) InputExpression(com.yahoo.vespa.indexinglanguage.expressions.InputExpression) ArrayList(java.util.ArrayList) HashSet(java.util.HashSet)

Example 2 with Expression

use of com.yahoo.vespa.indexinglanguage.expressions.Expression in project vespa by vespa-engine.

the class ExpressionConverterTestCase method requireThatSwitchElementsCanBeRemoved.

@Test
public void requireThatSwitchElementsCanBeRemoved() {
    Map<String, Expression> cases = new HashMap<>();
    Expression foo = new AttributeExpression("foo");
    Expression bar = new AttributeExpression("bar");
    cases.put("foo", foo);
    cases.put("bar", bar);
    SwitchExpression before = new SwitchExpression(cases);
    Expression after = new SearchReplace(foo, null).convert(before);
    assertTrue(after instanceof SwitchExpression);
    assertEquals(1, ((SwitchExpression) after).getCases().size());
    after = new SearchReplace(bar, null).convert(before);
    assertTrue(after instanceof SwitchExpression);
    assertEquals(1, ((SwitchExpression) after).getCases().size());
}
Also used : SwitchExpression(com.yahoo.vespa.indexinglanguage.expressions.SwitchExpression) GetFieldExpression(com.yahoo.vespa.indexinglanguage.expressions.GetFieldExpression) SetVarExpression(com.yahoo.vespa.indexinglanguage.expressions.SetVarExpression) Base64DecodeExpression(com.yahoo.vespa.indexinglanguage.expressions.Base64DecodeExpression) SummaryExpression(com.yahoo.vespa.indexinglanguage.expressions.SummaryExpression) SelectInputExpression(com.yahoo.vespa.indexinglanguage.expressions.SelectInputExpression) IndexExpression(com.yahoo.vespa.indexinglanguage.expressions.IndexExpression) SubstringExpression(com.yahoo.vespa.indexinglanguage.expressions.SubstringExpression) ClearStateExpression(com.yahoo.vespa.indexinglanguage.expressions.ClearStateExpression) EchoExpression(com.yahoo.vespa.indexinglanguage.expressions.EchoExpression) ToByteExpression(com.yahoo.vespa.indexinglanguage.expressions.ToByteExpression) TrimExpression(com.yahoo.vespa.indexinglanguage.expressions.TrimExpression) GuardExpression(com.yahoo.vespa.indexinglanguage.expressions.GuardExpression) ScriptExpression(com.yahoo.vespa.indexinglanguage.expressions.ScriptExpression) JoinExpression(com.yahoo.vespa.indexinglanguage.expressions.JoinExpression) LowerCaseExpression(com.yahoo.vespa.indexinglanguage.expressions.LowerCaseExpression) GetVarExpression(com.yahoo.vespa.indexinglanguage.expressions.GetVarExpression) NowExpression(com.yahoo.vespa.indexinglanguage.expressions.NowExpression) HostNameExpression(com.yahoo.vespa.indexinglanguage.expressions.HostNameExpression) SwitchExpression(com.yahoo.vespa.indexinglanguage.expressions.SwitchExpression) CompositeExpression(com.yahoo.vespa.indexinglanguage.expressions.CompositeExpression) AttributeExpression(com.yahoo.vespa.indexinglanguage.expressions.AttributeExpression) CatExpression(com.yahoo.vespa.indexinglanguage.expressions.CatExpression) SetLanguageExpression(com.yahoo.vespa.indexinglanguage.expressions.SetLanguageExpression) Expression(com.yahoo.vespa.indexinglanguage.expressions.Expression) SetValueExpression(com.yahoo.vespa.indexinglanguage.expressions.SetValueExpression) ToLongExpression(com.yahoo.vespa.indexinglanguage.expressions.ToLongExpression) ZCurveExpression(com.yahoo.vespa.indexinglanguage.expressions.ZCurveExpression) ToDoubleExpression(com.yahoo.vespa.indexinglanguage.expressions.ToDoubleExpression) ArithmeticExpression(com.yahoo.vespa.indexinglanguage.expressions.ArithmeticExpression) InputExpression(com.yahoo.vespa.indexinglanguage.expressions.InputExpression) ToFloatExpression(com.yahoo.vespa.indexinglanguage.expressions.ToFloatExpression) OptimizePredicateExpression(com.yahoo.vespa.indexinglanguage.expressions.OptimizePredicateExpression) TokenizeExpression(com.yahoo.vespa.indexinglanguage.expressions.TokenizeExpression) ToArrayExpression(com.yahoo.vespa.indexinglanguage.expressions.ToArrayExpression) ToIntegerExpression(com.yahoo.vespa.indexinglanguage.expressions.ToIntegerExpression) ThisExpression(com.yahoo.vespa.indexinglanguage.expressions.ThisExpression) ForEachExpression(com.yahoo.vespa.indexinglanguage.expressions.ForEachExpression) NormalizeExpression(com.yahoo.vespa.indexinglanguage.expressions.NormalizeExpression) RandomExpression(com.yahoo.vespa.indexinglanguage.expressions.RandomExpression) Base64EncodeExpression(com.yahoo.vespa.indexinglanguage.expressions.Base64EncodeExpression) IfThenExpression(com.yahoo.vespa.indexinglanguage.expressions.IfThenExpression) HexDecodeExpression(com.yahoo.vespa.indexinglanguage.expressions.HexDecodeExpression) StatementExpression(com.yahoo.vespa.indexinglanguage.expressions.StatementExpression) ToWsetExpression(com.yahoo.vespa.indexinglanguage.expressions.ToWsetExpression) ParenthesisExpression(com.yahoo.vespa.indexinglanguage.expressions.ParenthesisExpression) HexEncodeExpression(com.yahoo.vespa.indexinglanguage.expressions.HexEncodeExpression) ToPositionExpression(com.yahoo.vespa.indexinglanguage.expressions.ToPositionExpression) ToStringExpression(com.yahoo.vespa.indexinglanguage.expressions.ToStringExpression) SplitExpression(com.yahoo.vespa.indexinglanguage.expressions.SplitExpression) HashMap(java.util.HashMap) AttributeExpression(com.yahoo.vespa.indexinglanguage.expressions.AttributeExpression) Test(org.junit.Test)

Example 3 with Expression

use of com.yahoo.vespa.indexinglanguage.expressions.Expression in project vespa by vespa-engine.

the class ExpressionConverterTestCase method requireThatScriptElementsCanBeRemoved.

@Test
public void requireThatScriptElementsCanBeRemoved() {
    StatementExpression foo = new StatementExpression(new AttributeExpression("foo"));
    StatementExpression bar = new StatementExpression(new AttributeExpression("bar"));
    ScriptExpression before = new ScriptExpression(foo, bar);
    Expression after = new SearchReplace(foo, null).convert(before);
    assertTrue(after instanceof ScriptExpression);
    assertEquals(1, ((ScriptExpression) after).size());
    assertEquals(bar, ((ScriptExpression) after).get(0));
    after = new SearchReplace(bar, null).convert(before);
    assertTrue(after instanceof ScriptExpression);
    assertEquals(1, ((ScriptExpression) after).size());
    assertEquals(foo, ((ScriptExpression) after).get(0));
}
Also used : StatementExpression(com.yahoo.vespa.indexinglanguage.expressions.StatementExpression) GetFieldExpression(com.yahoo.vespa.indexinglanguage.expressions.GetFieldExpression) SetVarExpression(com.yahoo.vespa.indexinglanguage.expressions.SetVarExpression) Base64DecodeExpression(com.yahoo.vespa.indexinglanguage.expressions.Base64DecodeExpression) SummaryExpression(com.yahoo.vespa.indexinglanguage.expressions.SummaryExpression) SelectInputExpression(com.yahoo.vespa.indexinglanguage.expressions.SelectInputExpression) IndexExpression(com.yahoo.vespa.indexinglanguage.expressions.IndexExpression) SubstringExpression(com.yahoo.vespa.indexinglanguage.expressions.SubstringExpression) ClearStateExpression(com.yahoo.vespa.indexinglanguage.expressions.ClearStateExpression) EchoExpression(com.yahoo.vespa.indexinglanguage.expressions.EchoExpression) ToByteExpression(com.yahoo.vespa.indexinglanguage.expressions.ToByteExpression) TrimExpression(com.yahoo.vespa.indexinglanguage.expressions.TrimExpression) GuardExpression(com.yahoo.vespa.indexinglanguage.expressions.GuardExpression) ScriptExpression(com.yahoo.vespa.indexinglanguage.expressions.ScriptExpression) JoinExpression(com.yahoo.vespa.indexinglanguage.expressions.JoinExpression) LowerCaseExpression(com.yahoo.vespa.indexinglanguage.expressions.LowerCaseExpression) GetVarExpression(com.yahoo.vespa.indexinglanguage.expressions.GetVarExpression) NowExpression(com.yahoo.vespa.indexinglanguage.expressions.NowExpression) HostNameExpression(com.yahoo.vespa.indexinglanguage.expressions.HostNameExpression) SwitchExpression(com.yahoo.vespa.indexinglanguage.expressions.SwitchExpression) CompositeExpression(com.yahoo.vespa.indexinglanguage.expressions.CompositeExpression) AttributeExpression(com.yahoo.vespa.indexinglanguage.expressions.AttributeExpression) CatExpression(com.yahoo.vespa.indexinglanguage.expressions.CatExpression) SetLanguageExpression(com.yahoo.vespa.indexinglanguage.expressions.SetLanguageExpression) Expression(com.yahoo.vespa.indexinglanguage.expressions.Expression) SetValueExpression(com.yahoo.vespa.indexinglanguage.expressions.SetValueExpression) ToLongExpression(com.yahoo.vespa.indexinglanguage.expressions.ToLongExpression) ZCurveExpression(com.yahoo.vespa.indexinglanguage.expressions.ZCurveExpression) ToDoubleExpression(com.yahoo.vespa.indexinglanguage.expressions.ToDoubleExpression) ArithmeticExpression(com.yahoo.vespa.indexinglanguage.expressions.ArithmeticExpression) InputExpression(com.yahoo.vespa.indexinglanguage.expressions.InputExpression) ToFloatExpression(com.yahoo.vespa.indexinglanguage.expressions.ToFloatExpression) OptimizePredicateExpression(com.yahoo.vespa.indexinglanguage.expressions.OptimizePredicateExpression) TokenizeExpression(com.yahoo.vespa.indexinglanguage.expressions.TokenizeExpression) ToArrayExpression(com.yahoo.vespa.indexinglanguage.expressions.ToArrayExpression) ToIntegerExpression(com.yahoo.vespa.indexinglanguage.expressions.ToIntegerExpression) ThisExpression(com.yahoo.vespa.indexinglanguage.expressions.ThisExpression) ForEachExpression(com.yahoo.vespa.indexinglanguage.expressions.ForEachExpression) NormalizeExpression(com.yahoo.vespa.indexinglanguage.expressions.NormalizeExpression) RandomExpression(com.yahoo.vespa.indexinglanguage.expressions.RandomExpression) Base64EncodeExpression(com.yahoo.vespa.indexinglanguage.expressions.Base64EncodeExpression) IfThenExpression(com.yahoo.vespa.indexinglanguage.expressions.IfThenExpression) HexDecodeExpression(com.yahoo.vespa.indexinglanguage.expressions.HexDecodeExpression) StatementExpression(com.yahoo.vespa.indexinglanguage.expressions.StatementExpression) ToWsetExpression(com.yahoo.vespa.indexinglanguage.expressions.ToWsetExpression) ParenthesisExpression(com.yahoo.vespa.indexinglanguage.expressions.ParenthesisExpression) HexEncodeExpression(com.yahoo.vespa.indexinglanguage.expressions.HexEncodeExpression) ToPositionExpression(com.yahoo.vespa.indexinglanguage.expressions.ToPositionExpression) ToStringExpression(com.yahoo.vespa.indexinglanguage.expressions.ToStringExpression) SplitExpression(com.yahoo.vespa.indexinglanguage.expressions.SplitExpression) AttributeExpression(com.yahoo.vespa.indexinglanguage.expressions.AttributeExpression) ScriptExpression(com.yahoo.vespa.indexinglanguage.expressions.ScriptExpression) Test(org.junit.Test)

Example 4 with Expression

use of com.yahoo.vespa.indexinglanguage.expressions.Expression in project vespa by vespa-engine.

the class AssertIndexingScript method assertIndexing.

public static void assertIndexing(List<String> expected, Iterable<Expression> actual) {
    List<String> parsedExpected = new LinkedList<>();
    for (String str : expected) {
        try {
            parsedExpected.add(Expression.fromString(str).toString());
        } catch (ParseException e) {
            fail(e.getMessage());
        }
    }
    for (Expression actualExp : actual) {
        String str = actualExp.toString();
        assertTrue("Unexpected: " + str, parsedExpected.remove(str));
    }
    assertTrue("Missing: " + parsedExpected.toString(), parsedExpected.isEmpty());
}
Also used : Expression(com.yahoo.vespa.indexinglanguage.expressions.Expression) ParseException(com.yahoo.vespa.indexinglanguage.parser.ParseException) LinkedList(java.util.LinkedList)

Example 5 with Expression

use of com.yahoo.vespa.indexinglanguage.expressions.Expression in project vespa by vespa-engine.

the class DocumentTestCase method requireThatConcatenationOfEmbracedStatementsWorks.

@Test
public void requireThatConcatenationOfEmbracedStatementsWorks() throws ParseException {
    DocumentType docType = new DocumentType("my_input");
    docType.addField(new Field("str_a", DataType.STRING));
    docType.addField(new Field("str_b", DataType.STRING));
    docType.addField(new Field("out", DataType.getArray(DataType.STRING)));
    Expression exp = Expression.fromString("(input str_a | split ',') . (input str_b | split ',') | index out");
    {
        Document doc = new Document(docType, "doc:scheme:");
        assertNotNull(doc = Expression.execute(exp, doc));
        FieldValue val = doc.getFieldValue("out");
        assertNotNull(val);
        assertEquals(DataType.getArray(DataType.STRING), val.getDataType());
        assertTrue(val instanceof Array);
        Array arr = (Array) val;
        assertEquals(0, arr.size());
    }
    {
        Document doc = new Document(docType, "doc:scheme:");
        doc.setFieldValue("str_a", new StringFieldValue("a1"));
        assertNotNull(doc = Expression.execute(exp, doc));
        FieldValue val = doc.getFieldValue("out");
        assertNotNull(val);
        assertEquals(DataType.getArray(DataType.STRING), val.getDataType());
        assertTrue(val instanceof Array);
        Array arr = (Array) val;
        assertEquals(1, arr.size());
    }
    {
        Document doc = new Document(docType, "doc:scheme:");
        doc.setFieldValue("str_a", new StringFieldValue("a1,a2"));
        doc.setFieldValue("str_b", new StringFieldValue("b1"));
        assertNotNull(doc = Expression.execute(exp, doc));
        FieldValue val = doc.getFieldValue("out");
        assertNotNull(val);
        assertEquals(DataType.getArray(DataType.STRING), val.getDataType());
        assertTrue(val instanceof Array);
        Array arr = (Array) val;
        assertEquals(3, arr.size());
    }
}
Also used : Expression(com.yahoo.vespa.indexinglanguage.expressions.Expression) Test(org.junit.Test)

Aggregations

Expression (com.yahoo.vespa.indexinglanguage.expressions.Expression)9 GuardExpression (com.yahoo.vespa.indexinglanguage.expressions.GuardExpression)6 ScriptExpression (com.yahoo.vespa.indexinglanguage.expressions.ScriptExpression)6 StatementExpression (com.yahoo.vespa.indexinglanguage.expressions.StatementExpression)6 ArithmeticExpression (com.yahoo.vespa.indexinglanguage.expressions.ArithmeticExpression)5 AttributeExpression (com.yahoo.vespa.indexinglanguage.expressions.AttributeExpression)5 CatExpression (com.yahoo.vespa.indexinglanguage.expressions.CatExpression)5 ClearStateExpression (com.yahoo.vespa.indexinglanguage.expressions.ClearStateExpression)5 ForEachExpression (com.yahoo.vespa.indexinglanguage.expressions.ForEachExpression)5 IfThenExpression (com.yahoo.vespa.indexinglanguage.expressions.IfThenExpression)5 IndexExpression (com.yahoo.vespa.indexinglanguage.expressions.IndexExpression)5 InputExpression (com.yahoo.vespa.indexinglanguage.expressions.InputExpression)5 ParenthesisExpression (com.yahoo.vespa.indexinglanguage.expressions.ParenthesisExpression)5 SelectInputExpression (com.yahoo.vespa.indexinglanguage.expressions.SelectInputExpression)5 SwitchExpression (com.yahoo.vespa.indexinglanguage.expressions.SwitchExpression)5 Test (org.junit.Test)5 Base64DecodeExpression (com.yahoo.vespa.indexinglanguage.expressions.Base64DecodeExpression)4 Base64EncodeExpression (com.yahoo.vespa.indexinglanguage.expressions.Base64EncodeExpression)4 CompositeExpression (com.yahoo.vespa.indexinglanguage.expressions.CompositeExpression)4 EchoExpression (com.yahoo.vespa.indexinglanguage.expressions.EchoExpression)4