Search in sources :

Example 71 with ANTLRStringStream

use of org.antlr.runtime.ANTLRStringStream in project nifi by apache.

the class ExpressionCompiler method compile.

public CompiledExpression compile(final String expression) {
    try {
        final CharStream input = new ANTLRStringStream(expression);
        final AttributeExpressionLexer lexer = new AttributeExpressionLexer(input);
        final CommonTokenStream lexerTokenStream = new CommonTokenStream(lexer);
        final AttributeExpressionParser parser = new AttributeExpressionParser(lexerTokenStream);
        final Tree ast = (Tree) parser.query().getTree();
        final Tree tree = ast.getChild(0);
        final Evaluator<?> evaluator = buildEvaluator(tree);
        verifyMappingEvaluatorReduced(evaluator);
        final Set<Evaluator<?>> allEvaluators = new HashSet<>(evaluators);
        this.evaluators.clear();
        return new CompiledExpression(expression, evaluator, tree, allEvaluators);
    } catch (final AttributeExpressionLanguageParsingException e) {
        throw e;
    } catch (final Exception e) {
        throw new AttributeExpressionLanguageParsingException(e);
    }
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) CommonTokenStream(org.antlr.runtime.CommonTokenStream) AttributeExpressionParser(org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionParser) IfElseEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.IfElseEvaluator) OrEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.OrEvaluator) GreaterThanEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.GreaterThanEvaluator) LastIndexOfEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.LastIndexOfEvaluator) UrlEncodeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.UrlEncodeEvaluator) IteratingEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.IteratingEvaluator) SubstringAfterLastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.SubstringAfterLastEvaluator) ReplaceNullEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ReplaceNullEvaluator) ToLiteralEvaluator(org.apache.nifi.attribute.expression.language.evaluation.literals.ToLiteralEvaluator) AppendEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.AppendEvaluator) GetStateVariableEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.GetStateVariableEvaluator) MinusEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.MinusEvaluator) ReplaceAllEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ReplaceAllEvaluator) Base64DecodeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.Base64DecodeEvaluator) ToStringEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ToStringEvaluator) MultiAttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.MultiAttributeEvaluator) AttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.AttributeEvaluator) GreaterThanOrEqualEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.GreaterThanOrEqualEvaluator) AnyAttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.AnyAttributeEvaluator) MappingEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.MappingEvaluator) AndEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.AndEvaluator) NumberToDateEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.NumberToDateEvaluator) OneUpSequenceEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.OneUpSequenceEvaluator) StringLiteralEvaluator(org.apache.nifi.attribute.expression.language.evaluation.literals.StringLiteralEvaluator) WholeNumberCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.WholeNumberCastEvaluator) IsEmptyEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.IsEmptyEvaluator) InEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.InEvaluator) LessThanOrEqualEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.LessThanOrEqualEvaluator) UuidEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.UuidEvaluator) IPEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.IPEvaluator) EqualsEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.EqualsEvaluator) CharSequenceTranslatorEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.CharSequenceTranslatorEvaluator) IsNullEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.IsNullEvaluator) NowEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.NowEvaluator) IndexOfEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.IndexOfEvaluator) CountEvaluator(org.apache.nifi.attribute.expression.language.evaluation.reduce.CountEvaluator) ToRadixEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ToRadixEvaluator) ReplaceFirstEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ReplaceFirstEvaluator) ToUpperEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ToUpperEvaluator) ReplaceEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ReplaceEvaluator) EqualsIgnoreCaseEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.EqualsIgnoreCaseEvaluator) ContainsEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ContainsEvaluator) PlusEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.PlusEvaluator) FindEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.FindEvaluator) MatchesEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.MatchesEvaluator) Evaluator(org.apache.nifi.attribute.expression.language.evaluation.Evaluator) ModEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ModEvaluator) Base64EncodeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.Base64EncodeEvaluator) GetDelimitedFieldEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.GetDelimitedFieldEvaluator) DateEvaluator(org.apache.nifi.attribute.expression.language.evaluation.DateEvaluator) TrimEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.TrimEvaluator) FromRadixEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.FromRadixEvaluator) JsonPathEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.JsonPathEvaluator) BooleanLiteralEvaluator(org.apache.nifi.attribute.expression.language.evaluation.literals.BooleanLiteralEvaluator) DecimalLiteralEvaluator(org.apache.nifi.attribute.expression.language.evaluation.literals.DecimalLiteralEvaluator) LessThanEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.LessThanEvaluator) WholeNumberLiteralEvaluator(org.apache.nifi.attribute.expression.language.evaluation.literals.WholeNumberLiteralEvaluator) StringEvaluator(org.apache.nifi.attribute.expression.language.evaluation.StringEvaluator) LengthEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.LengthEvaluator) ReduceEvaluator(org.apache.nifi.attribute.expression.language.evaluation.reduce.ReduceEvaluator) SubstringAfterEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.SubstringAfterEvaluator) MathEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.MathEvaluator) MultiplyEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.MultiplyEvaluator) AllAttributesEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.AllAttributesEvaluator) SubstringEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.SubstringEvaluator) MultiNamedAttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.MultiNamedAttributeEvaluator) NotEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.NotEvaluator) SubstringBeforeLastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.SubstringBeforeLastEvaluator) FormatEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.FormatEvaluator) StartsWithEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.StartsWithEvaluator) PrependEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.PrependEvaluator) EndsWithEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.EndsWithEvaluator) ToLowerEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ToLowerEvaluator) MultiMatchAttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.MultiMatchAttributeEvaluator) BooleanEvaluator(org.apache.nifi.attribute.expression.language.evaluation.BooleanEvaluator) ReplaceEmptyEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.ReplaceEmptyEvaluator) DelineatedAttributeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.selection.DelineatedAttributeEvaluator) NumberCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.NumberCastEvaluator) HostnameEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.HostnameEvaluator) StringCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.StringCastEvaluator) BooleanCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.BooleanCastEvaluator) UrlDecodeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.UrlDecodeEvaluator) RandomNumberGeneratorEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.RandomNumberGeneratorEvaluator) NotNullEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.NotNullEvaluator) DecimalCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.DecimalCastEvaluator) JoinEvaluator(org.apache.nifi.attribute.expression.language.evaluation.reduce.JoinEvaluator) SubstringBeforeEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.SubstringBeforeEvaluator) StringToDateEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.StringToDateEvaluator) DateCastEvaluator(org.apache.nifi.attribute.expression.language.evaluation.cast.DateCastEvaluator) DivideEvaluator(org.apache.nifi.attribute.expression.language.evaluation.functions.DivideEvaluator) AttributeExpressionLanguageParsingException(org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException) CharStream(org.antlr.runtime.CharStream) CompiledExpression(org.apache.nifi.attribute.expression.language.CompiledExpression) AttributeExpressionLanguageException(org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageException) UnknownHostException(java.net.UnknownHostException) AttributeExpressionLanguageParsingException(org.apache.nifi.attribute.expression.language.exception.AttributeExpressionLanguageParsingException) Tree(org.antlr.runtime.tree.Tree) AttributeExpressionLexer(org.apache.nifi.attribute.expression.language.antlr.AttributeExpressionLexer) HashSet(java.util.HashSet)

Example 72 with ANTLRStringStream

use of org.antlr.runtime.ANTLRStringStream in project xtext-xtend by eclipse.

the class LexerBenchmark method timeLexing.

public int timeLexing(int reps) throws Exception {
    int result = reps;
    for (int i = 0; i < reps; i++) {
        Token tok = null;
        lexer.setCharStream(new ANTLRStringStream(contentToParse));
        while (tok != Token.EOF_TOKEN) {
            tok = lexer.nextToken();
            result++;
        }
        result += reps;
    }
    return result;
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) Token(org.antlr.runtime.Token)

Example 73 with ANTLRStringStream

use of org.antlr.runtime.ANTLRStringStream in project drools by kiegroup.

the class JavaExprAnalyzer method parse.

private JavaParser parse(final String expr) {
    final CharStream charStream = new ANTLRStringStream(expr);
    final JavaLexer lexer = new JavaLexer(charStream);
    final TokenStream tokenStream = new CommonTokenStream(lexer);
    return new JavaParser(tokenStream);
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) CommonTokenStream(org.antlr.runtime.CommonTokenStream) JavaParser(org.drools.compiler.rule.builder.dialect.java.parser.JavaParser) CommonTokenStream(org.antlr.runtime.CommonTokenStream) TokenStream(org.antlr.runtime.TokenStream) JavaLexer(org.drools.compiler.rule.builder.dialect.java.parser.JavaLexer) CharStream(org.antlr.runtime.CharStream)

Example 74 with ANTLRStringStream

use of org.antlr.runtime.ANTLRStringStream in project drools by kiegroup.

the class RuleParserTest method testChunkWithParensAndQuotedString.

@Test
public void testChunkWithParensAndQuotedString() throws Exception {
    String input = "( fnord( \"cheese\" ) )";
    createParser(new ANTLRStringStream(input));
    String returnData = parser.chunk(DRL6Lexer.LEFT_PAREN, DRL6Lexer.RIGHT_PAREN, -1);
    assertEquals("fnord( \"cheese\" )", returnData);
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) Test(org.junit.Test)

Example 75 with ANTLRStringStream

use of org.antlr.runtime.ANTLRStringStream in project drools by kiegroup.

the class RuleParserTest method testChunkWithoutParens.

@Test
public void testChunkWithoutParens() throws Exception {
    String input = "( foo )";
    createParser(new ANTLRStringStream(input));
    String returnData = parser.chunk(DRL6Lexer.LEFT_PAREN, DRL6Lexer.RIGHT_PAREN, -1);
    assertEquals("foo", returnData);
}
Also used : ANTLRStringStream(org.antlr.runtime.ANTLRStringStream) Test(org.junit.Test)

Aggregations

ANTLRStringStream (org.antlr.runtime.ANTLRStringStream)117 CommonTokenStream (org.antlr.runtime.CommonTokenStream)63 Token (org.antlr.runtime.Token)30 Test (org.junit.Test)28 CharStream (org.antlr.runtime.CharStream)24 CommonToken (org.antlr.runtime.CommonToken)20 RecognitionException (org.antlr.runtime.RecognitionException)18 Lexer (org.eclipse.xtext.parser.antlr.Lexer)16 ActionSplitter (org.antlr.v4.parse.ActionSplitter)12 TokenStream (org.antlr.runtime.TokenStream)10 CommonTree (org.antlr.runtime.tree.CommonTree)10 ExprLexer (org.apache.drill.common.expression.parser.ExprLexer)10 ExprParser (org.apache.drill.common.expression.parser.ExprParser)10 InternalSimpleExpressionsTestLanguageLexer (org.eclipse.xtext.testlanguages.parser.antlr.internal.InternalSimpleExpressionsTestLanguageLexer)8 File (java.io.File)4 ArrayList (java.util.ArrayList)4 CommonTreeNodeStream (org.antlr.runtime.tree.CommonTreeNodeStream)4 Tree (org.antlr.runtime.tree.Tree)4 ExprParser.parse_return (org.apache.drill.common.expression.parser.ExprParser.parse_return)4 ApexLexer (apex.jorje.parser.impl.ApexLexer)3