Search in sources :

Example 1 with ANTLRInputStream

use of org.antlr.v4.runtime.ANTLRInputStream in project elasticsearch by elastic.

the class Walker method buildAntlrTree.

private SourceContext buildAntlrTree(String source) {
    ANTLRInputStream stream = new ANTLRInputStream(source);
    PainlessLexer lexer = new EnhancedPainlessLexer(stream, sourceName);
    PainlessParser parser = new PainlessParser(new CommonTokenStream(lexer));
    ParserErrorStrategy strategy = new ParserErrorStrategy(sourceName);
    lexer.removeErrorListeners();
    parser.removeErrorListeners();
    if (settings.isPicky()) {
        setupPicky(parser);
    }
    parser.setErrorHandler(strategy);
    return parser.source();
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream)

Example 2 with ANTLRInputStream

use of org.antlr.v4.runtime.ANTLRInputStream in project pinot by linkedin.

the class Pql2Compiler method compileToBrokerRequest.

@Override
public BrokerRequest compileToBrokerRequest(String expression) throws Pql2CompilationException {
    try {
        //
        CharStream charStream = new ANTLRInputStream(expression);
        PQL2Lexer lexer = new PQL2Lexer(charStream);
        lexer.setTokenFactory(new CommonTokenFactory(true));
        TokenStream tokenStream = new UnbufferedTokenStream<CommonToken>(lexer);
        PQL2Parser parser = new PQL2Parser(tokenStream);
        parser.setErrorHandler(new BailErrorStrategy());
        // Parse
        ParseTree parseTree = parser.root();
        ParseTreeWalker walker = new ParseTreeWalker();
        Pql2AstListener listener = new Pql2AstListener(expression);
        walker.walk(listener, parseTree);
        AstNode rootNode = listener.getRootNode();
        BrokerRequest brokerRequest = new BrokerRequest();
        rootNode.updateBrokerRequest(brokerRequest);
        return brokerRequest;
    } catch (Pql2CompilationException e) {
        throw e;
    } catch (Exception e) {
        throw new Pql2CompilationException(e.getMessage());
    }
}
Also used : TokenStream(org.antlr.v4.runtime.TokenStream) UnbufferedTokenStream(org.antlr.v4.runtime.UnbufferedTokenStream) CommonTokenFactory(org.antlr.v4.runtime.CommonTokenFactory) BailErrorStrategy(org.antlr.v4.runtime.BailErrorStrategy) UnbufferedTokenStream(org.antlr.v4.runtime.UnbufferedTokenStream) CharStream(org.antlr.v4.runtime.CharStream) BrokerRequest(com.linkedin.pinot.common.request.BrokerRequest) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) ParseTree(org.antlr.v4.runtime.tree.ParseTree) ParseTreeWalker(org.antlr.v4.runtime.tree.ParseTreeWalker) AstNode(com.linkedin.pinot.pql.parsers.pql2.ast.AstNode)

Example 3 with ANTLRInputStream

use of org.antlr.v4.runtime.ANTLRInputStream in project neo4j by neo4j.

the class ResultsParser method matcherParse.

ValueMatcher matcherParse(String value, CypherMatchersCreator listener) {
    lexer.setInputStream(new ANTLRInputStream(value));
    walker.walk(listener, parser.value());
    return listener.parsed();
}
Also used : ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream)

Example 4 with ANTLRInputStream

use of org.antlr.v4.runtime.ANTLRInputStream in project neo4j by neo4j.

the class ResultsParser method parseParameter.

Object parseParameter(String value, CypherParametersCreator listener) {
    lexer.setInputStream(new ANTLRInputStream(value));
    walker.walk(listener, parser.value());
    return listener.parsed();
}
Also used : ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream)

Example 5 with ANTLRInputStream

use of org.antlr.v4.runtime.ANTLRInputStream in project aic-praise by aic-sri-international.

the class HOGMCodeArea method computeHighlighting.

private static StyleSpans<Collection<String>> computeHighlighting(String text) {
    StyleSpansBuilder<Collection<String>> spansBuilder = new StyleSpansBuilder<>();
    int lastTokenEnd = 0;
    ANTLRInputStream input = new ANTLRInputStream(text);
    HOGMLexer lexer = new HOGMLexer(input);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    tokens.fill();
    for (int i = 0; i < tokens.size(); i++) {
        Token t = tokens.get(i);
        if (t.getType() == Token.EOF) {
            break;
        }
        String styleClass;
        if (t.getType() == HOGMLexer.COMMENT || t.getType() == HOGMLexer.LINE_COMMENT) {
            styleClass = "hogmCodeComment";
        } else if (HOGMTerminalSymbols.isTerminalSymbol(t.getText())) {
            styleClass = "hogmCodeKeyword";
        } else {
            styleClass = "hogmCodeOther";
        }
        int spacing = t.getStartIndex() - lastTokenEnd;
        if (spacing > 0) {
            spansBuilder.add(Collections.emptyList(), spacing);
        }
        int stylesize = (t.getStopIndex() - t.getStartIndex()) + 1;
        spansBuilder.add(Collections.singleton(styleClass), stylesize);
        lastTokenEnd = t.getStopIndex() + 1;
    }
    return spansBuilder.create();
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) HOGMLexer(com.sri.ai.praise.model.v1.hogm.antlr.HOGMLexer) Collection(java.util.Collection) StyleSpansBuilder(org.fxmisc.richtext.StyleSpansBuilder) Token(org.antlr.v4.runtime.Token) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream)

Aggregations

ANTLRInputStream (org.antlr.v4.runtime.ANTLRInputStream)174 CommonTokenStream (org.antlr.v4.runtime.CommonTokenStream)144 Test (org.junit.Test)60 LexerInterpreter (org.antlr.v4.runtime.LexerInterpreter)59 LexerGrammar (org.antlr.v4.tool.LexerGrammar)52 TokenStreamRewriter (org.antlr.v4.runtime.TokenStreamRewriter)43 BaseJavaTest (org.antlr.v4.test.runtime.java.BaseJavaTest)43 ParseTree (org.antlr.v4.runtime.tree.ParseTree)42 IOException (java.io.IOException)21 CharStream (org.antlr.v4.runtime.CharStream)20 InputStream (java.io.InputStream)16 ByteArrayInputStream (java.io.ByteArrayInputStream)14 Token (org.antlr.v4.runtime.Token)14 ParseTreeWalker (org.antlr.v4.runtime.tree.ParseTreeWalker)13 TokenStream (org.antlr.v4.runtime.TokenStream)11 ArrayList (java.util.ArrayList)9 BailErrorStrategy (org.antlr.v4.runtime.BailErrorStrategy)9 ParseCancellationException (org.antlr.v4.runtime.misc.ParseCancellationException)9 SiddhiQLBaseVisitorImpl (org.wso2.siddhi.query.compiler.internal.SiddhiQLBaseVisitorImpl)9 StringReader (java.io.StringReader)7