Examples with LexerInterpreter org.antlr.v4.runtime.LexerInterpreter used on opensource projects

Example 61 with LexerInterpreter

use of org.antlr.v4.runtime.LexerInterpreter in project antlr4 by antlr.

the class TestUnbufferedTokenStream method testLookahead.

@Test
public void testLookahead() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar t;\n" + "ID : 'a'..'z'+;\n" + "INT : '0'..'9'+;\n" + "SEMI : ';';\n" + "ASSIGN : '=';\n" + "PLUS : '+';\n" + "MULT : '*';\n" + "WS : ' '+;\n");
    // Tokens: 012345678901234567
    // Input:  x = 302;
    CharStream input = new ANTLRInputStream(new StringReader("x = 302;"));
    LexerInterpreter lexEngine = g.createLexerInterpreter(input);
    TokenStream tokens = new UnbufferedTokenStream<Token>(lexEngine);
    assertEquals("x", tokens.LT(1).getText());
    assertEquals(" ", tokens.LT(2).getText());
    assertEquals("=", tokens.LT(3).getText());
    assertEquals(" ", tokens.LT(4).getText());
    assertEquals("302", tokens.LT(5).getText());
    assertEquals(";", tokens.LT(6).getText());
}

Example 62 with LexerInterpreter

use of org.antlr.v4.runtime.LexerInterpreter in project antlr4 by antlr.

the class TestUnicodeGrammar method parseTreeForGrammarWithInput.

private static String parseTreeForGrammarWithInput(String grammarText, String rootRule, String inputText) throws Exception {
    Grammar grammar = new Grammar(grammarText);
    LexerInterpreter lexEngine = grammar.createLexerInterpreter(CharStreams.fromString(inputText));
    CommonTokenStream tokens = new CommonTokenStream(lexEngine);
    GrammarParserInterpreter parser = grammar.createGrammarParserInterpreter(tokens);
    ParseTree parseTree = parser.parse(grammar.rules.get(rootRule).index);
    InterpreterTreeTextProvider nodeTextProvider = new InterpreterTreeTextProvider(grammar.getRuleNames());
    return Trees.toStringTree(parseTree, nodeTextProvider);
}

Example 63 with LexerInterpreter

use of org.antlr.v4.runtime.LexerInterpreter in project antlr4 by antlr.

the class TestParserInterpreter method testInterp.

ParseTree testInterp(LexerGrammar lg, Grammar g, String startRule, String input, String expectedParseTree) {
    LexerInterpreter lexEngine = lg.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream tokens = new CommonTokenStream(lexEngine);
    ParserInterpreter parser = g.createParserInterpreter(tokens);
    ParseTree t = parser.parse(g.rules.get(startRule).index);
    assertEquals(expectedParseTree, t.toStringTree(parser));
    return t;
}