Examples with LexerGrammar org.antlr.v4.tool.LexerGrammar used on opensource projects

Example 86 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class BaseTest method checkLexerDFA.

List<ANTLRMessage> checkLexerDFA(String gtext, String modeName, String expecting) throws Exception {
    ErrorQueue equeue = new ErrorQueue();
    LexerGrammar g = new LexerGrammar(gtext, equeue);
    g.atn = createATN(g, false);
    // return equeue.all;
    return null;
}

Example 87 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class BaseTest method getTokenTypes.

public List<String> getTokenTypes(LexerGrammar lg, ATN atn, CharStream input) {
    LexerATNSimulator interp = new LexerATNSimulator(atn);
    List<String> tokenTypes = new ArrayList<String>();
    int ttype;
    boolean hitEOF = false;
    do {
        if (hitEOF) {
            tokenTypes.add("EOF");
            break;
        }
        int t = input.LA(1);
        ttype = interp.match(input, Lexer.DEFAULT_MODE);
        if (ttype == Token.EOF) {
            tokenTypes.add("EOF");
        } else {
            tokenTypes.add(lg.typeToTokenList.get(ttype));
        }
        if (t == IntStream.EOF) {
            hitEOF = true;
        }
    } while (ttype != Token.EOF);
    return tokenTypes;
}

Example 88 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestBufferedTokenStream method testFirstToken.

@Test
public void testFirstToken() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar t;\n" + "ID : 'a'..'z'+;\n" + "INT : '0'..'9'+;\n" + "SEMI : ';';\n" + "ASSIGN : '=';\n" + "PLUS : '+';\n" + "MULT : '*';\n" + "WS : ' '+;\n");
    // Tokens: 012345678901234567
    // Input:  x = 3 * 0 + 2 * 0;
    CharStream input = CharStreams.fromString("x = 3 * 0 + 2 * 0;");
    LexerInterpreter lexEngine = g.createLexerInterpreter(input);
    TokenStream tokens = createTokenStream(lexEngine);
    String result = tokens.LT(1).getText();
    String expecting = "x";
    assertEquals(expecting, result);
}

Example 89 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestGrammarParserInterpreter method testAltsAsSet.

@Test
public void testAltsAsSet() throws Exception {
    LexerGrammar lg = new LexerGrammar(lexerText);
    Grammar g = new Grammar("parser grammar T;\n" + "s : ID\n" + "  | INT\n" + "  ;\n", lg);
    testInterp(lg, g, "s", "a", "(s:1 a)");
    testInterp(lg, g, "s", "3", "(s:1 3)");
}

Example 90 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestGrammarParserInterpreter method testAltsWithLabels.

@Test
public void testAltsWithLabels() throws Exception {
    LexerGrammar lg = new LexerGrammar(lexerText);
    Grammar g = new Grammar("parser grammar T;\n" + "s : ID  # foo\n" + "  | INT # bar\n" + "  ;\n", lg);
    // it won't show the labels here because my simple node text provider above just shows the alternative
    testInterp(lg, g, "s", "a", "(s:1 a)");
    testInterp(lg, g, "s", "3", "(s:2 3)");
}