Examples with LexerGrammar org.antlr.v4.tool.LexerGrammar used on opensource projects

Example 31 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method checkLexerMatches.

protected void checkLexerMatches(LexerGrammar lg, String inputString, String expecting) {
    ATN atn = createATN(lg, true);
    CharStream input = CharStreams.fromString(inputString);
    ATNState startState = atn.modeNameToStartState.get("DEFAULT_MODE");
    DOTGenerator dot = new DOTGenerator(lg);
    System.out.println(dot.getDOT(startState, true));
    List<String> tokenTypes = getTokenTypes(lg, atn, input);
    String result = Utils.join(tokenTypes.iterator(), ", ");
    System.out.println(tokenTypes);
    assertEquals(expecting, result);
}

Example 32 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerRangeUnicodeBMPToSMP.

@Test
public void testLexerRangeUnicodeBMPToSMP() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "ID : ('\\u611B'..'\\u{1F4B0}')\n ;");
    String expecting = "ID, EOF";
    checkLexerMatches(lg, new StringBuilder().appendCodePoint(0x12001).toString(), expecting);
}

Example 33 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerWildcardExplicitNonGreedyPlusLoop.

@Test
public void testLexerWildcardExplicitNonGreedyPlusLoop() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "CMT : '//' .+? '\\n' ;\n");
    String expecting = "CMT, CMT, EOF";
    checkLexerMatches(lg, "//x\n//y\n", expecting);
}

Example 34 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerTwoRules.

@Test
public void testLexerTwoRules() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "A : 'a' ;\n" + "B : 'b' ;\n");
    String expecting = "A, B, A, B, EOF";
    checkLexerMatches(lg, "abab", expecting);
}

Example 35 with LexerGrammar

use of org.antlr.v4.tool.LexerGrammar in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testShortLongRule.

@Test
public void testShortLongRule() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "A : 'xy'\n" + // this alt is preferred since there are no non-greedy configs
    "  | 'xyz'\n" + "  ;\n" + "Z : 'z'\n" + "  ;\n");
    checkLexerMatches(lg, "xy", "A, EOF");
    checkLexerMatches(lg, "xyz", "A, EOF");
}