Examples with Lexer org.antlr.v4.runtime.Lexer used on opensource projects

Example 91 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerNotBMPSetMatchesSMP.

@Test
public void testLexerNotBMPSetMatchesSMP() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "ID : ~('a'|'b')\n ;");
    String expecting = "ID, EOF";
    checkLexerMatches(lg, new StringBuilder().appendCodePoint(0x1F4A9).toString(), expecting);
}

Example 92 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerRangeUnicodeBMPToSMP.

@Test
public void testLexerRangeUnicodeBMPToSMP() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "ID : ('\\u611B'..'\\u{1F4B0}')\n ;");
    String expecting = "ID, EOF";
    checkLexerMatches(lg, new StringBuilder().appendCodePoint(0x12001).toString(), expecting);
}

Example 93 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerWildcardExplicitNonGreedyPlusLoop.

@Test
public void testLexerWildcardExplicitNonGreedyPlusLoop() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "CMT : '//' .+? '\\n' ;\n");
    String expecting = "CMT, CMT, EOF";
    checkLexerMatches(lg, "//x\n//y\n", expecting);
}

Example 94 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerTwoRules.

@Test
public void testLexerTwoRules() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "A : 'a' ;\n" + "B : 'b' ;\n");
    String expecting = "A, B, A, B, EOF";
    checkLexerMatches(lg, "abab", expecting);
}

Example 95 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testShortLongRule.

@Test
public void testShortLongRule() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "A : 'xy'\n" + // this alt is preferred since there are no non-greedy configs
    "  | 'xyz'\n" + "  ;\n" + "Z : 'z'\n" + "  ;\n");
    checkLexerMatches(lg, "xy", "A, EOF");
    checkLexerMatches(lg, "xyz", "A, EOF");
}