Examples with Lexer org.antlr.v4.runtime.Lexer used on opensource projects

Example 96 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testWildOnEndFirstAlt.

@Test
public void testWildOnEndFirstAlt() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + // should pursue '.' since xyz hits stop first, before 2nd alt
    "A : 'xy' .\n" + "  | 'xy'\n" + "  ;\n" + "Z : 'z'\n" + "  ;\n");
    checkLexerMatches(lg, "xy", "A, EOF");
    checkLexerMatches(lg, "xyz", "A, EOF");
}

Example 97 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testShortLongRule2.

@Test
public void testShortLongRule2() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + // make sure nongreedy mech cut off doesn't kill this alt
    "A : 'xyz'\n" + "  | 'xy'\n" + "  ;\n");
    checkLexerMatches(lg, "xy", "A, EOF");
    checkLexerMatches(lg, "xyz", "A, EOF");
}

Example 98 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerLoops.

@Test
public void testLexerLoops() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "INT : '0'..'9'+ ;\n" + "ID : 'a'..'z'+ ;\n");
    String expecting = "ID, INT, ID, INT, EOF";
    checkLexerMatches(lg, "a34bde3", expecting);
}

Example 99 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testLexerKeywordIDAmbiguity.

@Test
public void testLexerKeywordIDAmbiguity() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "KEND : 'end' ;\n" + "ID : 'a'..'z'+ ;\n" + "WS : (' '|'\\n')+ ;");
    String expecting = "ID, EOF";
    // checkLexerMatches(lg, "e", expecting);
    expecting = "KEND, EOF";
    checkLexerMatches(lg, "end", expecting);
    expecting = "ID, EOF";
    checkLexerMatches(lg, "ending", expecting);
    expecting = "ID, WS, KEND, WS, ID, EOF";
    checkLexerMatches(lg, "a end bcd", expecting);
}

Example 100 with Lexer

use of org.antlr.v4.runtime.Lexer in project antlr4 by tunnelvisionlabs.

the class TestATNLexerInterpreter method testEOFAtEndOfLineComment.

@Test
public void testEOFAtEndOfLineComment() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "CMT : '//' ~('\\n')* ;\n");
    String expecting = "CMT, EOF";
    checkLexerMatches(lg, "//x", expecting);
}