Search in sources :

Example 86 with Grammar

use of org.antlr.v4.tool.Grammar in project antlr4 by antlr.

the class TestATNSerialization method testLexerUnicodeEscapedSMPNotSet.

@Test
public void testLexerUnicodeEscapedSMPNotSet() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "ID : ~('\\u{1F4A9}'|'\\u{1F4AA}')\n ;");
    String expecting = "max type 1\n" + "0:TOKEN_START -1\n" + "1:RULE_START 0\n" + "2:RULE_STOP 0\n" + "3:BASIC 0\n" + "4:BASIC 0\n" + "rule 0:1 1\n" + "mode 0:0\n" + "0:128169..128170\n" + "0->1 EPSILON 0,0,0\n" + "1->3 EPSILON 0,0,0\n" + "3->4 NOT_SET 0,0,0\n" + "4->2 EPSILON 0,0,0\n" + "0:0\n";
    ATN atn = createATN(lg, true);
    String result = ATNSerializer.getDecoded(atn, Arrays.asList(lg.getTokenNames()));
    assertEquals(expecting, result);
}
Also used : ATN(org.antlr.v4.runtime.atn.ATN) LexerGrammar(org.antlr.v4.tool.LexerGrammar) Test(org.junit.Test)

Example 87 with Grammar

use of org.antlr.v4.tool.Grammar in project antlr4 by antlr.

the class TestATNSerialization method testLexerUnicodeSMPLiteralSerializedToSet.

@Test
public void testLexerUnicodeSMPLiteralSerializedToSet() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "INT : '\\u{1F4A9}' ;");
    String expecting = "max type 1\n" + "0:TOKEN_START -1\n" + "1:RULE_START 0\n" + "2:RULE_STOP 0\n" + "3:BASIC 0\n" + "4:BASIC 0\n" + "rule 0:1 1\n" + "mode 0:0\n" + "0:128169..128169\n" + "0->1 EPSILON 0,0,0\n" + "1->3 EPSILON 0,0,0\n" + "3->4 SET 0,0,0\n" + "4->2 EPSILON 0,0,0\n" + "0:0\n";
    ATN atn = createATN(lg, true);
    String result = ATNSerializer.getDecoded(atn, Arrays.asList(lg.getTokenNames()));
    assertEquals(expecting, result);
}
Also used : ATN(org.antlr.v4.runtime.atn.ATN) LexerGrammar(org.antlr.v4.tool.LexerGrammar) Test(org.junit.Test)

Example 88 with Grammar

use of org.antlr.v4.tool.Grammar in project antlr4 by antlr.

the class TestATNSerialization method testEOFInSet.

@Test
public void testEOFInSet() throws Exception {
    Grammar g = new Grammar("parser grammar T;\n" + "a : (A|EOF) ;");
    String expecting = "max type 1\n" + "0:RULE_START 0\n" + "1:RULE_STOP 0\n" + "2:BASIC 0\n" + "3:BASIC 0\n" + "4:BASIC 0\n" + "rule 0:0\n" + "0:EOF, A..A\n" + "0->2 EPSILON 0,0,0\n" + "2->3 SET 0,0,0\n" + "3->1 EPSILON 0,0,0\n";
    ATN atn = createATN(g, true);
    String result = ATNSerializer.getDecoded(atn, Arrays.asList(g.getTokenNames()));
    assertEquals(expecting, result);
}
Also used : Grammar(org.antlr.v4.tool.Grammar) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ATN(org.antlr.v4.runtime.atn.ATN) Test(org.junit.Test)

Example 89 with Grammar

use of org.antlr.v4.tool.Grammar in project antlr4 by antlr.

the class TestATNSerialization method testLexerRange.

@Test
public void testLexerRange() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "INT : '0'..'9' ;\n");
    String expecting = "max type 1\n" + "0:TOKEN_START -1\n" + "1:RULE_START 0\n" + "2:RULE_STOP 0\n" + "3:BASIC 0\n" + "4:BASIC 0\n" + "rule 0:1 1\n" + "mode 0:0\n" + "0->1 EPSILON 0,0,0\n" + "1->3 EPSILON 0,0,0\n" + "3->4 RANGE 48,57,0\n" + "4->2 EPSILON 0,0,0\n" + "0:0\n";
    ATN atn = createATN(lg, true);
    String result = ATNSerializer.getDecoded(atn, Arrays.asList(lg.getTokenNames()));
    assertEquals(expecting, result);
}
Also used : ATN(org.antlr.v4.runtime.atn.ATN) LexerGrammar(org.antlr.v4.tool.LexerGrammar) Test(org.junit.Test)

Example 90 with Grammar

use of org.antlr.v4.tool.Grammar in project antlr4 by antlr.

the class TestATNSerialization method testLexerUnicodeEscapedBMPNotSetWithRange.

@Test
public void testLexerUnicodeEscapedBMPNotSetWithRange() throws Exception {
    LexerGrammar lg = new LexerGrammar("lexer grammar L;\n" + "ID : ~('\\u4E9C'|'\\u4E9D'|'\\u6C5F'|'\\u305F'..'\\u307B')\n ;");
    String expecting = "max type 1\n" + "0:TOKEN_START -1\n" + "1:RULE_START 0\n" + "2:RULE_STOP 0\n" + "3:BASIC 0\n" + "4:BASIC 0\n" + "rule 0:1 1\n" + "mode 0:0\n" + "0:'\\u305F'..'\\u307B', '\\u4E9C'..'\\u4E9D', '\\u6C5F'..'\\u6C5F'\n" + "0->1 EPSILON 0,0,0\n" + "1->3 EPSILON 0,0,0\n" + "3->4 NOT_SET 0,0,0\n" + "4->2 EPSILON 0,0,0\n" + "0:0\n";
    ATN atn = createATN(lg, true);
    String result = ATNSerializer.getDecoded(atn, Arrays.asList(lg.getTokenNames()));
    assertEquals(expecting, result);
}
Also used : ATN(org.antlr.v4.runtime.atn.ATN) LexerGrammar(org.antlr.v4.tool.LexerGrammar) Test(org.junit.Test)

Aggregations

LexerGrammar (org.antlr.v4.tool.LexerGrammar)305 Test (org.junit.Test)304 Grammar (org.antlr.v4.tool.Grammar)183 ATN (org.antlr.v4.runtime.atn.ATN)65 LexerInterpreter (org.antlr.v4.runtime.LexerInterpreter)62 ANTLRInputStream (org.antlr.v4.runtime.ANTLRInputStream)59 CommonTokenStream (org.antlr.v4.runtime.CommonTokenStream)53 BaseJavaTest (org.antlr.v4.test.runtime.java.BaseJavaTest)47 TokenStreamRewriter (org.antlr.v4.runtime.TokenStreamRewriter)43 GrammarAST (org.antlr.v4.tool.ast.GrammarAST)31 ErrorQueue (org.antlr.v4.test.runtime.ErrorQueue)29 Rule (org.antlr.v4.tool.Rule)25 ParserATNFactory (org.antlr.v4.automata.ParserATNFactory)21 STGroupString (org.stringtemplate.v4.STGroupString)21 ArrayList (java.util.ArrayList)18 ATNState (org.antlr.v4.runtime.atn.ATNState)18 BaseRuntimeTest.antlrOnString (org.antlr.v4.test.runtime.BaseRuntimeTest.antlrOnString)17 ST (org.stringtemplate.v4.ST)17 ParseTree (org.antlr.v4.runtime.tree.ParseTree)15 BaseRuntimeTest (org.antlr.v4.test.runtime.BaseRuntimeTest)15