Search in sources :

Example 6 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method seekWithCJK.

@Test
public void seekWithCJK() {
    CodePointCharStream s = CharStreams.fromString("01234䂔6789");
    s.seek(5);
    assertEquals(0x4094, s.LA(1));
}
Also used : CodePointCharStream(org.antlr.v4.runtime.CodePointCharStream) Test(org.junit.Test)

Example 7 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method lookBehindWithEmoji.

@Test
public void lookBehindWithEmoji() {
    CodePointCharStream s = CharStreams.fromString(new StringBuilder("01234").appendCodePoint(0x1F522).append("6789").toString());
    s.seek(6);
    assertEquals(0x1F522, s.LA(-1));
}
Also used : CodePointCharStream(org.antlr.v4.runtime.CodePointCharStream) Test(org.junit.Test)

Example 8 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method consumingPastSingleCJKCodePointShouldThrow.

@Test
public void consumingPastSingleCJKCodePointShouldThrow() {
    CodePointCharStream s = CharStreams.fromString("愛");
    s.consume();
    thrown.expect(IllegalStateException.class);
    thrown.expectMessage("cannot consume EOF");
    s.consume();
}
Also used : CodePointCharStream(org.antlr.v4.runtime.CodePointCharStream) Test(org.junit.Test)

Example 9 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method lookAheadWithEmoji.

@Test
public void lookAheadWithEmoji() {
    CodePointCharStream s = CharStreams.fromString(new StringBuilder("01234").appendCodePoint(0x1F522).append("6789").toString());
    assertEquals(0x1F522, s.LA(6));
}
Also used : CodePointCharStream(org.antlr.v4.runtime.CodePointCharStream) Test(org.junit.Test)

Example 10 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method multipleLatinCodePointsLookAheadShouldReturnCodePoints.

@Test
public void multipleLatinCodePointsLookAheadShouldReturnCodePoints() {
    CodePointCharStream s = CharStreams.fromString("XYZ");
    assertEquals('X', s.LA(1));
    assertEquals(0, s.index());
    assertEquals('Y', s.LA(2));
    assertEquals(0, s.index());
    assertEquals('Z', s.LA(3));
    assertEquals(0, s.index());
}
Also used : CodePointCharStream(org.antlr.v4.runtime.CodePointCharStream) Test(org.junit.Test)

Aggregations

CodePointCharStream (org.antlr.v4.runtime.CodePointCharStream)37 Test (org.junit.Test)34 CommonTokenStream (org.antlr.v4.runtime.CommonTokenStream)5 JavaLanguageLexer (com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageLexer)2 JavaLanguageParser (com.puppycrawl.tools.checkstyle.grammar.java.JavaLanguageParser)2 BMoThLexer (de.bmoth.antlr.BMoThLexer)2 BMoThParser (de.bmoth.antlr.BMoThParser)2 CharStream (org.antlr.v4.runtime.CharStream)2 CheckstyleException (com.puppycrawl.tools.checkstyle.api.CheckstyleException)1 DetailAST (com.puppycrawl.tools.checkstyle.api.DetailAST)1 FileContents (com.puppycrawl.tools.checkstyle.api.FileContents)1 FileText (com.puppycrawl.tools.checkstyle.api.FileText)1 VisitorException (de.bmoth.parser.ErrorListener.VisitorException)1 java.io (java.io)1 File (java.io.File)1 InvocationTargetException (java.lang.reflect.InvocationTargetException)1 Method (java.lang.reflect.Method)1 Paths (java.nio.file.Paths)1 java.util (java.util)1 Logger (java.util.logging.Logger)1