Examples with CodePointCharStream org.antlr.v4.runtime.CodePointCharStream used on opensource projects

Example 36 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method singleCJKCodePointHasSize1.

@Test
public void singleCJKCodePointHasSize1() {
    CodePointCharStream s = CharStreams.fromString("愛");
    assertEquals(1, s.size());
    assertEquals(0, s.index());
}

Example 37 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method singleCJKCodePointLookAheadShouldReturnCodePoint.

@Test
public void singleCJKCodePointLookAheadShouldReturnCodePoint() {
    CodePointCharStream s = CharStreams.fromString("愛");
    assertEquals(0x611B, s.LA(1));
    assertEquals(0, s.index());
}

Example 38 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project antlr4 by antlr.

the class TestCodePointCharStream method seekWithLatin.

@Test
public void seekWithLatin() {
    CodePointCharStream s = CharStreams.fromString("0123456789");
    s.seek(5);
    assertEquals('5', s.LA(1));
}

Example 39 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project bmoth by hhu-stups.

the class Parser method getParser.

private BMoThParser getParser(String inputString) {
    CodePointCharStream fromString = CharStreams.fromString(inputString);
    final BMoThLexer lexer = new BMoThLexer(fromString);
    // create a buffer of tokens pulled from the lexer
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    BMoThParser bMoThParser = new BMoThParser(tokens);
    bMoThParser.removeErrorListeners();
    ErrorListener errorListener = new ErrorListener();
    bMoThParser.addErrorListener(errorListener);
    return bMoThParser;
}

Example 40 with CodePointCharStream

use of org.antlr.v4.runtime.CodePointCharStream in project drill by apache.

the class SchemaExprParser method initParser.

private static SchemaParser initParser(String value) {
    CodePointCharStream stream = CharStreams.fromString(value);
    UpperCaseCharStream upperCaseStream = new UpperCaseCharStream(stream);
    SchemaLexer lexer = new SchemaLexer(upperCaseStream);
    lexer.removeErrorListeners();
    lexer.addErrorListener(ErrorListener.INSTANCE);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    SchemaParser parser = new SchemaParser(tokens);
    parser.removeErrorListeners();
    parser.addErrorListener(ErrorListener.INSTANCE);
    return parser;
}