Examples with CharStream org.antlr.v4.runtime.CharStream used on opensource projects

Example 36 with CharStream

use of org.antlr.v4.runtime.CharStream in project antlr4 by tunnelvisionlabs.

the class TestCharStreams method fromSMPUTF32LEPathSMPHasExpectedSize.

@Test
public void fromSMPUTF32LEPathSMPHasExpectedSize() throws Exception {
    File p = folder.newFile();
    Utils.writeFile(p, "hello \uD83C\uDF0E".getBytes(Charset.forName("UTF-32LE")));
    CharStream s = CharStreams.fromFile(p, Charset.forName("UTF-32LE"));
    assertEquals(7, s.size());
    assertEquals(0, s.index());
    assertEquals("hello \uD83C\uDF0E", s.toString());
    assertEquals(p.toString(), s.getSourceName());
}

Example 37 with CharStream

use of org.antlr.v4.runtime.CharStream in project antlr4 by tunnelvisionlabs.

the class TestCharStreams method fromSMPUTF8SequenceStraddlingBufferBoundary.

@Test
public void fromSMPUTF8SequenceStraddlingBufferBoundary() throws Exception {
    File p = folder.newFile();
    Utils.writeFile(p, "hello \uD83C\uDF0E".getBytes(Charset.forName("UTF-8")));
    ReadableByteChannel c = Channels.newChannel(new FileInputStream(p));
    try {
        CharStream s = CharStreams.fromChannel(c, // straddles the boundary of two buffers
        8, CodingErrorAction.REPLACE, "foo");
        assertEquals(7, s.size());
        assertEquals(0, s.index());
        assertEquals("hello \uD83C\uDF0E", s.toString());
    } finally {
        c.close();
    }
}

Example 38 with CharStream

use of org.antlr.v4.runtime.CharStream in project antlr4 by tunnelvisionlabs.

the class TestCharStreams method fromBMPUTF8InputStreamHasExpectedSize.

@Test
public void fromBMPUTF8InputStreamHasExpectedSize() throws Exception {
    File p = folder.newFile();
    Utils.writeFile(p, "hello".getBytes(Charset.forName("UTF-8")));
    InputStream is = new FileInputStream(p);
    try {
        CharStream s = CharStreams.fromStream(is);
        assertEquals(5, s.size());
        assertEquals(0, s.index());
        assertEquals("hello", s.toString());
    } finally {
        is.close();
    }
}

Example 39 with CharStream

use of org.antlr.v4.runtime.CharStream in project antlr4 by tunnelvisionlabs.

the class TestCharStreams method fromBMPUTF8PathHasExpectedSize.

@Test
public void fromBMPUTF8PathHasExpectedSize() throws Exception {
    File p = folder.newFile();
    Utils.writeFile(p, "hello".getBytes(Charset.forName("UTF-8")));
    CharStream s = CharStreams.fromFile(p);
    assertEquals(5, s.size());
    assertEquals(0, s.index());
    assertEquals("hello", s.toString());
    assertEquals(p.toString(), s.getSourceName());
}

Example 40 with CharStream

use of org.antlr.v4.runtime.CharStream in project antlr4 by tunnelvisionlabs.

the class TestCharStreams method fromBMPStringHasExpectedSize.

@Test
public void fromBMPStringHasExpectedSize() {
    CharStream s = CharStreams.fromString("hello");
    assertEquals(5, s.size());
    assertEquals(0, s.index());
    assertEquals("hello", s.toString());
}