Examples with PeekReader org.apache.jena.atlas.io.PeekReader used on opensource projects

Example 6 with PeekReader

use of org.apache.jena.atlas.io.PeekReader in project jena by apache.

the class AbstractTestPeekReader method unread5.

@Test
public void unread5() {
    PeekReader r = make("");
    long lineNum = r.getLineNum();
    long colNum = r.getColNum();
    checkLineCol(r, lineNum, colNum);
    r.pushbackChar('0');
    // Unmoved.
    checkLineCol(r, lineNum, colNum);
    r.pushbackChar('1');
    checkLineCol(r, lineNum, colNum);
    assertEquals('1', r.readChar());
    // Unmoved.
    checkLineCol(r, lineNum, colNum);
    r.pushbackChar('2');
    r.pushbackChar('3');
    // Unmoved.
    checkLineCol(r, lineNum, colNum);
    assertEquals('3', r.peekChar());
    contains(r, "320");
}

Example 7 with PeekReader

use of org.apache.jena.atlas.io.PeekReader in project jena by apache.

the class TokenizerFactory method makeTokenizer.

/** Discouraged - be careful about character sets */
@Deprecated
public static Tokenizer makeTokenizer(Reader reader) {
    PeekReader peekReader = PeekReader.make(reader);
    Tokenizer tokenizer = new TokenizerText(peekReader);
    return tokenizer;
}

Example 8 with PeekReader

use of org.apache.jena.atlas.io.PeekReader in project jena by apache.

the class Token method createN.

static Iter<Token> createN(String s) {
    PeekReader pr = PeekReader.readString(s);
    TokenizerText tt = new TokenizerText(pr);
    List<Token> x = new ArrayList<>();
    while (tt.hasNext()) x.add(tt.next());
    return Iter.iter(x);
}

Example 9 with PeekReader

use of org.apache.jena.atlas.io.PeekReader in project jena by apache.

the class TokenizerFactory method makeTokenizerASCII.

public static Tokenizer makeTokenizerASCII(InputStream in) {
    PeekReader peekReader = PeekReader.makeASCII(in);
    Tokenizer tokenizer = new TokenizerText(peekReader);
    return tokenizer;
}

Example 10 with PeekReader

use of org.apache.jena.atlas.io.PeekReader in project jena by apache.

the class TestTokenizer method tokenizer.

private static Tokenizer tokenizer(String string, boolean lineMode) {
    PeekReader r = PeekReader.readString(string);
    Tokenizer tokenizer = new TokenizerText(r, lineMode);
    return tokenizer;
}