Search in sources :

Example 26 with TokenStreamRewriter

use of org.antlr.v4.runtime.TokenStreamRewriter in project antlr4 by antlr.

the class TestTokenStreamRewriter method testReplaceThenInsertBeforeLastIndex.

@Test
public void testReplaceThenInsertBeforeLastIndex() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar T;\n" + "A : 'a';\n" + "B : 'b';\n" + "C : 'c';\n");
    String input = "abc";
    LexerInterpreter lexEngine = g.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream stream = new CommonTokenStream(lexEngine);
    stream.fill();
    TokenStreamRewriter tokens = new TokenStreamRewriter(stream);
    tokens.replace(2, "x");
    tokens.insertBefore(2, "y");
    String result = tokens.getText();
    String expecting = "abyx";
    assertEquals(expecting, result);
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) LexerInterpreter(org.antlr.v4.runtime.LexerInterpreter) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) TokenStreamRewriter(org.antlr.v4.runtime.TokenStreamRewriter) BaseJavaTest(org.antlr.v4.test.runtime.java.BaseJavaTest) Test(org.junit.Test)

Example 27 with TokenStreamRewriter

use of org.antlr.v4.runtime.TokenStreamRewriter in project antlr4 by antlr.

the class TestTokenStreamRewriter method testCombineInsertOnLeftWithReplace.

@Test
public void testCombineInsertOnLeftWithReplace() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar T;\n" + "A : 'a';\n" + "B : 'b';\n" + "C : 'c';\n");
    String input = "abc";
    LexerInterpreter lexEngine = g.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream stream = new CommonTokenStream(lexEngine);
    stream.fill();
    TokenStreamRewriter tokens = new TokenStreamRewriter(stream);
    tokens.replace(0, 2, "foo");
    tokens.insertBefore(0, "z");
    stream.fill();
    // combine with left edge of rewrite
    String result = tokens.getText();
    String expecting = "zfoo";
    assertEquals(expecting, result);
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) LexerInterpreter(org.antlr.v4.runtime.LexerInterpreter) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) TokenStreamRewriter(org.antlr.v4.runtime.TokenStreamRewriter) BaseJavaTest(org.antlr.v4.test.runtime.java.BaseJavaTest) Test(org.junit.Test)

Example 28 with TokenStreamRewriter

use of org.antlr.v4.runtime.TokenStreamRewriter in project antlr4 by antlr.

the class TestTokenStreamRewriter method testReplaceRangeThenInsertAtLeftEdge.

@Test
public void testReplaceRangeThenInsertAtLeftEdge() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar T;\n" + "A : 'a';\n" + "B : 'b';\n" + "C : 'c';\n");
    String input = "abcccba";
    LexerInterpreter lexEngine = g.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream stream = new CommonTokenStream(lexEngine);
    stream.fill();
    TokenStreamRewriter tokens = new TokenStreamRewriter(stream);
    tokens.replace(2, 4, "x");
    tokens.insertBefore(2, "y");
    String result = tokens.getText();
    String expecting = "abyxba";
    assertEquals(expecting, result);
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) LexerInterpreter(org.antlr.v4.runtime.LexerInterpreter) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) TokenStreamRewriter(org.antlr.v4.runtime.TokenStreamRewriter) BaseJavaTest(org.antlr.v4.test.runtime.java.BaseJavaTest) Test(org.junit.Test)

Example 29 with TokenStreamRewriter

use of org.antlr.v4.runtime.TokenStreamRewriter in project antlr4 by antlr.

the class TestTokenStreamRewriter method testDropIdenticalReplace.

@Test
public void testDropIdenticalReplace() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar T;\n" + "A : 'a';\n" + "B : 'b';\n" + "C : 'c';\n");
    String input = "abcc";
    LexerInterpreter lexEngine = g.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream stream = new CommonTokenStream(lexEngine);
    stream.fill();
    TokenStreamRewriter tokens = new TokenStreamRewriter(stream);
    tokens.replace(1, 2, "foo");
    tokens.replace(1, 2, "foo");
    stream.fill();
    // drop previous, identical
    String result = tokens.getText();
    String expecting = "afooc";
    assertEquals(expecting, result);
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) LexerInterpreter(org.antlr.v4.runtime.LexerInterpreter) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) TokenStreamRewriter(org.antlr.v4.runtime.TokenStreamRewriter) BaseJavaTest(org.antlr.v4.test.runtime.java.BaseJavaTest) Test(org.junit.Test)

Example 30 with TokenStreamRewriter

use of org.antlr.v4.runtime.TokenStreamRewriter in project antlr4 by antlr.

the class TestTokenStreamRewriter method test2InsertBeforeAfterMiddleIndex.

@Test
public void test2InsertBeforeAfterMiddleIndex() throws Exception {
    LexerGrammar g = new LexerGrammar("lexer grammar T;\n" + "A : 'a';\n" + "B : 'b';\n" + "C : 'c';\n");
    String input = "abc";
    LexerInterpreter lexEngine = g.createLexerInterpreter(new ANTLRInputStream(input));
    CommonTokenStream stream = new CommonTokenStream(lexEngine);
    stream.fill();
    TokenStreamRewriter tokens = new TokenStreamRewriter(stream);
    tokens.insertBefore(1, "x");
    tokens.insertAfter(1, "x");
    String result = tokens.getText();
    String expecting = "axbxc";
    assertEquals(expecting, result);
}
Also used : CommonTokenStream(org.antlr.v4.runtime.CommonTokenStream) LexerInterpreter(org.antlr.v4.runtime.LexerInterpreter) LexerGrammar(org.antlr.v4.tool.LexerGrammar) ANTLRInputStream(org.antlr.v4.runtime.ANTLRInputStream) TokenStreamRewriter(org.antlr.v4.runtime.TokenStreamRewriter) BaseJavaTest(org.antlr.v4.test.runtime.java.BaseJavaTest) Test(org.junit.Test)

Aggregations

ANTLRInputStream (org.antlr.v4.runtime.ANTLRInputStream)43 CommonTokenStream (org.antlr.v4.runtime.CommonTokenStream)43 LexerInterpreter (org.antlr.v4.runtime.LexerInterpreter)43 TokenStreamRewriter (org.antlr.v4.runtime.TokenStreamRewriter)43 BaseJavaTest (org.antlr.v4.test.runtime.java.BaseJavaTest)43 LexerGrammar (org.antlr.v4.tool.LexerGrammar)43 Test (org.junit.Test)43