Examples with CharFilter org.apache.lucene.analysis.CharFilter used on opensource projects

Example 31 with CharFilter

use of org.apache.lucene.analysis.CharFilter in project lucene-solr by apache.

the class TestMappingCharFilter method test4to2.

public void test4to2() throws Exception {
    CharFilter cs = new MappingCharFilter(normMap, new StringReader("cccc"));
    TokenStream ts = whitespaceMockTokenizer(cs);
    assertTokenStreamContents(ts, new String[] { "cc" }, new int[] { 0 }, new int[] { 4 }, 4);
}

Example 32 with CharFilter

use of org.apache.lucene.analysis.CharFilter in project lucene-solr by apache.

the class TestMappingCharFilter method testReaderReset.

public void testReaderReset() throws Exception {
    CharFilter cs = new MappingCharFilter(normMap, new StringReader("x"));
    char[] buf = new char[10];
    int len = cs.read(buf, 0, 10);
    assertEquals(1, len);
    assertEquals('x', buf[0]);
    len = cs.read(buf, 0, 10);
    assertEquals(-1, len);
    // rewind
    cs.reset();
    len = cs.read(buf, 0, 10);
    assertEquals(1, len);
    assertEquals('x', buf[0]);
}

Example 33 with CharFilter

use of org.apache.lucene.analysis.CharFilter in project lucene-solr by apache.

the class TestMappingCharFilter method test2to1.

public void test2to1() throws Exception {
    CharFilter cs = new MappingCharFilter(normMap, new StringReader("aa"));
    TokenStream ts = whitespaceMockTokenizer(cs);
    assertTokenStreamContents(ts, new String[] { "a" }, new int[] { 0 }, new int[] { 2 }, 2);
}

Example 34 with CharFilter

use of org.apache.lucene.analysis.CharFilter in project lucene-solr by apache.

the class TestMappingCharFilter method testNonBMPChar.

public void testNonBMPChar() throws Exception {
    CharFilter cs = new MappingCharFilter(normMap, new StringReader(UnicodeUtil.newString(new int[] { 0x1D122 }, 0, 1)));
    TokenStream ts = whitespaceMockTokenizer(cs);
    assertTokenStreamContents(ts, new String[] { "fclef" }, new int[] { 0 }, new int[] { 2 }, 2);
}

Example 35 with CharFilter

use of org.apache.lucene.analysis.CharFilter in project lucene-solr by apache.

the class TestMappingCharFilter method test3to1.

public void test3to1() throws Exception {
    CharFilter cs = new MappingCharFilter(normMap, new StringReader("bbb"));
    TokenStream ts = whitespaceMockTokenizer(cs);
    assertTokenStreamContents(ts, new String[] { "b" }, new int[] { 0 }, new int[] { 3 }, 3);
}