Search in sources :

Example 1 with CapitalizationFilter

use of org.apache.lucene.analysis.miscellaneous.CapitalizationFilter in project lucene-solr by apache.

the class TestCapitalizationFilter method testRandomString.

/** blast some random strings through the analyzer */
public void testRandomString() throws Exception {
    Analyzer a = new Analyzer() {

        @Override
        protected TokenStreamComponents createComponents(String fieldName) {
            Tokenizer tokenizer = new MockTokenizer(MockTokenizer.WHITESPACE, false);
            return new TokenStreamComponents(tokenizer, new CapitalizationFilter(tokenizer));
        }
    };
    checkRandomData(random(), a, 1000 * RANDOM_MULTIPLIER);
    a.close();
}
Also used : MockTokenizer(org.apache.lucene.analysis.MockTokenizer) Analyzer(org.apache.lucene.analysis.Analyzer) Tokenizer(org.apache.lucene.analysis.Tokenizer) MockTokenizer(org.apache.lucene.analysis.MockTokenizer) KeywordTokenizer(org.apache.lucene.analysis.core.KeywordTokenizer) CapitalizationFilter(org.apache.lucene.analysis.miscellaneous.CapitalizationFilter)

Example 2 with CapitalizationFilter

use of org.apache.lucene.analysis.miscellaneous.CapitalizationFilter in project lucene-solr by apache.

the class TestCapitalizationFilter method testEmptyTerm.

public void testEmptyTerm() throws IOException {
    Analyzer a = new Analyzer() {

        @Override
        protected TokenStreamComponents createComponents(String fieldName) {
            Tokenizer tokenizer = new KeywordTokenizer();
            return new TokenStreamComponents(tokenizer, new CapitalizationFilter(tokenizer));
        }
    };
    checkOneTerm(a, "", "");
    a.close();
}
Also used : Analyzer(org.apache.lucene.analysis.Analyzer) KeywordTokenizer(org.apache.lucene.analysis.core.KeywordTokenizer) Tokenizer(org.apache.lucene.analysis.Tokenizer) MockTokenizer(org.apache.lucene.analysis.MockTokenizer) KeywordTokenizer(org.apache.lucene.analysis.core.KeywordTokenizer) CapitalizationFilter(org.apache.lucene.analysis.miscellaneous.CapitalizationFilter)

Example 3 with CapitalizationFilter

use of org.apache.lucene.analysis.miscellaneous.CapitalizationFilter in project lucene-solr by apache.

the class TestCapitalizationFilter method assertCapitalizesTo.

static void assertCapitalizesTo(Tokenizer tokenizer, String[] expected, boolean onlyFirstWord, CharArraySet keep, boolean forceFirstLetter, Collection<char[]> okPrefix, int minWordLength, int maxWordCount, int maxTokenLength) throws IOException {
    CapitalizationFilter filter = new CapitalizationFilter(tokenizer, onlyFirstWord, keep, forceFirstLetter, okPrefix, minWordLength, maxWordCount, maxTokenLength);
    assertTokenStreamContents(filter, expected);
}
Also used : CapitalizationFilter(org.apache.lucene.analysis.miscellaneous.CapitalizationFilter)

Aggregations

CapitalizationFilter (org.apache.lucene.analysis.miscellaneous.CapitalizationFilter)3 Analyzer (org.apache.lucene.analysis.Analyzer)2 MockTokenizer (org.apache.lucene.analysis.MockTokenizer)2 Tokenizer (org.apache.lucene.analysis.Tokenizer)2 KeywordTokenizer (org.apache.lucene.analysis.core.KeywordTokenizer)2