Search in sources :

Example 1 with TurkishLowerCaseFilter

use of org.apache.lucene.analysis.tr.TurkishLowerCaseFilter in project elasticsearch by elastic.

the class SnowballAnalyzer method createComponents.

/** Constructs a {@link StandardTokenizer} filtered by a {@link
      StandardFilter}, a {@link LowerCaseFilter}, a {@link StopFilter},
      and a {@link SnowballFilter} */
@Override
public TokenStreamComponents createComponents(String fieldName) {
    final Tokenizer tokenizer = new StandardTokenizer();
    TokenStream result = tokenizer;
    // remove the possessive 's for english stemmers
    if (name.equals("English") || name.equals("Porter") || name.equals("Lovins"))
        result = new EnglishPossessiveFilter(result);
    // Use a special lowercase filter for turkish, the stemmer expects it.
    if (name.equals("Turkish"))
        result = new TurkishLowerCaseFilter(result);
    else
        result = new LowerCaseFilter(result);
    if (stopSet != null)
        result = new StopFilter(result, stopSet);
    result = new SnowballFilter(result, name);
    return new TokenStreamComponents(tokenizer, result);
}
Also used : TokenStream(org.apache.lucene.analysis.TokenStream) EnglishPossessiveFilter(org.apache.lucene.analysis.en.EnglishPossessiveFilter) StandardTokenizer(org.apache.lucene.analysis.standard.StandardTokenizer) StopFilter(org.apache.lucene.analysis.StopFilter) TurkishLowerCaseFilter(org.apache.lucene.analysis.tr.TurkishLowerCaseFilter) SnowballFilter(org.apache.lucene.analysis.snowball.SnowballFilter) Tokenizer(org.apache.lucene.analysis.Tokenizer) StandardTokenizer(org.apache.lucene.analysis.standard.StandardTokenizer) LowerCaseFilter(org.apache.lucene.analysis.LowerCaseFilter) TurkishLowerCaseFilter(org.apache.lucene.analysis.tr.TurkishLowerCaseFilter)

Aggregations

LowerCaseFilter (org.apache.lucene.analysis.LowerCaseFilter)1 StopFilter (org.apache.lucene.analysis.StopFilter)1 TokenStream (org.apache.lucene.analysis.TokenStream)1 Tokenizer (org.apache.lucene.analysis.Tokenizer)1 EnglishPossessiveFilter (org.apache.lucene.analysis.en.EnglishPossessiveFilter)1 SnowballFilter (org.apache.lucene.analysis.snowball.SnowballFilter)1 StandardTokenizer (org.apache.lucene.analysis.standard.StandardTokenizer)1 TurkishLowerCaseFilter (org.apache.lucene.analysis.tr.TurkishLowerCaseFilter)1