Examples with HunspellStemFilter org.apache.lucene.analysis.hunspell.HunspellStemFilter used on opensource projects

Example 1 with HunspellStemFilter

use of org.apache.lucene.analysis.hunspell.HunspellStemFilter in project omegat by omegat-org.

the class HunspellTokenizer method getTokenStream.

@Override
protected TokenStream getTokenStream(final String strOrig, final boolean stemsAllowed, final boolean stopWordsAllowed) throws IOException {
    StandardTokenizer tokenizer = new StandardTokenizer();
    tokenizer.setReader(new StringReader(strOrig));
    if (stemsAllowed) {
        Dictionary dictionary = getDict();
        if (dictionary == null) {
            return tokenizer;
        }
        return new HunspellStemFilter(tokenizer, dictionary);
    // / TODO: implement stop words checks
    } else {
        return tokenizer;
    }
}