Examples with PolishAnalyzer org.apache.lucene.analysis.pl.PolishAnalyzer used on opensource projects

Example 1 with PolishAnalyzer

use of org.apache.lucene.analysis.pl.PolishAnalyzer in project omegat by omegat-org.

the class LucenePolishTokenizer method getTokenStream.

@SuppressWarnings("resource")
@Override
protected TokenStream getTokenStream(final String strOrig, final boolean stemsAllowed, final boolean stopWordsAllowed) throws IOException {
    if (stemsAllowed) {
        CharArraySet stopWords = stopWordsAllowed ? PolishAnalyzer.getDefaultStopSet() : CharArraySet.EMPTY_SET;
        PolishAnalyzer analyzer = new PolishAnalyzer(stopWords);
        return analyzer.tokenStream("", new StringReader(strOrig));
    } else {
        return getStandardTokenStream(strOrig);
    }
}