Examples with PatternTokenizer org.apache.lucene.analysis.pattern.PatternTokenizer used on opensource projects

Example 1 with PatternTokenizer

use of org.apache.lucene.analysis.pattern.PatternTokenizer in project elasticsearch by elastic.

the class PatternAnalyzer method createComponents.

@Override
protected TokenStreamComponents createComponents(String s) {
    final Tokenizer tokenizer = new PatternTokenizer(pattern, -1);
    TokenStream stream = tokenizer;
    if (lowercase) {
        stream = new LowerCaseFilter(stream);
    }
    if (stopWords != null) {
        stream = new StopFilter(stream, stopWords);
    }
    return new TokenStreamComponents(tokenizer, stream);
}