Search in sources :

Example 6 with TokenFilterFactory

use of org.elasticsearch.index.analysis.TokenFilterFactory in project elasticsearch by elastic.

the class AnalysisModuleTests method assertTokenFilter.

private void assertTokenFilter(String name, Class<?> clazz) throws IOException {
    Settings settings = Settings.builder().put(IndexMetaData.SETTING_VERSION_CREATED, Version.CURRENT).put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toString()).build();
    TestAnalysis analysis = AnalysisTestsHelper.createTestAnalysisFromSettings(settings);
    TokenFilterFactory tokenFilter = analysis.tokenFilter.get(name);
    Tokenizer tokenizer = new WhitespaceTokenizer();
    tokenizer.setReader(new StringReader("foo bar"));
    TokenStream stream = tokenFilter.create(tokenizer);
    assertThat(stream, instanceOf(clazz));
}
Also used : WhitespaceTokenizer(org.apache.lucene.analysis.core.WhitespaceTokenizer) TokenStream(org.apache.lucene.analysis.TokenStream) StringReader(java.io.StringReader) WhitespaceTokenizer(org.apache.lucene.analysis.core.WhitespaceTokenizer) Tokenizer(org.apache.lucene.analysis.Tokenizer) Settings(org.elasticsearch.common.settings.Settings) IndexSettings(org.elasticsearch.index.IndexSettings) MyFilterTokenFilterFactory(org.elasticsearch.index.analysis.filter1.MyFilterTokenFilterFactory) StopTokenFilterFactory(org.elasticsearch.index.analysis.StopTokenFilterFactory) TokenFilterFactory(org.elasticsearch.index.analysis.TokenFilterFactory)

Aggregations

TokenFilterFactory (org.elasticsearch.index.analysis.TokenFilterFactory)6 Tokenizer (org.apache.lucene.analysis.Tokenizer)3 Settings (org.elasticsearch.common.settings.Settings)3 CustomAnalyzer (org.elasticsearch.index.analysis.CustomAnalyzer)3 NamedAnalyzer (org.elasticsearch.index.analysis.NamedAnalyzer)3 StringReader (java.io.StringReader)2 TokenStream (org.apache.lucene.analysis.TokenStream)2 WhitespaceTokenizer (org.apache.lucene.analysis.core.WhitespaceTokenizer)2 IndexSettings (org.elasticsearch.index.IndexSettings)2 Reader (java.io.Reader)1 HashSet (java.util.HashSet)1 FastStringReader (org.elasticsearch.common.io.FastStringReader)1 CharFilterFactory (org.elasticsearch.index.analysis.CharFilterFactory)1 EdgeNGramTokenFilterFactory (org.elasticsearch.index.analysis.EdgeNGramTokenFilterFactory)1 NGramTokenFilterFactory (org.elasticsearch.index.analysis.NGramTokenFilterFactory)1 ShingleTokenFilterFactory (org.elasticsearch.index.analysis.ShingleTokenFilterFactory)1 StopTokenFilterFactory (org.elasticsearch.index.analysis.StopTokenFilterFactory)1 TokenizerFactory (org.elasticsearch.index.analysis.TokenizerFactory)1 WordDelimiterTokenFilterFactory (org.elasticsearch.index.analysis.WordDelimiterTokenFilterFactory)1 MyFilterTokenFilterFactory (org.elasticsearch.index.analysis.filter1.MyFilterTokenFilterFactory)1