Search in sources :

Example 1 with StopwordAnalyzerBase

use of org.apache.lucene.analysis.StopwordAnalyzerBase in project neo4j by neo4j.

the class AnalyzerProvider method stopwords.

public List<String> stopwords() {
    Analyzer analyzer = createAnalyzer();
    if (analyzer instanceof StopwordAnalyzerBase) {
        StopwordAnalyzerBase stopwordAnalyzer = (StopwordAnalyzerBase) analyzer;
        CharArraySet stopwords = stopwordAnalyzer.getStopwordSet();
        return stopwords.stream().map(obj -> new String((char[]) obj)).collect(Collectors.toList());
    }
    return List.of();
}
Also used : List(java.util.List) Service(org.neo4j.annotations.service.Service) CharArraySet(org.apache.lucene.analysis.CharArraySet) Analyzer(org.apache.lucene.analysis.Analyzer) StopwordAnalyzerBase(org.apache.lucene.analysis.StopwordAnalyzerBase) Objects.requireNonNull(java.util.Objects.requireNonNull) PublicApi(org.neo4j.annotations.api.PublicApi) Collectors(java.util.stream.Collectors) NamedService(org.neo4j.service.NamedService) CharArraySet(org.apache.lucene.analysis.CharArraySet) StopwordAnalyzerBase(org.apache.lucene.analysis.StopwordAnalyzerBase) Analyzer(org.apache.lucene.analysis.Analyzer)

Example 2 with StopwordAnalyzerBase

use of org.apache.lucene.analysis.StopwordAnalyzerBase in project xodus by JetBrains.

the class ExodusLuceneTestsBase method removeStopWord.

protected void removeStopWord(final String stopWord) {
    final HashSet<Object> stopSet = new HashSet<>();
    for (Object word : ((StopwordAnalyzerBase) analyzer).getStopwordSet()) {
        if (!stopWord.equals(new String((char[]) word))) {
            stopSet.add(word);
        }
    }
    analyzer = new EnglishAnalyzer(LUCENE_VERSION, stopSet);
}
Also used : StopwordAnalyzerBase(org.apache.lucene.analysis.StopwordAnalyzerBase) EnglishAnalyzer(org.apache.lucene.analysis.en.EnglishAnalyzer) HashSet(jetbrains.exodus.core.dataStructures.hash.HashSet)

Aggregations

StopwordAnalyzerBase (org.apache.lucene.analysis.StopwordAnalyzerBase)2 List (java.util.List)1 Objects.requireNonNull (java.util.Objects.requireNonNull)1 Collectors (java.util.stream.Collectors)1 HashSet (jetbrains.exodus.core.dataStructures.hash.HashSet)1 Analyzer (org.apache.lucene.analysis.Analyzer)1 CharArraySet (org.apache.lucene.analysis.CharArraySet)1 EnglishAnalyzer (org.apache.lucene.analysis.en.EnglishAnalyzer)1 PublicApi (org.neo4j.annotations.api.PublicApi)1 Service (org.neo4j.annotations.service.Service)1 NamedService (org.neo4j.service.NamedService)1