Search in sources :

Example 1 with ESSourceReader

use of org.codelibs.fess.suggest.index.contents.document.ESSourceReader in project fess by codelibs.

the class SuggestHelper method indexFromDocuments.

public void indexFromDocuments(final Consumer<Boolean> success, final Consumer<Throwable> error) {
    final FessConfig fessConfig = ComponentUtil.getFessConfig();
    suggester.indexer().indexFromDocument(() -> {
        final ESSourceReader reader = new ESSourceReader(fessEsClient, suggester.settings(), fessConfig.getIndexDocumentSearchIndex(), fessConfig.getIndexDocumentType());
        reader.setScrollSize(fessConfig.getSuggestSourceReaderScrollSizeAsInteger());
        reader.setLimitDocNumPercentage(fessConfig.getSuggestUpdateContentsLimitNumPercentage());
        reader.setLimitNumber(fessConfig.getSuggestUpdateContentsLimitNumAsInteger());
        final List<FunctionScoreQueryBuilder.FilterFunctionBuilder> flist = new ArrayList<>();
        flist.add(new FunctionScoreQueryBuilder.FilterFunctionBuilder(ScoreFunctionBuilders.randomFunction(System.currentTimeMillis())));
        reader.setQuery(QueryBuilders.functionScoreQuery(QueryBuilders.matchAllQuery(), flist.toArray(new FunctionScoreQueryBuilder.FilterFunctionBuilder[flist.size()])).boostMode(CombineFunction.MULTIPLY));
        reader.addSort(SortBuilders.fieldSort(fessConfig.getIndexFieldClickCount()));
        reader.addSort(SortBuilders.scoreSort());
        return reader;
    }, 2, fessConfig.getSuggestUpdateRequestIntervalAsInteger().longValue()).then(response -> {
        suggester.refresh();
        success.accept(true);
    }).error(t -> error.accept(t));
}
Also used : Arrays(java.util.Arrays) SortBuilders(org.elasticsearch.search.sort.SortBuilders) LocalDateTime(java.time.LocalDateTime) LoggerFactory(org.slf4j.LoggerFactory) BadWordBhv(org.codelibs.fess.es.config.exbhv.BadWordBhv) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) FieldNames(org.codelibs.fess.suggest.constants.FieldNames) ScoreFunctionBuilders(org.elasticsearch.index.query.functionscore.ScoreFunctionBuilders) ArrayList(java.util.ArrayList) SuggestUtil(org.codelibs.fess.suggest.util.SuggestUtil) HashSet(java.util.HashSet) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) StreamUtil.split(org.codelibs.core.stream.StreamUtil.split) BadWord(org.codelibs.fess.es.config.exentity.BadWord) FunctionScoreQueryBuilder(org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder) ElevateWordBhv(org.codelibs.fess.es.config.exbhv.ElevateWordBhv) SuggestDeleteResponse(org.codelibs.fess.suggest.index.SuggestDeleteResponse) SearchFieldLog(org.codelibs.fess.es.log.exentity.SearchFieldLog) ESSourceReader(org.codelibs.fess.suggest.index.contents.document.ESSourceReader) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) Suggester(org.codelibs.fess.suggest.Suggester) Logger(org.slf4j.Logger) FessEsClient(org.codelibs.fess.es.client.FessEsClient) SearchLog(org.codelibs.fess.es.log.exentity.SearchLog) Resource(javax.annotation.Resource) StringUtil(org.codelibs.core.lang.StringUtil) Set(java.util.Set) ZoneId(java.time.ZoneId) Consumer(java.util.function.Consumer) SuggestSettings(org.codelibs.fess.suggest.settings.SuggestSettings) List(java.util.List) CombineFunction(org.elasticsearch.common.lucene.search.function.CombineFunction) ComponentUtil(org.codelibs.fess.util.ComponentUtil) SuggestItem(org.codelibs.fess.suggest.entity.SuggestItem) PostConstruct(javax.annotation.PostConstruct) ElevateWord(org.codelibs.fess.es.config.exentity.ElevateWord) BoolQueryBuilder(org.elasticsearch.index.query.BoolQueryBuilder) FunctionScoreQueryBuilder(org.elasticsearch.index.query.functionscore.FunctionScoreQueryBuilder) ArrayList(java.util.ArrayList) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) ESSourceReader(org.codelibs.fess.suggest.index.contents.document.ESSourceReader)

Aggregations

LocalDateTime (java.time.LocalDateTime)1 ZoneId (java.time.ZoneId)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 HashSet (java.util.HashSet)1 List (java.util.List)1 Set (java.util.Set)1 Consumer (java.util.function.Consumer)1 PostConstruct (javax.annotation.PostConstruct)1 Resource (javax.annotation.Resource)1 StringUtil (org.codelibs.core.lang.StringUtil)1 StreamUtil.split (org.codelibs.core.stream.StreamUtil.split)1 StreamUtil.stream (org.codelibs.core.stream.StreamUtil.stream)1 FessEsClient (org.codelibs.fess.es.client.FessEsClient)1 BadWordBhv (org.codelibs.fess.es.config.exbhv.BadWordBhv)1 ElevateWordBhv (org.codelibs.fess.es.config.exbhv.ElevateWordBhv)1 BadWord (org.codelibs.fess.es.config.exentity.BadWord)1 ElevateWord (org.codelibs.fess.es.config.exentity.ElevateWord)1 SearchFieldLog (org.codelibs.fess.es.log.exentity.SearchFieldLog)1 SearchLog (org.codelibs.fess.es.log.exentity.SearchLog)1