Search in sources :

Example 1 with QueryHelper

use of org.codelibs.fess.helper.QueryHelper in project fess by codelibs.

the class FessEsClient method search.

public <T> T search(final String index, final String type, final SearchCondition<SearchRequestBuilder> condition, final SearchResult<T, SearchRequestBuilder, SearchResponse> searchResult) {
    final long startTime = System.currentTimeMillis();
    SearchResponse searchResponse = null;
    final SearchRequestBuilder searchRequestBuilder = client.prepareSearch(index).setTypes(type);
    if (condition.build(searchRequestBuilder)) {
        if (ComponentUtil.hasQueryHelper()) {
            final QueryHelper queryHelper = ComponentUtil.getQueryHelper();
            if (queryHelper.getTimeAllowed() >= 0) {
                searchRequestBuilder.setTimeout(TimeValue.timeValueMillis(queryHelper.getTimeAllowed()));
            }
        }
        try {
            if (logger.isDebugEnabled()) {
                logger.debug("Query DSL:\n" + searchRequestBuilder.toString());
            }
            searchResponse = searchRequestBuilder.execute().actionGet(ComponentUtil.getFessConfig().getIndexSearchTimeout());
        } catch (final SearchPhaseExecutionException e) {
            throw new InvalidQueryException(messages -> messages.addErrorsInvalidQueryParseError(UserMessages.GLOBAL_PROPERTY_KEY), "Invalid query: " + searchRequestBuilder, e);
        }
    }
    final long execTime = System.currentTimeMillis() - startTime;
    return searchResult.build(searchRequestBuilder, execTime, OptionalEntity.ofNullable(searchResponse, () -> {
    }));
}
Also used : Arrays(java.util.Arrays) ExplainRequest(org.elasticsearch.action.explain.ExplainRequest) SearchHits(org.elasticsearch.search.SearchHits) Curl(org.codelibs.elasticsearch.runner.net.Curl) InetAddress(java.net.InetAddress) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest) QueryContext(org.codelibs.fess.entity.QueryContext) IndexRequest(org.elasticsearch.action.index.IndexRequest) DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder) UpdateResponse(org.elasticsearch.action.update.UpdateResponse) Map(java.util.Map) TermVectorsRequest(org.elasticsearch.action.termvectors.TermVectorsRequest) IllegalBehaviorStateException(org.dbflute.exception.IllegalBehaviorStateException) RefreshPolicy(org.elasticsearch.action.support.WriteRequest.RefreshPolicy) ActionRequest(org.elasticsearch.action.ActionRequest) MultiTermVectorsRequest(org.elasticsearch.action.termvectors.MultiTermVectorsRequest) ResourceUtil(org.codelibs.core.io.ResourceUtil) ExplainResponse(org.elasticsearch.action.explain.ExplainResponse) BulkResponse(org.elasticsearch.action.bulk.BulkResponse) MultiSearchRequestBuilder(org.elasticsearch.action.search.MultiSearchRequestBuilder) StandardCharsets(java.nio.charset.StandardCharsets) CreateIndexResponse(org.elasticsearch.action.admin.indices.create.CreateIndexResponse) DocMap(org.codelibs.fess.util.DocMap) MultiSearchResponse(org.elasticsearch.action.search.MultiSearchResponse) FileUtil(org.codelibs.core.io.FileUtil) ComponentUtil(org.codelibs.fess.util.ComponentUtil) TermVectorsResponse(org.elasticsearch.action.termvectors.TermVectorsResponse) DocumentHelper(org.codelibs.fess.helper.DocumentHelper) TermVectorsRequestBuilder(org.elasticsearch.action.termvectors.TermVectorsRequestBuilder) ClearScrollRequestBuilder(org.elasticsearch.action.search.ClearScrollRequestBuilder) AdminClient(org.elasticsearch.client.AdminClient) XContentFactory(org.elasticsearch.common.xcontent.XContentFactory) ExplainRequestBuilder(org.elasticsearch.action.explain.ExplainRequestBuilder) ImmutableOpenMap(org.elasticsearch.common.collect.ImmutableOpenMap) FessSystemException(org.codelibs.fess.exception.FessSystemException) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) IndicesAliasesResponse(org.elasticsearch.action.admin.indices.alias.IndicesAliasesResponse) GetMappingsResponse(org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse) TransportClient(org.elasticsearch.client.transport.TransportClient) SimpleDateFormat(java.text.SimpleDateFormat) ResultOffsetExceededException(org.codelibs.fess.exception.ResultOffsetExceededException) SearchRequest(org.elasticsearch.action.search.SearchRequest) ResourceNotFoundRuntimeException(org.codelibs.core.exception.ResourceNotFoundRuntimeException) Order(org.elasticsearch.search.aggregations.bucket.terms.Terms.Order) ArrayList(java.util.ArrayList) HighlightBuilder(org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder) OpType(org.elasticsearch.action.DocWriteRequest.OpType) SearchRequestType(org.codelibs.fess.entity.SearchRequestParams.SearchRequestType) MultiSearchRequest(org.elasticsearch.action.search.MultiSearchRequest) FacetInfo(org.codelibs.fess.entity.FacetInfo) ActionRequestBuilder(org.elasticsearch.action.ActionRequestBuilder) FlushResponse(org.elasticsearch.action.admin.indices.flush.FlushResponse) TermsAggregationBuilder(org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder) CollapseBuilder(org.elasticsearch.search.collapse.CollapseBuilder) QueryBuilder(org.elasticsearch.index.query.QueryBuilder) BaseEncoding(com.google.common.io.BaseEncoding) ActionResponse(org.elasticsearch.action.ActionResponse) Client(org.elasticsearch.client.Client) StringUtil(org.codelibs.core.lang.StringUtil) MultiGetRequestBuilder(org.elasticsearch.action.get.MultiGetRequestBuilder) IOException(java.io.IOException) PreBuiltTransportClient(org.elasticsearch.transport.client.PreBuiltTransportClient) UnknownHostException(java.net.UnknownHostException) File(java.io.File) IndicesExistsResponse(org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsResponse) ElasticsearchClusterRunner(org.codelibs.elasticsearch.runner.ElasticsearchClusterRunner) GetRequestBuilder(org.elasticsearch.action.get.GetRequestBuilder) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) ActionListener(org.elasticsearch.action.ActionListener) Builder(org.elasticsearch.common.settings.Settings.Builder) ElasticsearchException(org.elasticsearch.ElasticsearchException) Constants(org.codelibs.fess.Constants) GetResponse(org.elasticsearch.action.get.GetResponse) FieldStatsRequest(org.elasticsearch.action.fieldstats.FieldStatsRequest) Date(java.util.Date) MappingMetaData(org.elasticsearch.cluster.metadata.MappingMetaData) LoggerFactory(org.slf4j.LoggerFactory) FieldCapabilitiesResponse(org.elasticsearch.action.fieldcaps.FieldCapabilitiesResponse) MultiTermVectorsResponse(org.elasticsearch.action.termvectors.MultiTermVectorsResponse) QueryBuilders(org.elasticsearch.index.query.QueryBuilders) GeoInfo(org.codelibs.fess.entity.GeoInfo) PreDestroy(javax.annotation.PreDestroy) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) Settings(org.elasticsearch.common.settings.Settings) PingResponse(org.codelibs.fess.entity.PingResponse) QueryHelper(org.codelibs.fess.helper.QueryHelper) SearchResponse(org.elasticsearch.action.search.SearchResponse) CurlResponse(org.codelibs.elasticsearch.runner.net.CurlResponse) FieldCapabilitiesRequest(org.elasticsearch.action.fieldcaps.FieldCapabilitiesRequest) ThreadPool(org.elasticsearch.threadpool.ThreadPool) DeleteResponse(org.elasticsearch.action.delete.DeleteResponse) FieldStatsResponse(org.elasticsearch.action.fieldstats.FieldStatsResponse) TypeReference(com.fasterxml.jackson.core.type.TypeReference) Failure(org.elasticsearch.action.bulk.BulkItemResponse.Failure) SearchHit(org.elasticsearch.search.SearchHit) GetRequest(org.elasticsearch.action.get.GetRequest) MultiGetResponse(org.elasticsearch.action.get.MultiGetResponse) BulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse) InetSocketTransportAddress(org.elasticsearch.common.transport.InetSocketTransportAddress) MultiTermVectorsRequestBuilder(org.elasticsearch.action.termvectors.MultiTermVectorsRequestBuilder) FilterAggregationBuilder(org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) Collectors(java.util.stream.Collectors) SearchPhaseExecutionException(org.elasticsearch.action.search.SearchPhaseExecutionException) ElasticsearchClusterRunner.newConfigs(org.codelibs.elasticsearch.runner.ElasticsearchClusterRunner.newConfigs) List(java.util.List) FieldCapabilitiesRequestBuilder(org.elasticsearch.action.fieldcaps.FieldCapabilitiesRequestBuilder) IndexRequestBuilder(org.elasticsearch.action.index.IndexRequestBuilder) TransportAddress(org.elasticsearch.common.transport.TransportAddress) PostConstruct(javax.annotation.PostConstruct) Pattern(java.util.regex.Pattern) BulkRequestBuilder(org.elasticsearch.action.bulk.BulkRequestBuilder) BeanUtil(org.codelibs.core.beans.util.BeanUtil) FieldStatsRequestBuilder(org.elasticsearch.action.fieldstats.FieldStatsRequestBuilder) PutMappingResponse(org.elasticsearch.action.admin.indices.mapping.put.PutMappingResponse) MultiGetRequest(org.elasticsearch.action.get.MultiGetRequest) UpdateRequestBuilder(org.elasticsearch.action.update.UpdateRequestBuilder) HashMap(java.util.HashMap) ActionFuture(org.elasticsearch.action.ActionFuture) ClearScrollResponse(org.elasticsearch.action.search.ClearScrollResponse) SearchHitField(org.elasticsearch.search.SearchHitField) TimeValue(org.elasticsearch.common.unit.TimeValue) GetIndexResponse(org.elasticsearch.action.admin.indices.get.GetIndexResponse) IndexResponse(org.elasticsearch.action.index.IndexResponse) Result(org.elasticsearch.action.DocWriteResponse.Result) SearchQueryException(org.codelibs.fess.exception.SearchQueryException) UserMessages(org.lastaflute.core.message.UserMessages) StreamUtil.stream(org.codelibs.core.stream.StreamUtil.stream) Logger(org.slf4j.Logger) OptionalEntity(org.dbflute.optional.OptionalEntity) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) ClusterHealthResponse(org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse) UpdateRequest(org.elasticsearch.action.update.UpdateRequest) AggregationBuilders(org.elasticsearch.search.aggregations.AggregationBuilders) DocWriteRequest(org.elasticsearch.action.DocWriteRequest) Configs(org.codelibs.elasticsearch.runner.ElasticsearchClusterRunner.Configs) Action(org.elasticsearch.action.Action) IndicesAliasesRequestBuilder(org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequestBuilder) SearchRequestBuilder(org.elasticsearch.action.search.SearchRequestBuilder) RefreshResponse(org.elasticsearch.action.admin.indices.refresh.RefreshResponse) InnerHitBuilder(org.elasticsearch.index.query.InnerHitBuilder) InvalidQueryException(org.codelibs.fess.exception.InvalidQueryException) Collections(java.util.Collections) BulkRequest(org.elasticsearch.action.bulk.BulkRequest) SearchScrollRequestBuilder(org.elasticsearch.action.search.SearchScrollRequestBuilder) QueryHelper(org.codelibs.fess.helper.QueryHelper) MultiSearchRequestBuilder(org.elasticsearch.action.search.MultiSearchRequestBuilder) SearchRequestBuilder(org.elasticsearch.action.search.SearchRequestBuilder) SearchPhaseExecutionException(org.elasticsearch.action.search.SearchPhaseExecutionException) InvalidQueryException(org.codelibs.fess.exception.InvalidQueryException) MultiSearchResponse(org.elasticsearch.action.search.MultiSearchResponse) SearchResponse(org.elasticsearch.action.search.SearchResponse)

Example 2 with QueryHelper

use of org.codelibs.fess.helper.QueryHelper in project fess by codelibs.

the class QueryResponseList method init.

public void init(final OptionalEntity<SearchResponse> searchResponseOpt, final int start, final int pageSize) {
    searchResponseOpt.ifPresent(searchResponse -> {
        final FessConfig fessConfig = ComponentUtil.getFessConfig();
        final SearchHits searchHits = searchResponse.getHits();
        allRecordCount = searchHits.getTotalHits();
        queryTime = searchResponse.getTookInMillis();
        if (searchResponse.getTotalShards() != searchResponse.getSuccessfulShards()) {
            partialResults = true;
        }
        final QueryHelper queryHelper = ComponentUtil.getQueryHelper();
        final String hlPrefix = queryHelper.getHighlightPrefix();
        for (final SearchHit searchHit : searchHits.getHits()) {
            final Map<String, Object> docMap = parseSearchHit(fessConfig, hlPrefix, searchHit);
            if (fessConfig.isResultCollapsed()) {
                final Map<String, SearchHits> innerHits = searchHit.getInnerHits();
                if (innerHits != null) {
                    final SearchHits innerSearchHits = innerHits.get(fessConfig.getQueryCollapseInnerHitsName());
                    if (innerSearchHits != null) {
                        final long totalHits = innerSearchHits.getTotalHits();
                        if (totalHits > 1) {
                            docMap.put(fessConfig.getQueryCollapseInnerHitsName() + "_count", totalHits);
                            final SearchHitField bitsField = searchHit.getFields().get(fessConfig.getIndexFieldContentMinhashBits());
                            if (bitsField != null && !bitsField.getValues().isEmpty()) {
                                docMap.put(fessConfig.getQueryCollapseInnerHitsName() + "_hash", bitsField.getValues().get(0));
                            }
                            docMap.put(fessConfig.getQueryCollapseInnerHitsName(), StreamUtil.stream(innerSearchHits.getHits()).get(stream -> stream.map(v -> parseSearchHit(fessConfig, hlPrefix, v)).toArray(n -> new Map[n])));
                        }
                    }
                }
            }
            parent.add(docMap);
        }
        final Aggregations aggregations = searchResponse.getAggregations();
        if (aggregations != null) {
            facetResponse = new FacetResponse(aggregations);
        }
    });
    calculatePageInfo(start, pageSize);
}
Also used : SearchHit(org.elasticsearch.search.SearchHit) Logger(org.slf4j.Logger) Iterator(java.util.Iterator) ListIterator(java.util.ListIterator) SearchHits(org.elasticsearch.search.SearchHits) Aggregations(org.elasticsearch.search.aggregations.Aggregations) OptionalEntity(org.dbflute.optional.OptionalEntity) Collection(java.util.Collection) LoggerFactory(org.slf4j.LoggerFactory) HashMap(java.util.HashMap) StringUtils(org.apache.commons.lang3.StringUtils) ArrayList(java.util.ArrayList) List(java.util.List) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) StreamUtil(org.codelibs.core.stream.StreamUtil) Text(org.elasticsearch.common.text.Text) SearchHitField(org.elasticsearch.search.SearchHitField) Map(java.util.Map) QueryHelper(org.codelibs.fess.helper.QueryHelper) SearchResponse(org.elasticsearch.action.search.SearchResponse) HighlightField(org.elasticsearch.search.fetch.subphase.highlight.HighlightField) ViewHelper(org.codelibs.fess.helper.ViewHelper) SearchHit(org.elasticsearch.search.SearchHit) Aggregations(org.elasticsearch.search.aggregations.Aggregations) FessConfig(org.codelibs.fess.mylasta.direction.FessConfig) QueryHelper(org.codelibs.fess.helper.QueryHelper) SearchHitField(org.elasticsearch.search.SearchHitField) SearchHits(org.elasticsearch.search.SearchHits)

Aggregations

ArrayList (java.util.ArrayList)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 QueryHelper (org.codelibs.fess.helper.QueryHelper)2 FessConfig (org.codelibs.fess.mylasta.direction.FessConfig)2 OptionalEntity (org.dbflute.optional.OptionalEntity)2 SearchResponse (org.elasticsearch.action.search.SearchResponse)2 SearchHit (org.elasticsearch.search.SearchHit)2 SearchHitField (org.elasticsearch.search.SearchHitField)2 SearchHits (org.elasticsearch.search.SearchHits)2 Logger (org.slf4j.Logger)2 LoggerFactory (org.slf4j.LoggerFactory)2 TypeReference (com.fasterxml.jackson.core.type.TypeReference)1 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)1 BaseEncoding (com.google.common.io.BaseEncoding)1 File (java.io.File)1 IOException (java.io.IOException)1 InetAddress (java.net.InetAddress)1 UnknownHostException (java.net.UnknownHostException)1