Search in sources :

Example 1 with SuggestionRequest

use of org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest in project graylog2-server by Graylog2.

the class QuerySuggestionsES7 method suggest.

@Override
public SuggestionResponse suggest(SuggestionRequest req) {
    final Set<String> affectedIndices = indexLookup.indexNamesForStreamsInTimeRange(req.streams(), req.timerange());
    final TermSuggestionBuilder suggestionBuilder = SuggestBuilders.termSuggestion(req.field()).text(req.input()).size(req.size());
    final SearchSourceBuilder search = new SearchSourceBuilder().query(QueryBuilders.prefixQuery(req.field(), req.input())).size(0).aggregation(AggregationBuilders.terms("fieldvalues").field(req.field()).size(req.size())).suggest(new SuggestBuilder().addSuggestion("corrections", suggestionBuilder));
    try {
        final SearchResponse result = client.singleSearch(new SearchRequest(affectedIndices.toArray(new String[] {})).source(search), "Failed to execute aggregation");
        final ParsedStringTerms fieldValues = result.getAggregations().get("fieldvalues");
        final List<SuggestionEntry> entries = fieldValues.getBuckets().stream().map(b -> new SuggestionEntry(b.getKeyAsString(), b.getDocCount())).collect(Collectors.toList());
        if (!entries.isEmpty()) {
            return SuggestionResponse.forSuggestions(req.field(), req.input(), entries, fieldValues.getSumOfOtherDocCounts());
        } else {
            TermSuggestion suggestion = result.getSuggest().getSuggestion("corrections");
            final List<SuggestionEntry> corrections = suggestion.getEntries().stream().flatMap(e -> e.getOptions().stream()).map(o -> new SuggestionEntry(o.getText().string(), o.getFreq())).collect(Collectors.toList());
            return SuggestionResponse.forSuggestions(req.field(), req.input(), corrections, null);
        }
    } catch (org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchException exception) {
        final SuggestionError err = tryResponseException(exception).orElseGet(() -> parseException(exception));
        return SuggestionResponse.forError(req.field(), req.input(), err);
    }
}
Also used : TermSuggestionBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestionBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) ParsedStringTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedStringTerms) SuggestBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) Set(java.util.Set) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) SuggestionRequest(org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest) Collectors(java.util.stream.Collectors) AggregationBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilders) Inject(javax.inject.Inject) List(java.util.List) QuerySuggestionsService(org.graylog.plugins.views.search.engine.QuerySuggestionsService) ResponseError(org.graylog.storage.elasticsearch7.errors.ResponseError) Optional(java.util.Optional) SuggestBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilders) SuggestionResponse(org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) QueryBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.query.QueryBuilders) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) ParsedStringTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedStringTerms) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) SuggestBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) TermSuggestionBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestionBuilder) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry)

Example 2 with SuggestionRequest

use of org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest in project graylog2-server by Graylog2.

the class QuerySuggestionsES6 method suggest.

@Override
public SuggestionResponse suggest(SuggestionRequest req) {
    final Set<String> affectedIndices = indexLookup.indexNamesForStreamsInTimeRange(req.streams(), req.timerange());
    final SearchSourceBuilder search = new SearchSourceBuilder().query(QueryBuilders.prefixQuery(req.field(), req.input())).size(0).aggregation(AggregationBuilders.terms("fieldvalues").field(req.field()).size(req.size())).suggest(new SuggestBuilder().addSuggestion("corrections", SuggestBuilders.termSuggestion(req.field()).text(req.input()).size(req.size())));
    final Search.Builder searchBuilder = new Search.Builder(search.toString()).addType(IndexMapping.TYPE_MESSAGE).addIndex(affectedIndices.isEmpty() ? Collections.singleton("") : affectedIndices).allowNoIndices(false).ignoreUnavailable(false);
    try {
        final SearchResult result = JestUtils.execute(jestClient, searchBuilder.build(), () -> "Unable to perform aggregation: ");
        final TermsAggregation aggregation = result.getAggregations().getTermsAggregation("fieldvalues");
        final List<SuggestionEntry> entries = aggregation.getBuckets().stream().map(b -> new SuggestionEntry(b.getKeyAsString(), b.getCount())).collect(Collectors.toList());
        if (!entries.isEmpty()) {
            return SuggestionResponse.forSuggestions(req.field(), req.input(), entries, aggregation.getSumOtherDocCount());
        } else {
            final List<SuggestionEntry> corrections = Optional.of(result.getJsonObject()).map(o -> o.get("suggest")).map(o -> o.get("corrections")).map(o -> o.get(0)).map(o -> o.get("options")).map(options -> StreamSupport.stream(Spliterators.spliteratorUnknownSize(options.elements(), Spliterator.ORDERED), false).map(option -> new SuggestionEntry(option.get("text").textValue(), option.get("freq").longValue())).collect(Collectors.toList())).orElseGet(Collections::emptyList);
            return SuggestionResponse.forSuggestions(req.field(), req.input(), corrections, null);
        }
    } catch (Exception e) {
        final SuggestionError err = SuggestionError.create(e.getClass().getSimpleName(), e.getMessage());
        return SuggestionResponse.forError(req.field(), req.input(), err);
    }
}
Also used : TermsAggregation(io.searchbox.core.search.aggregation.TermsAggregation) SuggestBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry) Spliterators(java.util.Spliterators) JestClient(io.searchbox.client.JestClient) Inject(javax.inject.Inject) JestUtils(org.graylog.storage.elasticsearch6.jest.JestUtils) StreamSupport(java.util.stream.StreamSupport) SuggestionResponse(org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse) IndexMapping(org.graylog2.indexer.IndexMapping) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) QueryBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders) Search(io.searchbox.core.Search) Set(java.util.Set) SuggestBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.suggest.SuggestBuilders) SuggestionRequest(org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest) Collectors(java.util.stream.Collectors) List(java.util.List) QuerySuggestionsService(org.graylog.plugins.views.search.engine.QuerySuggestionsService) AggregationBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilders) Optional(java.util.Optional) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) Collections(java.util.Collections) Spliterator(java.util.Spliterator) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(io.searchbox.core.SearchResult) TermsAggregation(io.searchbox.core.search.aggregation.TermsAggregation) SuggestBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.suggest.SuggestBuilder) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(io.searchbox.core.SearchResult) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SuggestBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) Search(io.searchbox.core.Search) Collections(java.util.Collections) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry)

Example 3 with SuggestionRequest

use of org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest in project graylog2-server by Graylog2.

the class SuggestionsResource method suggestFieldValue.

@POST
@Produces(MediaType.APPLICATION_JSON)
@ApiOperation("Suggest field value")
@NoAuditEvent("Only suggesting field value for query, not changing any data")
public SuggestionsDTO suggestFieldValue(@ApiParam(name = "validationRequest") SuggestionsRequestDTO suggestionsRequest, @Context SearchUser searchUser) {
    final SuggestionRequest req = SuggestionRequest.builder().field(suggestionsRequest.field()).input(suggestionsRequest.input()).streams(adaptStreams(suggestionsRequest.streams(), searchUser)).size(Math.min(suggestionsRequest.size(), SUGGESTIONS_COUNT_MAX)).timerange(Optional.ofNullable(suggestionsRequest.timerange()).orElse(defaultTimeRange())).build();
    SuggestionResponse res = querySuggestionsService.suggest(req);
    final List<SuggestionEntryDTO> suggestions = res.suggestions().stream().map(s -> SuggestionEntryDTO.create(s.getValue(), s.getOccurrence())).collect(Collectors.toList());
    final SuggestionsDTO.Builder suggestionsBuilder = SuggestionsDTO.builder(res.field(), res.input()).suggestions(suggestions).sumOtherDocsCount(res.sumOtherDocsCount());
    res.suggestionError().map(e -> SuggestionsErrorDTO.create(e.type(), e.reason())).ifPresent(suggestionsBuilder::error);
    return suggestionsBuilder.build();
}
Also used : InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Produces(javax.ws.rs.Produces) SuggestionEntryDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionEntryDTO) SuggestionsDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionsDTO) Path(javax.ws.rs.Path) ApiParam(io.swagger.annotations.ApiParam) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) Inject(javax.inject.Inject) ApiOperation(io.swagger.annotations.ApiOperation) MediaType(javax.ws.rs.core.MediaType) SuggestionsErrorDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionsErrorDTO) SuggestionsRequestDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionsRequestDTO) PluginRestResource(org.graylog2.plugin.rest.PluginRestResource) Api(io.swagger.annotations.Api) SuggestionResponse(org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent) ImmutableSet(com.google.common.collect.ImmutableSet) POST(javax.ws.rs.POST) Context(javax.ws.rs.core.Context) Set(java.util.Set) RestResource(org.graylog2.shared.rest.resources.RestResource) SuggestionRequest(org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest) Collectors(java.util.stream.Collectors) List(java.util.List) QuerySuggestionsService(org.graylog.plugins.views.search.engine.QuerySuggestionsService) Optional(java.util.Optional) SearchUser(org.graylog.plugins.views.search.permissions.SearchUser) RequiresAuthentication(org.apache.shiro.authz.annotation.RequiresAuthentication) SuggestionsDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionsDTO) SuggestionResponse(org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse) SuggestionRequest(org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest) SuggestionEntryDTO(org.graylog.plugins.views.search.rest.suggestions.SuggestionEntryDTO) POST(javax.ws.rs.POST) Produces(javax.ws.rs.Produces) ApiOperation(io.swagger.annotations.ApiOperation) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent)

Aggregations

List (java.util.List)3 Optional (java.util.Optional)3 Set (java.util.Set)3 Collectors (java.util.stream.Collectors)3 Inject (javax.inject.Inject)3 QuerySuggestionsService (org.graylog.plugins.views.search.engine.QuerySuggestionsService)3 SuggestionRequest (org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest)3 SuggestionResponse (org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse)3 IndexLookup (org.graylog.plugins.views.search.elasticsearch.IndexLookup)2 SuggestionEntry (org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry)2 SuggestionError (org.graylog.plugins.views.search.engine.suggestions.SuggestionError)2 ImmutableSet (com.google.common.collect.ImmutableSet)1 JestClient (io.searchbox.client.JestClient)1 Search (io.searchbox.core.Search)1 SearchResult (io.searchbox.core.SearchResult)1 TermsAggregation (io.searchbox.core.search.aggregation.TermsAggregation)1 Api (io.swagger.annotations.Api)1 ApiOperation (io.swagger.annotations.ApiOperation)1 ApiParam (io.swagger.annotations.ApiParam)1 Collections (java.util.Collections)1