Search in sources :

Example 11 with ResultMessageSummary

use of org.graylog2.rest.models.messages.responses.ResultMessageSummary in project graylog2-server by Graylog2.

the class DecoratorProcessorImpl method getSearchDecoratorStats.

private SearchDecorationStats getSearchDecoratorStats(List<ResultMessageSummary> decoratedMessages) {
    final Set<String> addedFields = new HashSet<>();
    final Set<String> changedFields = new HashSet<>();
    final Set<String> removedFields = new HashSet<>();
    decoratedMessages.forEach(message -> {
        final DecorationStats decorationStats = message.decorationStats();
        if (decorationStats != null) {
            addedFields.addAll(decorationStats.addedFields().keySet());
            changedFields.addAll(decorationStats.changedFields().keySet());
            removedFields.addAll(decorationStats.removedFields().keySet());
        }
    });
    return SearchDecorationStats.create(addedFields, changedFields, removedFields);
}
Also used : SearchDecorationStats(org.graylog2.rest.resources.search.responses.SearchDecorationStats) DecorationStats(org.graylog2.rest.models.messages.responses.DecorationStats) HashSet(java.util.HashSet)

Example 12 with ResultMessageSummary

use of org.graylog2.rest.models.messages.responses.ResultMessageSummary in project graylog2-server by Graylog2.

the class DecoratorProcessorImpl method decorate.

@Override
public SearchResponse decorate(SearchResponse searchResponse, List<SearchResponseDecorator> searchResponseDecorators) {
    try {
        final Optional<SearchResponseDecorator> metaDecorator = searchResponseDecorators.stream().reduce((f, g) -> (v) -> g.apply(f.apply(v)));
        if (metaDecorator.isPresent()) {
            final Map<String, ResultMessageSummary> originalMessages = searchResponse.messages().stream().collect(Collectors.toMap(this::getMessageKey, Function.identity()));
            final SearchResponse newSearchResponse = metaDecorator.get().apply(searchResponse);
            final Set<String> newFields = extractFields(newSearchResponse.messages());
            final List<ResultMessageSummary> decoratedMessages = newSearchResponse.messages().stream().map(resultMessage -> {
                final ResultMessageSummary originalMessage = originalMessages.get(getMessageKey(resultMessage));
                if (originalMessage != null) {
                    return resultMessage.toBuilder().decorationStats(DecorationStats.create(originalMessage.message(), resultMessage.message())).build();
                }
                return resultMessage;
            }).collect(Collectors.toList());
            return newSearchResponse.toBuilder().messages(decoratedMessages).fields(newFields).decorationStats(this.getSearchDecoratorStats(decoratedMessages)).build();
        }
    } catch (Exception e) {
        LOG.error("Error decorating search response", e);
    }
    return searchResponse;
}
Also used : Logger(org.slf4j.Logger) LoggerFactory(org.slf4j.LoggerFactory) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse) Set(java.util.Set) SearchResponseDecorator(org.graylog2.plugin.decorators.SearchResponseDecorator) SearchDecorationStats(org.graylog2.rest.resources.search.responses.SearchDecorationStats) DecorationStats(org.graylog2.rest.models.messages.responses.DecorationStats) Function(java.util.function.Function) Collectors(java.util.stream.Collectors) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) Inject(javax.inject.Inject) HashSet(java.util.HashSet) List(java.util.List) Map(java.util.Map) Optional(java.util.Optional) Message(org.graylog2.plugin.Message) SearchResponseDecorator(org.graylog2.plugin.decorators.SearchResponseDecorator) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse)

Example 13 with ResultMessageSummary

use of org.graylog2.rest.models.messages.responses.ResultMessageSummary in project graylog2-server by Graylog2.

the class ESMessageList method doExtractResult.

@Override
public SearchType.Result doExtractResult(SearchJob job, Query query, MessageList searchType, SearchResult result, MetricAggregation aggregations, ESGeneratedQueryContext queryContext) {
    // noinspection unchecked
    final List<ResultMessageSummary> messages = result.getHits(Map.class, false).stream().map(hit -> ResultMessage.parseFromSource(hit.id, hit.index, (Map<String, Object>) hit.source, hit.highlight)).map((resultMessage) -> ResultMessageSummary.create(resultMessage.highlightRanges, resultMessage.getMessage().getFields(), resultMessage.getIndex())).collect(Collectors.toList());
    final String undecoratedQueryString = query.query().queryString();
    final String queryString = this.esQueryDecorators.decorate(undecoratedQueryString, job, query);
    final DateTime from = query.effectiveTimeRange(searchType).getFrom();
    final DateTime to = query.effectiveTimeRange(searchType).getTo();
    final SearchResponse searchResponse = SearchResponse.create(undecoratedQueryString, queryString, Collections.emptySet(), messages, Collections.emptySet(), 0, result.getTotal(), from, to);
    final SearchResponse decoratedSearchResponse = decoratorProcessor.decorateSearchResponse(searchResponse, searchType.decorators());
    final MessageList.Result.Builder resultBuilder = MessageList.Result.result(searchType.id()).messages(decoratedSearchResponse.messages()).effectiveTimerange(AbsoluteRange.create(from, to)).totalResults(decoratedSearchResponse.totalResults());
    return searchType.name().map(resultBuilder::name).orElse(resultBuilder).build();
}
Also used : ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) Query(org.graylog.plugins.views.search.Query) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse) FieldSortBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.sort.FieldSortBuilder) SortOrder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.sort.SortOrder) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) Inject(javax.inject.Inject) SortBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.sort.SortBuilders) ResultMessage(org.graylog2.indexer.results.ResultMessage) SearchType(org.graylog.plugins.views.search.SearchType) Sort(org.graylog.plugins.views.search.searchtypes.Sort) Map(java.util.Map) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) LegacyDecoratorProcessor(org.graylog.plugins.views.search.LegacyDecoratorProcessor) QueryBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders) HighlightBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder) SearchJob(org.graylog.plugins.views.search.SearchJob) DateTime(org.joda.time.DateTime) QueryStringQueryBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryStringQueryBuilder) Set(java.util.Set) Collectors(java.util.stream.Collectors) List(java.util.List) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) Optional(java.util.Optional) Named(com.google.inject.name.Named) MoreObjects.firstNonNull(com.google.common.base.MoreObjects.firstNonNull) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Message(org.graylog2.plugin.Message) Collections(java.util.Collections) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(io.searchbox.core.SearchResult) Map(java.util.Map) DateTime(org.joda.time.DateTime) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse) SearchResult(io.searchbox.core.SearchResult)

Example 14 with ResultMessageSummary

use of org.graylog2.rest.models.messages.responses.ResultMessageSummary in project graylog2-server by Graylog2.

the class ESMessageList method doExtractResult.

@Override
public SearchType.Result doExtractResult(SearchJob job, Query query, MessageList searchType, org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse result, Aggregations aggregations, ESGeneratedQueryContext queryContext) {
    final List<ResultMessageSummary> messages = StreamSupport.stream(result.getHits().spliterator(), false).map(ESMessageList::resultMessageFromSearchHit).map((resultMessage) -> ResultMessageSummary.create(resultMessage.highlightRanges, resultMessage.getMessage().getFields(), resultMessage.getIndex())).collect(Collectors.toList());
    final String undecoratedQueryString = query.query().queryString();
    final String queryString = this.esQueryDecorators.decorate(undecoratedQueryString, job, query);
    final DateTime from = query.effectiveTimeRange(searchType).getFrom();
    final DateTime to = query.effectiveTimeRange(searchType).getTo();
    final SearchResponse searchResponse = SearchResponse.create(undecoratedQueryString, queryString, Collections.emptySet(), messages, Collections.emptySet(), 0, result.getHits().getTotalHits().value, from, to);
    final SearchResponse decoratedSearchResponse = decoratorProcessor.decorateSearchResponse(searchResponse, searchType.decorators());
    final MessageList.Result.Builder resultBuilder = MessageList.Result.result(searchType.id()).messages(decoratedSearchResponse.messages()).effectiveTimerange(AbsoluteRange.create(from, to)).totalResults(decoratedSearchResponse.totalResults());
    return searchType.name().map(resultBuilder::name).orElse(resultBuilder).build();
}
Also used : Arrays(java.util.Arrays) Query(org.graylog.plugins.views.search.Query) Text(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.text.Text) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) Inject(javax.inject.Inject) ResultMessage(org.graylog2.indexer.results.ResultMessage) SearchType(org.graylog.plugins.views.search.SearchType) Sort(org.graylog.plugins.views.search.searchtypes.Sort) Map(java.util.Map) ESGeneratedQueryContext(org.graylog.storage.elasticsearch7.views.ESGeneratedQueryContext) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) StreamSupport(java.util.stream.StreamSupport) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) QueryStringQueryBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.query.QueryStringQueryBuilder) LegacyDecoratorProcessor(org.graylog.plugins.views.search.LegacyDecoratorProcessor) SortBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.sort.SortBuilders) FieldSortBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.sort.FieldSortBuilder) SearchJob(org.graylog.plugins.views.search.SearchJob) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) DateTime(org.joda.time.DateTime) Set(java.util.Set) Collectors(java.util.stream.Collectors) SortOrder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.sort.SortOrder) Aggregations(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregations) HighlightBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder) List(java.util.List) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) SearchHit(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHit) Optional(java.util.Optional) Named(com.google.inject.name.Named) MoreObjects.firstNonNull(com.google.common.base.MoreObjects.firstNonNull) VisibleForTesting(com.google.common.annotations.VisibleForTesting) HighlightField(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.fetch.subphase.highlight.HighlightField) QueryBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.query.QueryBuilders) Message(org.graylog2.plugin.Message) Collections(java.util.Collections) DateTime(org.joda.time.DateTime) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse)

Aggregations

ResultMessageSummary (org.graylog2.rest.models.messages.responses.ResultMessageSummary)13 SearchResponse (org.graylog2.rest.resources.search.responses.SearchResponse)7 Message (org.graylog2.plugin.Message)6 List (java.util.List)4 Map (java.util.Map)4 Optional (java.util.Optional)4 Set (java.util.Set)4 Collectors (java.util.stream.Collectors)4 Inject (javax.inject.Inject)4 DecorationStats (org.graylog2.rest.models.messages.responses.DecorationStats)4 HashSet (java.util.HashSet)3 LegacyDecoratorProcessor (org.graylog.plugins.views.search.LegacyDecoratorProcessor)3 AbsoluteRange (org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange)3 IndexRangeSummary (org.graylog2.rest.models.system.indexer.responses.IndexRangeSummary)3 SearchDecorationStats (org.graylog2.rest.resources.search.responses.SearchDecorationStats)3 DateTime (org.joda.time.DateTime)3 VisibleForTesting (com.google.common.annotations.VisibleForTesting)2 MoreObjects.firstNonNull (com.google.common.base.MoreObjects.firstNonNull)2 Named (com.google.inject.name.Named)2 ArrayList (java.util.ArrayList)2