Search in sources :

Example 31 with IndexRange

use of org.graylog2.indexer.ranges.IndexRange in project graylog2-server by Graylog2.

the class IndexLookupTest method mockIndexRange.

private IndexRange mockIndexRange(String name) {
    final IndexRange indexRange1 = mock(IndexRange.class);
    when(indexRange1.indexName()).thenReturn(name);
    return indexRange1;
}
Also used : IndexRange(org.graylog2.indexer.ranges.IndexRange)

Example 32 with IndexRange

use of org.graylog2.indexer.ranges.IndexRange in project graylog2-server by Graylog2.

the class SearchesAdapterES6 method search.

@Override
public SearchResult search(Set<String> indices, Set<IndexRange> indexRanges, SearchesConfig config) {
    final SearchSourceBuilder requestBuilder = searchRequest(config);
    if (indexRanges.isEmpty()) {
        return SearchResult.empty(config.query(), requestBuilder.toString());
    }
    final Search.Builder searchBuilder = new Search.Builder(requestBuilder.toString()).addType(IndexMapping.TYPE_MESSAGE).addIndex(indices);
    final io.searchbox.core.SearchResult searchResult = multiSearch.wrap(searchBuilder.build(), () -> "Unable to perform search query");
    final List<ResultMessage> hits = searchResult.getHits(Map.class, false).stream().map(hit -> ResultMessage.parseFromSource(hit.id, hit.index, (Map<String, Object>) hit.source, hit.highlight)).collect(Collectors.toList());
    return new SearchResult(hits, searchResult.getTotal(), indexRanges, config.query(), requestBuilder.toString(), multiSearch.tookMsFromSearchResult(searchResult));
}
Also used : FilterAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) NO_LIMIT(org.graylog2.indexer.searches.ScrollCommand.NO_LIMIT) FieldStatsResult(org.graylog2.indexer.results.FieldStatsResult) SearchesConfig(org.graylog2.indexer.searches.SearchesConfig) Strings.isNullOrEmpty(com.google.common.base.Strings.isNullOrEmpty) ScrollResult(org.graylog2.indexer.results.ScrollResult) SearchesAdapter(org.graylog2.indexer.searches.SearchesAdapter) QueryBuilders.existsQuery(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders.existsQuery) Inject(javax.inject.Inject) ScrollCommand(org.graylog2.indexer.searches.ScrollCommand) ResultMessage(org.graylog2.indexer.results.ResultMessage) Map(java.util.Map) CardinalityAggregation(io.searchbox.core.search.aggregation.CardinalityAggregation) IndexMapping(org.graylog2.indexer.IndexMapping) SearchResult(org.graylog2.indexer.results.SearchResult) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) QueryBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilder) Nullable(javax.annotation.Nullable) QueryBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders) HighlightBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder) Sorting(org.graylog2.indexer.searches.Sorting) NO_BATCHSIZE(org.graylog2.indexer.searches.ScrollCommand.NO_BATCHSIZE) Search(io.searchbox.core.Search) Set(java.util.Set) QueryBuilders.matchAllQuery(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders.matchAllQuery) QueryBuilders.termsQuery(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders.termsQuery) ValueCountAggregation(io.searchbox.core.search.aggregation.ValueCountAggregation) Collectors(java.util.stream.Collectors) ExtendedStatsAggregation(io.searchbox.core.search.aggregation.ExtendedStatsAggregation) Strings(org.graylog.shaded.elasticsearch6.org.elasticsearch.common.Strings) IndexRange(org.graylog2.indexer.ranges.IndexRange) CountResult(org.graylog2.indexer.results.CountResult) QueryBuilders.queryStringQuery(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders.queryStringQuery) List(java.util.List) Parameters(io.searchbox.params.Parameters) Configuration(org.graylog2.Configuration) Stream(org.graylog2.plugin.streams.Stream) AggregationBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilders) Optional(java.util.Optional) BoolQueryBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder) VisibleForTesting(com.google.common.annotations.VisibleForTesting) Message(org.graylog2.plugin.Message) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) QueryBuilders.boolQuery(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilders.boolQuery) FilterAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) QueryBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.QueryBuilder) HighlightBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder) BoolQueryBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.index.query.BoolQueryBuilder) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(org.graylog2.indexer.results.SearchResult) ResultMessage(org.graylog2.indexer.results.ResultMessage) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) Search(io.searchbox.core.Search) Map(java.util.Map)

Example 33 with IndexRange

use of org.graylog2.indexer.ranges.IndexRange in project graylog2-server by Graylog2.

the class SearchesAdapterES7 method search.

@Override
public SearchResult search(Set<String> indices, Set<IndexRange> indexRanges, SearchesConfig config) {
    final SearchSourceBuilder searchSourceBuilder = searchRequestFactory.create(config);
    if (indexRanges.isEmpty()) {
        return SearchResult.empty(config.query(), searchSourceBuilder.toString());
    }
    final SearchRequest searchRequest = new SearchRequest(indices.toArray(new String[0])).source(searchSourceBuilder);
    final SearchResponse searchResult = client.search(searchRequest, "Unable to perform search query");
    final List<ResultMessage> resultMessages = extractResultMessages(searchResult);
    final long totalResults = searchResult.getHits().getTotalHits().value;
    final long tookMs = searchResult.getTook().getMillis();
    final String builtQuery = searchSourceBuilder.toString();
    return new SearchResult(resultMessages, totalResults, indexRanges, config.query(), builtQuery, tookMs);
}
Also used : SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) SearchResult(org.graylog2.indexer.results.SearchResult) ResultMessage(org.graylog2.indexer.results.ResultMessage) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse)

Example 34 with IndexRange

use of org.graylog2.indexer.ranges.IndexRange in project graylog2-server by Graylog2.

the class MoreSearch method getAffectedIndices.

private Set<String> getAffectedIndices(Set<String> streamIds, TimeRange timeRange) {
    final SortedSet<IndexRange> indexRanges = indexRangeService.find(timeRange.getFrom(), timeRange.getTo());
    // We support an empty streams list and return all affected indices in that case.
    if (streamIds.isEmpty()) {
        return indexRanges.stream().map(IndexRange::indexName).collect(Collectors.toSet());
    } else {
        final Set<Stream> streams = loadStreams(streamIds);
        final IndexRangeContainsOneOfStreams indexRangeContainsOneOfStreams = new IndexRangeContainsOneOfStreams(streams);
        return indexRanges.stream().filter(indexRangeContainsOneOfStreams).map(IndexRange::indexName).collect(Collectors.toSet());
    }
}
Also used : IndexRange(org.graylog2.indexer.ranges.IndexRange) Stream(org.graylog2.plugin.streams.Stream) IndexRangeContainsOneOfStreams(org.graylog.plugins.views.search.IndexRangeContainsOneOfStreams)

Aggregations

IndexRange (org.graylog2.indexer.ranges.IndexRange)29 DateTime (org.joda.time.DateTime)18 Test (org.junit.Test)17 MongoIndexRange (org.graylog2.indexer.ranges.MongoIndexRange)15 ZonedDateTime (java.time.ZonedDateTime)14 TimeRange (org.graylog2.plugin.indexer.searches.timeranges.TimeRange)13 IndexSet (org.graylog2.indexer.IndexSet)7 SearchResult (org.graylog2.indexer.results.SearchResult)7 ElasticsearchBaseTest (org.graylog.testing.elasticsearch.ElasticsearchBaseTest)6 Stream (org.graylog2.plugin.streams.Stream)6 Set (java.util.Set)4 Collectors (java.util.stream.Collectors)4 ImmutableSortedSet (com.google.common.collect.ImmutableSortedSet)3 List (java.util.List)3 ResultMessage (org.graylog2.indexer.results.ResultMessage)3 Sorting (org.graylog2.indexer.searches.Sorting)3 Histogram (com.codahale.metrics.Histogram)2 MetricRegistry (com.codahale.metrics.MetricRegistry)2 Timer (com.codahale.metrics.Timer)2 Timed (com.codahale.metrics.annotation.Timed)2