Search in sources :

Example 61 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project graylog2-server by Graylog2.

the class ElasticsearchBackendErrorHandlingTest method setUp.

@Before
public void setUp() throws Exception {
    final FieldTypesLookup fieldTypesLookup = mock(FieldTypesLookup.class);
    this.backend = new ElasticsearchBackend(ImmutableMap.of("dummy", () -> mock(DummyHandler.class)), jestClient, indexLookup, new QueryStringDecorators(Collections.emptySet()), (elasticsearchBackend, ssb, job, query) -> new ESGeneratedQueryContext(elasticsearchBackend, ssb, job, query, fieldTypesLookup), false, objectMapper);
    when(indexLookup.indexNamesForStreamsInTimeRange(any(), any())).thenReturn(Collections.emptySet());
    final SearchType searchType1 = mock(SearchType.class);
    when(searchType1.id()).thenReturn("deadbeef");
    when(searchType1.type()).thenReturn("dummy");
    final SearchType searchType2 = mock(SearchType.class);
    when(searchType2.id()).thenReturn("cafeaffe");
    when(searchType2.type()).thenReturn("dummy");
    final Set<SearchType> searchTypes = ImmutableSet.of(searchType1, searchType2);
    this.query = Query.builder().id("query1").timerange(RelativeRange.create(300)).query(ElasticsearchQueryString.of("*")).searchTypes(searchTypes).build();
    final Search search = Search.builder().id("search1").queries(ImmutableSet.of(query)).build();
    this.searchJob = new SearchJob("job1", search, "admin");
    this.queryContext = new ESGeneratedQueryContext(this.backend, new SearchSourceBuilder(), searchJob, query, mock(FieldTypesLookup.class));
    searchTypes.forEach(queryContext::searchSourceBuilder);
    when(jestClient.execute(any())).thenReturn(result);
}
Also used : ESSearchTypeHandler(org.graylog.storage.elasticsearch6.views.searchtypes.ESSearchTypeHandler) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Query(org.graylog.plugins.views.search.Query) Mock(org.mockito.Mock) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) MultiSearchResult(io.searchbox.core.MultiSearchResult) ElasticsearchException(org.graylog2.indexer.ElasticsearchException) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) JestClient(io.searchbox.client.JestClient) ElasticsearchQueryString(org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString) SearchType(org.graylog.plugins.views.search.SearchType) Assertions.assertThatExceptionOfType(org.assertj.core.api.Assertions.assertThatExceptionOfType) FieldTypesLookup(org.graylog.plugins.views.search.elasticsearch.FieldTypesLookup) MockitoJUnit(org.mockito.junit.MockitoJUnit) JsonNode(com.fasterxml.jackson.databind.JsonNode) Search(org.graylog.plugins.views.search.Search) QueryResult(org.graylog.plugins.views.search.QueryResult) Before(org.junit.Before) SearchJob(org.graylog.plugins.views.search.SearchJob) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) Test(org.junit.Test) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) Collectors(java.util.stream.Collectors) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) Rule(org.junit.Rule) SearchError(org.graylog.plugins.views.search.errors.SearchError) MockitoRule(org.mockito.junit.MockitoRule) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) Collections(java.util.Collections) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) Mockito.mock(org.mockito.Mockito.mock) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) Search(org.graylog.plugins.views.search.Search) SearchJob(org.graylog.plugins.views.search.SearchJob) FieldTypesLookup(org.graylog.plugins.views.search.elasticsearch.FieldTypesLookup) SearchType(org.graylog.plugins.views.search.SearchType) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) Before(org.junit.Before)

Example 62 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project graylog2-server by Graylog2.

the class CountsAdapterES7 method totalCount.

@Override
public long totalCount(List<String> indices) {
    final SearchSourceBuilder query = new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()).size(0).trackTotalHits(true);
    final SearchRequest searchRequest = new SearchRequest(indices.toArray(new String[0])).source(query);
    final SearchResponse result = client.search(searchRequest, "Fetching message count failed for indices ");
    return result.getHits().getTotalHits().value;
}
Also used : SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse)

Example 63 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project graylog2-server by Graylog2.

the class QuerySuggestionsES7 method suggest.

@Override
public SuggestionResponse suggest(SuggestionRequest req) {
    final Set<String> affectedIndices = indexLookup.indexNamesForStreamsInTimeRange(req.streams(), req.timerange());
    final TermSuggestionBuilder suggestionBuilder = SuggestBuilders.termSuggestion(req.field()).text(req.input()).size(req.size());
    final SearchSourceBuilder search = new SearchSourceBuilder().query(QueryBuilders.prefixQuery(req.field(), req.input())).size(0).aggregation(AggregationBuilders.terms("fieldvalues").field(req.field()).size(req.size())).suggest(new SuggestBuilder().addSuggestion("corrections", suggestionBuilder));
    try {
        final SearchResponse result = client.singleSearch(new SearchRequest(affectedIndices.toArray(new String[] {})).source(search), "Failed to execute aggregation");
        final ParsedStringTerms fieldValues = result.getAggregations().get("fieldvalues");
        final List<SuggestionEntry> entries = fieldValues.getBuckets().stream().map(b -> new SuggestionEntry(b.getKeyAsString(), b.getDocCount())).collect(Collectors.toList());
        if (!entries.isEmpty()) {
            return SuggestionResponse.forSuggestions(req.field(), req.input(), entries, fieldValues.getSumOfOtherDocCounts());
        } else {
            TermSuggestion suggestion = result.getSuggest().getSuggestion("corrections");
            final List<SuggestionEntry> corrections = suggestion.getEntries().stream().flatMap(e -> e.getOptions().stream()).map(o -> new SuggestionEntry(o.getText().string(), o.getFreq())).collect(Collectors.toList());
            return SuggestionResponse.forSuggestions(req.field(), req.input(), corrections, null);
        }
    } catch (org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchException exception) {
        final SuggestionError err = tryResponseException(exception).orElseGet(() -> parseException(exception));
        return SuggestionResponse.forError(req.field(), req.input(), err);
    }
}
Also used : TermSuggestionBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestionBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) ParsedStringTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedStringTerms) SuggestBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) Set(java.util.Set) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) SuggestionRequest(org.graylog.plugins.views.search.engine.suggestions.SuggestionRequest) Collectors(java.util.stream.Collectors) AggregationBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilders) Inject(javax.inject.Inject) List(java.util.List) QuerySuggestionsService(org.graylog.plugins.views.search.engine.QuerySuggestionsService) ResponseError(org.graylog.storage.elasticsearch7.errors.ResponseError) Optional(java.util.Optional) SuggestBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilders) SuggestionResponse(org.graylog.plugins.views.search.engine.suggestions.SuggestionResponse) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) QueryBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.query.QueryBuilders) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) ParsedStringTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedStringTerms) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) SuggestBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.SuggestBuilder) SuggestionError(org.graylog.plugins.views.search.engine.suggestions.SuggestionError) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) TermSuggestionBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestionBuilder) SuggestionEntry(org.graylog.plugins.views.search.engine.suggestions.SuggestionEntry)

Example 64 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project graylog2-server by Graylog2.

the class Scroll method scroll.

public ScrollResult scroll(ScrollCommand scrollCommand) {
    final SearchSourceBuilder searchQuery = searchRequestFactory.create(scrollCommand);
    searchQuery.fetchSource(scrollCommand.fields().toArray(new String[0]), new String[0]);
    scrollCommand.batchSize().ifPresent(batchSize -> searchQuery.size(Math.toIntExact(batchSize)));
    final SearchRequest request = scrollBuilder(searchQuery, scrollCommand.indices());
    final SearchResponse result = client.singleSearch(request, "Unable to perform scroll search");
    return scrollResultFactory.create(result, searchQuery.toString(), DEFAULT_SCROLLTIME, scrollCommand.fields(), scrollCommand.limit().orElse(-1));
}
Also used : SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse)

Example 65 with SearchSourceBuilder

use of org.elasticsearch.search.builder.SearchSourceBuilder in project graylog2-server by Graylog2.

the class SearchesAdapterES7 method count.

@Override
public CountResult count(Set<String> affectedIndices, String query, TimeRange range, String filter) {
    final SearchesConfig config = SearchesConfig.builder().query(query).range(range).filter(filter).limit(0).offset(0).build();
    final SearchSourceBuilder searchSourceBuilder = searchRequestFactory.create(config);
    final SearchRequest searchRequest = new SearchRequest(affectedIndices.toArray(new String[0])).source(searchSourceBuilder);
    final SearchResponse result = client.search(searchRequest, "Fetching message count failed for indices ");
    return CountResult.create(result.getHits().getTotalHits().value, result.getTook().getMillis());
}
Also used : SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) SearchesConfig(org.graylog2.indexer.searches.SearchesConfig) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse)

Aggregations

SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)152 SearchResponse (org.elasticsearch.action.search.SearchResponse)77 SearchRequest (org.elasticsearch.action.search.SearchRequest)52 ElasticsearchAssertions.assertSearchResponse (org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse)37 Matchers.containsString (org.hamcrest.Matchers.containsString)32 IOException (java.io.IOException)31 QueryBuilder (org.elasticsearch.index.query.QueryBuilder)28 BoolQueryBuilder (org.elasticsearch.index.query.BoolQueryBuilder)26 List (java.util.List)25 SearchSourceBuilder (org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder)24 SearchSourceBuilder (org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder)24 Search (io.searchbox.core.Search)23 Map (java.util.Map)23 Set (java.util.Set)19 ArrayList (java.util.ArrayList)18 Collectors (java.util.stream.Collectors)18 Test (org.junit.Test)18 Collections (java.util.Collections)15 SearchHit (org.elasticsearch.search.SearchHit)15 TermsAggregationBuilder (org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder)15