Search in sources :

Example 6 with ESGeneratedQueryContext

use of org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext in project graylog2-server by Graylog2.

the class ElasticsearchBackendErrorHandlingTest method setUp.

@Before
public void setUp() throws Exception {
    final FieldTypesLookup fieldTypesLookup = mock(FieldTypesLookup.class);
    this.backend = new ElasticsearchBackend(ImmutableMap.of("dummy", () -> mock(DummyHandler.class)), jestClient, indexLookup, new QueryStringDecorators(Collections.emptySet()), (elasticsearchBackend, ssb, job, query) -> new ESGeneratedQueryContext(elasticsearchBackend, ssb, job, query, fieldTypesLookup), false, objectMapper);
    when(indexLookup.indexNamesForStreamsInTimeRange(any(), any())).thenReturn(Collections.emptySet());
    final SearchType searchType1 = mock(SearchType.class);
    when(searchType1.id()).thenReturn("deadbeef");
    when(searchType1.type()).thenReturn("dummy");
    final SearchType searchType2 = mock(SearchType.class);
    when(searchType2.id()).thenReturn("cafeaffe");
    when(searchType2.type()).thenReturn("dummy");
    final Set<SearchType> searchTypes = ImmutableSet.of(searchType1, searchType2);
    this.query = Query.builder().id("query1").timerange(RelativeRange.create(300)).query(ElasticsearchQueryString.of("*")).searchTypes(searchTypes).build();
    final Search search = Search.builder().id("search1").queries(ImmutableSet.of(query)).build();
    this.searchJob = new SearchJob("job1", search, "admin");
    this.queryContext = new ESGeneratedQueryContext(this.backend, new SearchSourceBuilder(), searchJob, query, mock(FieldTypesLookup.class));
    searchTypes.forEach(queryContext::searchSourceBuilder);
    when(jestClient.execute(any())).thenReturn(result);
}
Also used : ESSearchTypeHandler(org.graylog.storage.elasticsearch6.views.searchtypes.ESSearchTypeHandler) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) Query(org.graylog.plugins.views.search.Query) Mock(org.mockito.Mock) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) MultiSearchResult(io.searchbox.core.MultiSearchResult) ElasticsearchException(org.graylog2.indexer.ElasticsearchException) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) JestClient(io.searchbox.client.JestClient) ElasticsearchQueryString(org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString) SearchType(org.graylog.plugins.views.search.SearchType) Assertions.assertThatExceptionOfType(org.assertj.core.api.Assertions.assertThatExceptionOfType) FieldTypesLookup(org.graylog.plugins.views.search.elasticsearch.FieldTypesLookup) MockitoJUnit(org.mockito.junit.MockitoJUnit) JsonNode(com.fasterxml.jackson.databind.JsonNode) Search(org.graylog.plugins.views.search.Search) QueryResult(org.graylog.plugins.views.search.QueryResult) Before(org.junit.Before) SearchJob(org.graylog.plugins.views.search.SearchJob) ImmutableSet(com.google.common.collect.ImmutableSet) ImmutableMap(com.google.common.collect.ImmutableMap) Set(java.util.Set) Test(org.junit.Test) IOException(java.io.IOException) Mockito.when(org.mockito.Mockito.when) Collectors(java.util.stream.Collectors) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) Rule(org.junit.Rule) SearchError(org.graylog.plugins.views.search.errors.SearchError) MockitoRule(org.mockito.junit.MockitoRule) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) Collections(java.util.Collections) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) Mockito.mock(org.mockito.Mockito.mock) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) Search(org.graylog.plugins.views.search.Search) SearchJob(org.graylog.plugins.views.search.SearchJob) FieldTypesLookup(org.graylog.plugins.views.search.elasticsearch.FieldTypesLookup) SearchType(org.graylog.plugins.views.search.SearchType) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) Before(org.junit.Before)

Example 7 with ESGeneratedQueryContext

use of org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext in project graylog2-server by Graylog2.

the class ESMessageListTest method passesNullForUnmappedTypeIfTypeIsNotFound.

@Test
public void passesNullForUnmappedTypeIfTypeIsNotFound() {
    final MessageList messageList = someMessageListWithSorting("stream1", "somefield");
    final ESGeneratedQueryContext context = mockQueryContext(messageList);
    when(context.fieldType(Collections.singleton("stream1"), "somefield")).thenReturn(Optional.empty());
    final ESGeneratedQueryContext queryContext = generateQueryPartWithContextFor(messageList, true, Collections.emptySet(), context);
    final DocumentContext doc = JsonPath.parse(queryContext.searchSourceBuilder(messageList).toString());
    assertThat(doc.read("$.sort[0].somefield", Map.class)).doesNotContainKey("unmapped_type");
}
Also used : ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) DocumentContext(com.jayway.jsonpath.DocumentContext) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) Test(org.junit.Test)

Example 8 with ESGeneratedQueryContext

use of org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext in project graylog2-server by Graylog2.

the class ESMessageListTest method doesNotUseHighlightingIfDeactivatedInConfig.

@Test
public void doesNotUseHighlightingIfDeactivatedInConfig() {
    MessageList messageList = someMessageList();
    ESGeneratedQueryContext context = generateQueryPartWithoutHighlighting(messageList);
    assertThat(context.searchSourceBuilder(messageList).highlighter()).as("there should be no highlighter configured").isNull();
}
Also used : ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) Test(org.junit.Test)

Example 9 with ESGeneratedQueryContext

use of org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext in project graylog2-server by Graylog2.

the class ESEventListTest method testSortingOfStreamsInDoExtractResult.

@Test
public void testSortingOfStreamsInDoExtractResult() {
    final ESEventList esEventList = new TestESEventList();
    final SearchJob searchJob = mock(SearchJob.class);
    final Query query = mock(Query.class);
    final SearchResult searchResult = mock(SearchResult.class);
    final MetricAggregation metricAggregation = mock(MetricAggregation.class);
    final ESGeneratedQueryContext queryContext = mock(ESGeneratedQueryContext.class);
    final EventList eventList = EventList.builder().id("search-type-id").streams(ImmutableSet.of("stream-id-1", "stream-id-2")).build();
    final EventList.Result eventResult = (EventList.Result) esEventList.doExtractResult(searchJob, query, eventList, searchResult, metricAggregation, queryContext);
    assertThat(eventResult.events()).containsExactly(eventSummary("find-1", ImmutableSet.of("stream-id-1")), eventSummary("find-2", ImmutableSet.of("stream-id-2")), eventSummary("find-3", ImmutableSet.of("stream-id-1", "stream-id-2")));
}
Also used : Query(org.graylog.plugins.views.search.Query) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) ESEventList(org.graylog.storage.elasticsearch6.views.searchtypes.ESEventList) EventList(org.graylog.plugins.views.search.searchtypes.events.EventList) SearchJob(org.graylog.plugins.views.search.SearchJob) SearchResult(io.searchbox.core.SearchResult) ESEventList(org.graylog.storage.elasticsearch6.views.searchtypes.ESEventList) ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) SearchResult(io.searchbox.core.SearchResult) Test(org.junit.Test)

Example 10 with ESGeneratedQueryContext

use of org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext in project graylog2-server by Graylog2.

the class ESPivot method processSeries.

private void processSeries(PivotResult.Row.Builder rowBuilder, SearchResult searchResult, ESGeneratedQueryContext queryContext, Pivot pivot, ArrayDeque<String> columnKeys, MetricAggregation aggregation, boolean rollup, String source) {
    pivot.series().forEach(seriesSpec -> {
        final ESPivotSeriesSpecHandler<? extends SeriesSpec, ? extends Aggregation> seriesHandler = seriesHandlers.get(seriesSpec.type());
        final Aggregation series = seriesHandler.extractAggregationFromResult(pivot, seriesSpec, aggregation, queryContext);
        seriesHandler.handleResult(pivot, seriesSpec, searchResult, series, this, queryContext).map(value -> {
            columnKeys.addLast(value.id());
            final PivotResult.Value v = PivotResult.Value.create(columnKeys, value.value(), rollup, source);
            columnKeys.removeLast();
            return v;
        }).forEach(rowBuilder::addValue);
    });
}
Also used : Aggregation(io.searchbox.core.search.aggregation.Aggregation) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) ESSearchTypeHandler(org.graylog.storage.elasticsearch6.views.searchtypes.ESSearchTypeHandler) MinAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.metrics.min.MinAggregationBuilder) DateTimeZone(org.joda.time.DateTimeZone) InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Query(org.graylog.plugins.views.search.Query) PivotResult(org.graylog.plugins.views.search.searchtypes.pivot.PivotResult) LoggerFactory(org.slf4j.LoggerFactory) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) Inject(javax.inject.Inject) Tuple2(org.jooq.lambda.tuple.Tuple2) ImmutableList(com.google.common.collect.ImmutableList) SearchType(org.graylog.plugins.views.search.SearchType) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) Map(java.util.Map) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) MaxAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.metrics.max.MaxAggregationBuilder) AggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilder) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) PivotSpec(org.graylog.plugins.views.search.searchtypes.pivot.PivotSpec) SearchJob(org.graylog.plugins.views.search.SearchJob) Logger(org.slf4j.Logger) IdentityHashMap(java.util.IdentityHashMap) Iterator(java.util.Iterator) DateTime(org.joda.time.DateTime) Aggregation(io.searchbox.core.search.aggregation.Aggregation) EntryStream(one.util.streamex.EntryStream) List(java.util.List) Tuple(org.jooq.lambda.tuple.Tuple) Stream(java.util.stream.Stream) AggregationBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilders) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) ArrayDeque(java.util.ArrayDeque) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(io.searchbox.core.SearchResult)

Aggregations

Test (org.junit.Test)15 MessageList (org.graylog.plugins.views.search.searchtypes.MessageList)14 Query (org.graylog.plugins.views.search.Query)13 SearchJob (org.graylog.plugins.views.search.SearchJob)11 SearchType (org.graylog.plugins.views.search.SearchType)10 ESGeneratedQueryContext (org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext)9 ESGeneratedQueryContext (org.graylog.storage.elasticsearch7.views.ESGeneratedQueryContext)9 Map (java.util.Map)8 QueryStringDecorators (org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators)7 ESSearchTypeHandler (org.graylog.storage.elasticsearch6.views.searchtypes.ESSearchTypeHandler)7 DocumentContext (com.jayway.jsonpath.DocumentContext)6 SearchSourceBuilder (org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder)6 Collections (java.util.Collections)5 List (java.util.List)5 Optional (java.util.Optional)5 Set (java.util.Set)5 ElasticsearchQueryString (org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString)5 IndexLookup (org.graylog.plugins.views.search.elasticsearch.IndexLookup)5 ImmutableSet (com.google.common.collect.ImmutableSet)4 Collectors (java.util.stream.Collectors)4