Search in sources :

Example 21 with SearchJob

use of org.graylog.plugins.views.search.SearchJob in project graylog2-server by Graylog2.

the class SearchExecutorTest method appliesSearchExecutionState.

@Test
public void appliesSearchExecutionState() {
    final Search search = mockSearch();
    final SearchUser searchUser = mock(SearchUser.class);
    when(searchUser.canReadView(any())).thenReturn(true);
    when(searchUser.canReadStream(any())).thenReturn(true);
    when(searchUser.username()).thenReturn("frank-drebin");
    final SearchJob searchJob = mock(SearchJob.class);
    when(searchJobService.create(search, "frank-drebin")).thenReturn(searchJob);
    when(searchJob.getResultFuture()).thenReturn(CompletableFuture.completedFuture(null));
    when(queryEngine.execute(searchJob)).thenReturn(searchJob);
    when(searchDomain.getForUser(eq("search1"), eq(searchUser))).thenReturn(Optional.of(search));
    final ExecutionState executionState = ExecutionState.builder().addAdditionalParameter("foo", 42).build();
    this.searchExecutor.execute("search1", searchUser, executionState);
    verify(search, times(1)).applyExecutionState(any(), executionStateCaptor.capture());
    assertThat(executionStateCaptor.getValue()).isEqualTo(executionState);
}
Also used : Search(org.graylog.plugins.views.search.Search) SearchUser(org.graylog.plugins.views.search.permissions.SearchUser) SearchJob(org.graylog.plugins.views.search.SearchJob) RestResourceBaseTest(org.graylog2.rest.resources.RestResourceBaseTest) Test(org.junit.Test)

Example 22 with SearchJob

use of org.graylog.plugins.views.search.SearchJob in project graylog2-server by Graylog2.

the class PivotAggregationSearch method getSearchJob.

private SearchJob getSearchJob(AggregationEventProcessorParameters parameters, String username, long searchWithinMs, long executeEveryMs) throws EventProcessorException {
    Search search = Search.builder().queries(ImmutableSet.of(getAggregationQuery(parameters, searchWithinMs, executeEveryMs), getSourceStreamsQuery(parameters))).parameters(config.queryParameters()).build();
    // This adds all streams if none were provided
    // TODO: Once we introduce "EventProcessor owners" this should only load the permitted streams of the
    // user who created this EventProcessor.
    search = search.addStreamsToQueriesWithoutStreams(() -> permittedStreams.load((streamId) -> true));
    final SearchJob searchJob = queryEngine.execute(searchJobService.create(search, username));
    try {
        Uninterruptibles.getUninterruptibly(searchJob.getResultFuture(), configurationProvider.get().eventsSearchTimeout(), TimeUnit.MILLISECONDS);
    } catch (ExecutionException e) {
        throw new EventProcessorException("Error executing search job: " + e.getMessage(), false, eventDefinition, e);
    } catch (TimeoutException e) {
        throw new EventProcessorException("Timeout while executing search job.", false, eventDefinition, e);
    } catch (Exception e) {
        throw new EventProcessorException("Unhandled exception in search job.", false, eventDefinition, e);
    }
    return searchJob;
}
Also used : EventProcessorException(org.graylog.events.processor.EventProcessorException) MoreSearch(org.graylog.events.search.MoreSearch) Search(org.graylog.plugins.views.search.Search) SearchJob(org.graylog.plugins.views.search.SearchJob) ExecutionException(java.util.concurrent.ExecutionException) TimeoutException(java.util.concurrent.TimeoutException) EventProcessorException(org.graylog.events.processor.EventProcessorException) ExecutionException(java.util.concurrent.ExecutionException) TimeoutException(java.util.concurrent.TimeoutException)

Example 23 with SearchJob

use of org.graylog.plugins.views.search.SearchJob in project graylog2-server by Graylog2.

the class SearchResource method executeSyncJobv2.

@POST
@ApiOperation(value = "Execute a new synchronous search", notes = "Executes a new search and waits for its result", response = SearchJobDTO.class)
@Path("sync")
@NoAuditEvent("Creating audit event manually in method body.")
@Consumes({ SEARCH_FORMAT_V2 })
@Produces({ SEARCH_FORMAT_V2 })
public Response executeSyncJobv2(@ApiParam @NotNull(message = "Search body is mandatory") SearchDTOv2 searchRequest, @ApiParam(name = "timeout", defaultValue = "60000") @QueryParam("timeout") @DefaultValue("60000") long timeout, @Context SearchUser searchUser) {
    final Search search = searchRequest.toSearch();
    final SearchJob searchJob = searchExecutor.execute(search, searchUser, ExecutionState.empty());
    postAuditEvent(searchJob);
    final SearchJobDTO searchJobDTO = SearchJobDTO.fromSearchJob(searchJob);
    return Response.ok(searchJobDTO).build();
}
Also used : Search(org.graylog.plugins.views.search.Search) SearchJob(org.graylog.plugins.views.search.SearchJob) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) ApiOperation(io.swagger.annotations.ApiOperation) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent)

Example 24 with SearchJob

use of org.graylog.plugins.views.search.SearchJob in project graylog2-server by Graylog2.

the class SearchResource method executeSyncJob.

@POST
@ApiOperation(value = "Execute a new synchronous search", notes = "Executes a new search and waits for its result", response = SearchJobDTO.class)
@Path("sync")
@NoAuditEvent("Creating audit event manually in method body.")
@Consumes({ MediaType.APPLICATION_JSON, SEARCH_FORMAT_V1 })
@Produces({ MediaType.APPLICATION_JSON, SEARCH_FORMAT_V1 })
public Response executeSyncJob(@ApiParam @NotNull(message = "Search body is mandatory") SearchDTO searchRequest, @ApiParam(name = "timeout", defaultValue = "60000") @QueryParam("timeout") @DefaultValue("60000") long timeout, @Context SearchUser searchUser) {
    final Search search = searchRequest.toSearch();
    final SearchJob searchJob = searchExecutor.execute(search, searchUser, ExecutionState.empty());
    postAuditEvent(searchJob);
    final SearchJobDTO searchJobDTO = SearchJobDTO.fromSearchJob(searchJob);
    return Response.ok(searchJobDTO).build();
}
Also used : Search(org.graylog.plugins.views.search.Search) SearchJob(org.graylog.plugins.views.search.SearchJob) Path(javax.ws.rs.Path) POST(javax.ws.rs.POST) Consumes(javax.ws.rs.Consumes) Produces(javax.ws.rs.Produces) ApiOperation(io.swagger.annotations.ApiOperation) NoAuditEvent(org.graylog2.audit.jersey.NoAuditEvent)

Example 25 with SearchJob

use of org.graylog.plugins.views.search.SearchJob in project graylog2-server by Graylog2.

the class SearchResource method extractSearchResponse.

protected SearchResponse extractSearchResponse(SearchJob searchJob, String query, boolean decorate, List<String> fieldList, TimeRange timeRange, Optional<String> streamId) {
    final QueryResult queryResult = searchJob.results().values().stream().findFirst().orElseThrow(() -> new IllegalStateException("Missing query result"));
    final MessageList.Result result = queryResult.searchTypes().values().stream().findFirst().map(searchTypeResult -> (MessageList.Result) searchTypeResult).orElseThrow(() -> new IllegalStateException("Missing search type result!"));
    final long tookMs = queryResult.executionStats().duration();
    return buildSearchResponse(query, result, fieldList, tookMs, timeRange, decorate, streamId);
}
Also used : LoggerFactory(org.slf4j.LoggerFactory) SearchResponse(org.graylog2.rest.resources.search.responses.SearchResponse) Searches(org.graylog2.indexer.searches.Searches) NotEmpty(javax.validation.constraints.NotEmpty) ResultMessage(org.graylog2.indexer.results.ResultMessage) ClusterConfigService(org.graylog2.plugin.cluster.ClusterConfigService) Locale(java.util.Locale) DecoratorProcessor(org.graylog2.decorators.DecoratorProcessor) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) BadRequestException(javax.ws.rs.BadRequestException) Splitter(com.google.common.base.Splitter) SearchResult(org.graylog2.indexer.results.SearchResult) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) Sorting(org.graylog2.indexer.searches.Sorting) ImmutableSet(com.google.common.collect.ImmutableSet) Set(java.util.Set) Collectors(java.util.stream.Collectors) Sets(com.google.common.collect.Sets) List(java.util.List) ExecutionState(org.graylog.plugins.views.search.rest.ExecutionState) Optional(java.util.Optional) SearchUser(org.graylog.plugins.views.search.permissions.SearchUser) ChunkedOutput(org.glassfish.jersey.server.ChunkedOutput) Query(org.graylog.plugins.views.search.Query) IndexRangeSummary(org.graylog2.rest.models.system.indexer.responses.IndexRangeSummary) Strings.isNullOrEmpty(com.google.common.base.Strings.isNullOrEmpty) ScrollResult(org.graylog2.indexer.results.ScrollResult) ResultMessageSummary(org.graylog2.rest.models.messages.responses.ResultMessageSummary) ArrayList(java.util.ArrayList) ElasticsearchQueryString(org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString) Strings(com.google.common.base.Strings) Lists(com.google.common.collect.Lists) SearchType(org.graylog.plugins.views.search.SearchType) SearchExecutor(org.graylog.plugins.views.search.rest.SearchExecutor) QueryStringFilter(org.graylog.plugins.views.search.filter.QueryStringFilter) Sort(org.graylog.plugins.views.search.searchtypes.Sort) Search(org.graylog.plugins.views.search.Search) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) QueryResult(org.graylog.plugins.views.search.QueryResult) Period(org.joda.time.Period) SearchJob(org.graylog.plugins.views.search.SearchJob) Logger(org.slf4j.Logger) ForbiddenException(javax.ws.rs.ForbiddenException) DateTime(org.joda.time.DateTime) RestResource(org.graylog2.shared.rest.resources.RestResource) IOException(java.io.IOException) IndexRange(org.graylog2.indexer.ranges.IndexRange) SearchesClusterConfig(org.graylog2.indexer.searches.SearchesClusterConfig) RestPermissions(org.graylog2.shared.security.RestPermissions) Message(org.graylog2.plugin.Message) Collections(java.util.Collections) QueryResult(org.graylog.plugins.views.search.QueryResult) MessageList(org.graylog.plugins.views.search.searchtypes.MessageList) SearchResult(org.graylog2.indexer.results.SearchResult) ScrollResult(org.graylog2.indexer.results.ScrollResult) QueryResult(org.graylog.plugins.views.search.QueryResult)

Aggregations

SearchJob (org.graylog.plugins.views.search.SearchJob)49 Query (org.graylog.plugins.views.search.Query)24 Search (org.graylog.plugins.views.search.Search)24 Test (org.junit.Test)16 SearchType (org.graylog.plugins.views.search.SearchType)13 SearchConfig (org.graylog.plugins.views.search.engine.SearchConfig)12 QueryResult (org.graylog.plugins.views.search.QueryResult)10 Collections (java.util.Collections)9 Set (java.util.Set)9 Collectors (java.util.stream.Collectors)9 List (java.util.List)8 QueryStringDecorators (org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators)8 Optional (java.util.Optional)7 ElasticsearchQueryString (org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString)7 Message (org.graylog2.plugin.Message)7 IndexLookup (org.graylog.plugins.views.search.elasticsearch.IndexLookup)6 Named (com.google.inject.name.Named)5 MultiSearch (io.searchbox.core.MultiSearch)5 ArrayList (java.util.ArrayList)5 Map (java.util.Map)5