Search in sources :

Example 6 with SearchScrollRequest

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest in project elasticsearch by elastic.

the class ClientScrollableHitSource method doStartNextScroll.

@Override
protected void doStartNextScroll(String scrollId, TimeValue extraKeepAlive, Consumer<? super Response> onResponse) {
    searchWithRetry(listener -> {
        SearchScrollRequest request = new SearchScrollRequest();
        request.scrollId(scrollId).scroll(timeValueNanos(firstSearchRequest.scroll().keepAlive().nanos() + extraKeepAlive.nanos()));
        client.searchScroll(request, listener);
    }, r -> consume(r, onResponse));
}
Also used : SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest)

Example 7 with SearchScrollRequest

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest in project elasticsearch by elastic.

the class RestSearchScrollActionTests method testParseSearchScrollRequestWithUnknownParamThrowsException.

public void testParseSearchScrollRequestWithUnknownParamThrowsException() throws Exception {
    SearchScrollRequest searchScrollRequest = new SearchScrollRequest();
    XContentParser invalidContent = createParser(XContentFactory.jsonBuilder().startObject().field("scroll_id", "value_2").field("unknown", "keyword").endObject());
    Exception e = expectThrows(IllegalArgumentException.class, () -> RestSearchScrollAction.buildFromContent(invalidContent, searchScrollRequest));
    assertThat(e.getMessage(), startsWith("Unknown parameter [unknown]"));
}
Also used : SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) XContentParser(org.elasticsearch.common.xcontent.XContentParser)

Example 8 with SearchScrollRequest

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest in project hazelcast by hazelcast.

the class ElasticSourcePTest method givenMultipleResults_when_runProcessor_then_useScrollIdInFollowupScrollRequest.

@Test
public void givenMultipleResults_when_runProcessor_then_useScrollIdInFollowupScrollRequest() throws Exception {
    SearchHit hit = new SearchHit(0, "id-0", new Text("ignored"), emptyMap(), emptyMap());
    hit.sourceRef(new BytesArray(HIT_SOURCE));
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] { hit }, new TotalHits(3, EQUAL_TO), Float.NaN));
    SearchResponse response2 = mock(SearchResponse.class);
    SearchHit hit2 = new SearchHit(1, "id-1", new Text("ignored"), emptyMap(), emptyMap());
    hit2.sourceRef(new BytesArray(HIT_SOURCE2));
    when(response2.getHits()).thenReturn(new SearchHits(new SearchHit[] { hit2 }, new TotalHits(3, EQUAL_TO), Float.NaN));
    SearchResponse response3 = mock(SearchResponse.class);
    when(response3.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(3, EQUAL_TO), Float.NaN));
    when(mockClient.scroll(any(), any())).thenReturn(response2, response3);
    TestSupport testSupport = runProcessor();
    testSupport.expectOutput(newArrayList(HIT_SOURCE, HIT_SOURCE2));
    ArgumentCaptor<SearchScrollRequest> captor = forClass(SearchScrollRequest.class);
    verify(mockClient, times(2)).scroll(captor.capture(), any());
    SearchScrollRequest request = captor.getValue();
    assertThat(request.scrollId()).isEqualTo(SCROLL_ID);
    assertThat(request.scroll().keepAlive().getStringRep()).isEqualTo(KEEP_ALIVE);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) BytesArray(org.elasticsearch.common.bytes.BytesArray) SearchHit(org.elasticsearch.search.SearchHit) TestSupport(com.hazelcast.jet.core.test.TestSupport) Text(org.elasticsearch.common.text.Text) SearchHits(org.elasticsearch.search.SearchHits) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) SearchResponse(org.elasticsearch.action.search.SearchResponse) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 9 with SearchScrollRequest

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest in project sonarqube by SonarSource.

the class EsRequestDetailsTest method should_format_search_SearchScrollRequest.

@Test
public void should_format_search_SearchScrollRequest() {
    SearchScrollRequest scrollRequest = Requests.searchScrollRequest("scroll-id").scroll(TimeValue.ZERO);
    assertThat(EsRequestDetails.computeDetailsAsString(scrollRequest)).isEqualTo("ES search scroll request for scroll id 'Scroll{keepAlive=0s}'");
}
Also used : SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) Test(org.junit.Test)

Example 10 with SearchScrollRequest

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest in project sonarqube by SonarSource.

the class ViewIndex method findAllViewUuids.

public List<String> findAllViewUuids() {
    SearchRequest esSearch = EsClient.prepareSearch(ViewIndexDefinition.TYPE_VIEW).source(new SearchSourceBuilder().sort("_doc", SortOrder.ASC).fetchSource(false).size(100).query(matchAllQuery())).scroll(TimeValue.timeValueMinutes(SCROLL_TIME_IN_MINUTES));
    SearchResponse response = esClient.search(esSearch);
    List<String> result = new ArrayList<>();
    while (true) {
        List<SearchHit> hits = newArrayList(response.getHits());
        for (SearchHit hit : hits) {
            result.add(hit.getId());
        }
        String scrollId = response.getScrollId();
        response = esClient.scroll(new SearchScrollRequest().scrollId(scrollId).scroll(TimeValue.timeValueMinutes(SCROLL_TIME_IN_MINUTES)));
        // Break condition: No hits are returned
        if (response.getHits().getHits().length == 0) {
            ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
            clearScrollRequest.addScrollId(scrollId);
            esClient.clearScroll(clearScrollRequest);
            break;
        }
    }
    return result;
}
Also used : SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) ArrayList(java.util.ArrayList) Lists.newArrayList(com.google.common.collect.Lists.newArrayList) ClearScrollRequest(org.elasticsearch.action.search.ClearScrollRequest) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) SearchSourceBuilder(org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.elasticsearch.action.search.SearchResponse)

Aggregations

SearchScrollRequest (org.elasticsearch.action.search.SearchScrollRequest)9 SearchResponse (org.elasticsearch.action.search.SearchResponse)4 SearchHit (org.elasticsearch.search.SearchHit)4 ClearScrollRequest (org.elasticsearch.action.search.ClearScrollRequest)3 XContentParser (org.elasticsearch.common.xcontent.XContentParser)3 Test (org.junit.Test)2 Lists.newArrayList (com.google.common.collect.Lists.newArrayList)1 TestSupport (com.hazelcast.jet.core.test.TestSupport)1 ParallelJVMTest (com.hazelcast.test.annotation.ParallelJVMTest)1 QuickTest (com.hazelcast.test.annotation.QuickTest)1 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 TotalHits (org.apache.lucene.search.TotalHits)1 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)1 SearchRequest (org.elasticsearch.action.search.SearchRequest)1 NodeClient (org.elasticsearch.client.node.NodeClient)1 BytesArray (org.elasticsearch.common.bytes.BytesArray)1 DocumentField (org.elasticsearch.common.document.DocumentField)1 Settings (org.elasticsearch.common.settings.Settings)1 Text (org.elasticsearch.common.text.Text)1