Search in sources :

Example 51 with SearchHits

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithOptionsFn_then_shouldUseOptionsFnForScrollRequest.

@Test
public void when_runProcessorWithOptionsFn_then_shouldUseOptionsFnForScrollRequest() throws Exception {
    SearchHit hit = new SearchHit(0, "id-0", new Text("ignored"), emptyMap(), emptyMap());
    hit.sourceRef(new BytesArray(HIT_SOURCE));
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] { hit }, new TotalHits(1, EQUAL_TO), Float.NaN));
    SearchResponse response2 = mock(SearchResponse.class);
    when(response2.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(1, EQUAL_TO), Float.NaN));
    when(mockClient.scroll(any(), any())).thenReturn(response2);
    // get different instance than default
    TestSupport testSupport = runProcessor(request -> {
        Builder builder = RequestOptions.DEFAULT.toBuilder();
        builder.addHeader("TestHeader", "value");
        return builder.build();
    });
    testSupport.expectOutput(newArrayList(HIT_SOURCE));
    ArgumentCaptor<RequestOptions> captor = forClass(RequestOptions.class);
    verify(mockClient).scroll(any(), captor.capture());
    RequestOptions capturedOptions = captor.getValue();
    assertThat(capturedOptions.getHeaders()).extracting(h -> tuple(h.getName(), h.getValue())).containsExactly(tuple("TestHeader", "value"));
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) RestClient(org.elasticsearch.client.RestClient) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) SearchHits(org.elasticsearch.search.SearchHits) QuickTest(com.hazelcast.test.annotation.QuickTest) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) TestOutbox(com.hazelcast.jet.core.test.TestOutbox) SearchRequest(org.elasticsearch.action.search.SearchRequest) Node(org.elasticsearch.client.Node) Builder(org.elasticsearch.client.RequestOptions.Builder) Prirep(com.hazelcast.jet.elastic.impl.Shard.Prirep) BytesArray(org.elasticsearch.common.bytes.BytesArray) ArgumentCaptor(org.mockito.ArgumentCaptor) Text(org.elasticsearch.common.text.Text) SearchResponse(org.elasticsearch.action.search.SearchResponse) RequestOptions(org.elasticsearch.client.RequestOptions) RETURNS_DEEP_STUBS(org.mockito.Mockito.RETURNS_DEEP_STUBS) Before(org.junit.Before) SearchHit(org.elasticsearch.search.SearchHit) FunctionEx(com.hazelcast.function.FunctionEx) Collections.emptyMap(java.util.Collections.emptyMap) ActionRequest(org.elasticsearch.action.ActionRequest) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) Assertions.tuple(org.assertj.core.api.Assertions.tuple) Collections.emptyList(java.util.Collections.emptyList) Collection(java.util.Collection) Test(org.junit.Test) Mockito.times(org.mockito.Mockito.times) Mockito.when(org.mockito.Mockito.when) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Category(org.junit.experimental.categories.Category) EQUAL_TO(org.apache.lucene.search.TotalHits.Relation.EQUAL_TO) Serializable(java.io.Serializable) Mockito.verify(org.mockito.Mockito.verify) TotalHits(org.apache.lucene.search.TotalHits) List(java.util.List) ArgumentCaptor.forClass(org.mockito.ArgumentCaptor.forClass) TestSupport(com.hazelcast.jet.core.test.TestSupport) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) Lists.newArrayList(org.assertj.core.util.Lists.newArrayList) Mockito.mock(org.mockito.Mockito.mock) BytesArray(org.elasticsearch.common.bytes.BytesArray) SearchHit(org.elasticsearch.search.SearchHit) RequestOptions(org.elasticsearch.client.RequestOptions) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) Builder(org.elasticsearch.client.RequestOptions.Builder) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) Text(org.elasticsearch.common.text.Text) SearchHits(org.elasticsearch.search.SearchHits) SearchResponse(org.elasticsearch.action.search.SearchResponse) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 52 with SearchHits

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithParallelism_thenUseSlicingBasedOnGlobalValues.

@Test
public void when_runProcessorWithParallelism_thenUseSlicingBasedOnGlobalValues() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport testSupport = runProcessor((r) -> RequestOptions.DEFAULT, emptyList(), true, false);
    testSupport.localProcessorIndex(1);
    testSupport.localParallelism(2);
    testSupport.globalProcessorIndex(4);
    testSupport.totalParallelism(6);
    testSupport.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    SliceBuilder slice = request.source().slice();
    // Slicing across all, should use global index / total parallelism
    assertThat(slice.getId()).isEqualTo(4);
    assertThat(slice.getMax()).isEqualTo(6);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 53 with SearchHits

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithCoLocation_thenSearchShardsWithPreference.

@Test
public void when_runProcessorWithCoLocation_thenSearchShardsWithPreference() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport processor = runProcessorWithCoLocation(newArrayList(new Shard("my-index", 0, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 1, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 2, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1")));
    processor.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    assertThat(request.preference()).isEqualTo("_shards:0,1,2|_only_local");
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 54 with SearchHits

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessor_then_executeSearchRequestWithScroll.

@Test
public void when_runProcessor_then_executeSearchRequestWithScroll() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport support = runProcessor();
    support.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    assertThat(request.scroll().keepAlive().getStringRep()).isEqualTo(KEEP_ALIVE);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 55 with SearchHits

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.search.SearchHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithCoLocationAndSlicing_thenUseSlicingBasedOnLocalValues.

@Test
public void when_runProcessorWithCoLocationAndSlicing_thenUseSlicingBasedOnLocalValues() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport testSupport = runProcessor((r) -> RequestOptions.DEFAULT, newArrayList(new Shard("my-index", 0, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 1, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 2, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1")), true, true);
    testSupport.localProcessorIndex(1);
    testSupport.localParallelism(2);
    testSupport.globalProcessorIndex(4);
    testSupport.totalParallelism(6);
    testSupport.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    SliceBuilder slice = request.source().slice();
    // Slicing across single node, should use local values
    assertThat(slice.getId()).isEqualTo(1);
    assertThat(slice.getMax()).isEqualTo(2);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Aggregations

SearchHits (org.elasticsearch.search.SearchHits)137 SearchResponse (org.elasticsearch.action.search.SearchResponse)93 SearchHit (org.elasticsearch.search.SearchHit)86 ArrayList (java.util.ArrayList)45 SearchRequestBuilder (org.elasticsearch.action.search.SearchRequestBuilder)27 IOException (java.io.IOException)25 Map (java.util.Map)23 ElasticsearchAssertions.assertSearchResponse (org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse)20 HashMap (java.util.HashMap)19 IndexRequestBuilder (org.elasticsearch.action.index.IndexRequestBuilder)17 SearchRequest (org.elasticsearch.action.search.SearchRequest)16 Terms (org.elasticsearch.search.aggregations.bucket.terms.Terms)16 TopHits (org.elasticsearch.search.aggregations.metrics.tophits.TopHits)15 SearchSourceBuilder (org.elasticsearch.search.builder.SearchSourceBuilder)14 TotalHits (org.apache.lucene.search.TotalHits)12 Test (org.junit.Test)12 ScoreDoc (org.apache.lucene.search.ScoreDoc)11 Text (org.elasticsearch.common.text.Text)11 InnerHitBuilder (org.elasticsearch.index.query.InnerHitBuilder)11 QueryBuilder (org.elasticsearch.index.query.QueryBuilder)10