Search in sources :

Example 1 with TotalHits

use of org.graylog.shaded.elasticsearch7.org.apache.lucene.search.TotalHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithOptionsFn_then_shouldUseOptionsFnForScrollRequest.

@Test
public void when_runProcessorWithOptionsFn_then_shouldUseOptionsFnForScrollRequest() throws Exception {
    SearchHit hit = new SearchHit(0, "id-0", new Text("ignored"), emptyMap(), emptyMap());
    hit.sourceRef(new BytesArray(HIT_SOURCE));
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] { hit }, new TotalHits(1, EQUAL_TO), Float.NaN));
    SearchResponse response2 = mock(SearchResponse.class);
    when(response2.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(1, EQUAL_TO), Float.NaN));
    when(mockClient.scroll(any(), any())).thenReturn(response2);
    // get different instance than default
    TestSupport testSupport = runProcessor(request -> {
        Builder builder = RequestOptions.DEFAULT.toBuilder();
        builder.addHeader("TestHeader", "value");
        return builder.build();
    });
    testSupport.expectOutput(newArrayList(HIT_SOURCE));
    ArgumentCaptor<RequestOptions> captor = forClass(RequestOptions.class);
    verify(mockClient).scroll(any(), captor.capture());
    RequestOptions capturedOptions = captor.getValue();
    assertThat(capturedOptions.getHeaders()).extracting(h -> tuple(h.getName(), h.getValue())).containsExactly(tuple("TestHeader", "value"));
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) RestClient(org.elasticsearch.client.RestClient) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) SearchHits(org.elasticsearch.search.SearchHits) QuickTest(com.hazelcast.test.annotation.QuickTest) Assertions.assertThat(org.assertj.core.api.Assertions.assertThat) RunWith(org.junit.runner.RunWith) TestOutbox(com.hazelcast.jet.core.test.TestOutbox) SearchRequest(org.elasticsearch.action.search.SearchRequest) Node(org.elasticsearch.client.Node) Builder(org.elasticsearch.client.RequestOptions.Builder) Prirep(com.hazelcast.jet.elastic.impl.Shard.Prirep) BytesArray(org.elasticsearch.common.bytes.BytesArray) ArgumentCaptor(org.mockito.ArgumentCaptor) Text(org.elasticsearch.common.text.Text) SearchResponse(org.elasticsearch.action.search.SearchResponse) RequestOptions(org.elasticsearch.client.RequestOptions) RETURNS_DEEP_STUBS(org.mockito.Mockito.RETURNS_DEEP_STUBS) Before(org.junit.Before) SearchHit(org.elasticsearch.search.SearchHit) FunctionEx(com.hazelcast.function.FunctionEx) Collections.emptyMap(java.util.Collections.emptyMap) ActionRequest(org.elasticsearch.action.ActionRequest) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) Assertions.tuple(org.assertj.core.api.Assertions.tuple) Collections.emptyList(java.util.Collections.emptyList) Collection(java.util.Collection) Test(org.junit.Test) Mockito.times(org.mockito.Mockito.times) Mockito.when(org.mockito.Mockito.when) RestHighLevelClient(org.elasticsearch.client.RestHighLevelClient) Category(org.junit.experimental.categories.Category) EQUAL_TO(org.apache.lucene.search.TotalHits.Relation.EQUAL_TO) Serializable(java.io.Serializable) Mockito.verify(org.mockito.Mockito.verify) TotalHits(org.apache.lucene.search.TotalHits) List(java.util.List) ArgumentCaptor.forClass(org.mockito.ArgumentCaptor.forClass) TestSupport(com.hazelcast.jet.core.test.TestSupport) HazelcastParallelClassRunner(com.hazelcast.test.HazelcastParallelClassRunner) SearchScrollRequest(org.elasticsearch.action.search.SearchScrollRequest) Lists.newArrayList(org.assertj.core.util.Lists.newArrayList) Mockito.mock(org.mockito.Mockito.mock) BytesArray(org.elasticsearch.common.bytes.BytesArray) SearchHit(org.elasticsearch.search.SearchHit) RequestOptions(org.elasticsearch.client.RequestOptions) RestClientBuilder(org.elasticsearch.client.RestClientBuilder) Builder(org.elasticsearch.client.RequestOptions.Builder) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) Text(org.elasticsearch.common.text.Text) SearchHits(org.elasticsearch.search.SearchHits) SearchResponse(org.elasticsearch.action.search.SearchResponse) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 2 with TotalHits

use of org.graylog.shaded.elasticsearch7.org.apache.lucene.search.TotalHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithParallelism_thenUseSlicingBasedOnGlobalValues.

@Test
public void when_runProcessorWithParallelism_thenUseSlicingBasedOnGlobalValues() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport testSupport = runProcessor((r) -> RequestOptions.DEFAULT, emptyList(), true, false);
    testSupport.localProcessorIndex(1);
    testSupport.localParallelism(2);
    testSupport.globalProcessorIndex(4);
    testSupport.totalParallelism(6);
    testSupport.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    SliceBuilder slice = request.source().slice();
    // Slicing across all, should use global index / total parallelism
    assertThat(slice.getId()).isEqualTo(4);
    assertThat(slice.getMax()).isEqualTo(6);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 3 with TotalHits

use of org.graylog.shaded.elasticsearch7.org.apache.lucene.search.TotalHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithCoLocation_thenSearchShardsWithPreference.

@Test
public void when_runProcessorWithCoLocation_thenSearchShardsWithPreference() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport processor = runProcessorWithCoLocation(newArrayList(new Shard("my-index", 0, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 1, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 2, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1")));
    processor.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    assertThat(request.preference()).isEqualTo("_shards:0,1,2|_only_local");
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 4 with TotalHits

use of org.graylog.shaded.elasticsearch7.org.apache.lucene.search.TotalHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessor_then_executeSearchRequestWithScroll.

@Test
public void when_runProcessor_then_executeSearchRequestWithScroll() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport support = runProcessor();
    support.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    assertThat(request.scroll().keepAlive().getStringRep()).isEqualTo(KEEP_ALIVE);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Example 5 with TotalHits

use of org.graylog.shaded.elasticsearch7.org.apache.lucene.search.TotalHits in project hazelcast by hazelcast.

the class ElasticSourcePTest method when_runProcessorWithCoLocationAndSlicing_thenUseSlicingBasedOnLocalValues.

@Test
public void when_runProcessorWithCoLocationAndSlicing_thenUseSlicingBasedOnLocalValues() throws Exception {
    when(response.getHits()).thenReturn(new SearchHits(new SearchHit[] {}, new TotalHits(0, EQUAL_TO), Float.NaN));
    TestSupport testSupport = runProcessor((r) -> RequestOptions.DEFAULT, newArrayList(new Shard("my-index", 0, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 1, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1"), new Shard("my-index", 2, Prirep.p, 42, "STARTED", "10.0.0.1", "10.0.0.1:9200", "es1")), true, true);
    testSupport.localProcessorIndex(1);
    testSupport.localParallelism(2);
    testSupport.globalProcessorIndex(4);
    testSupport.totalParallelism(6);
    testSupport.expectOutput(emptyList());
    ArgumentCaptor<SearchRequest> captor = forClass(SearchRequest.class);
    verify(mockClient).search(captor.capture(), any());
    SearchRequest request = captor.getValue();
    SliceBuilder slice = request.source().slice();
    // Slicing across single node, should use local values
    assertThat(slice.getId()).isEqualTo(1);
    assertThat(slice.getMax()).isEqualTo(2);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) SearchRequest(org.elasticsearch.action.search.SearchRequest) SearchHit(org.elasticsearch.search.SearchHit) SliceBuilder(org.elasticsearch.search.slice.SliceBuilder) TestSupport(com.hazelcast.jet.core.test.TestSupport) SearchHits(org.elasticsearch.search.SearchHits) ParallelJVMTest(com.hazelcast.test.annotation.ParallelJVMTest) QuickTest(com.hazelcast.test.annotation.QuickTest) Test(org.junit.Test)

Aggregations

TotalHits (org.apache.lucene.search.TotalHits)18 Test (org.junit.Test)14 SearchHits (org.elasticsearch.search.SearchHits)13 SearchHit (org.elasticsearch.search.SearchHit)12 TestSupport (com.hazelcast.jet.core.test.TestSupport)9 ParallelJVMTest (com.hazelcast.test.annotation.ParallelJVMTest)9 QuickTest (com.hazelcast.test.annotation.QuickTest)9 SearchRequest (org.elasticsearch.action.search.SearchRequest)8 SearchResponse (org.elasticsearch.action.search.SearchResponse)7 BytesArray (org.elasticsearch.common.bytes.BytesArray)3 Text (org.elasticsearch.common.text.Text)3 SliceBuilder (org.elasticsearch.search.slice.SliceBuilder)3 FunctionEx (com.hazelcast.function.FunctionEx)2 TestOutbox (com.hazelcast.jet.core.test.TestOutbox)2 Prirep (com.hazelcast.jet.elastic.impl.Shard.Prirep)2 HazelcastParallelClassRunner (com.hazelcast.test.HazelcastParallelClassRunner)2 RerankerCascade (io.anserini.rerank.RerankerCascade)2 RerankerContext (io.anserini.rerank.RerankerContext)2 ScoredDocuments (io.anserini.rerank.ScoredDocuments)2 ScoreTiesAdjusterReranker (io.anserini.rerank.lib.ScoreTiesAdjusterReranker)2