Search in sources :

Example 1 with MAX_BATCH_RESULT_SIZE

use of org.icij.datashare.tasks.BatchSearchRunner.MAX_BATCH_RESULT_SIZE in project datashare by ICIJ.

the class BatchSearchRunnerTest method test_run_batch_search_truncate_to_60k_max_results.

@Test
public void test_run_batch_search_truncate_to_60k_max_results() throws Exception {
    Document[] documents = IntStream.range(0, MAX_SCROLL_SIZE).mapToObj(i -> createDoc("doc" + i).build()).toArray(Document[]::new);
    mockSearch.willReturn(MAX_BATCH_RESULT_SIZE / MAX_SCROLL_SIZE + 1, documents);
    BatchSearch batchSearch = new BatchSearch("uuid1", project("test-datashare"), "name", "desc", asSet("query"), new Date(), BatchSearch.State.QUEUED, local());
    assertThat(new BatchSearchRunner(indexer, new PropertiesProvider(), batchSearch, resultConsumer).call()).isLessThan(60000);
}
Also used : IntStream(java.util.stream.IntStream) MAX_SCROLL_SIZE(org.icij.datashare.tasks.BatchSearchRunner.MAX_SCROLL_SIZE) MockitoAnnotations.initMocks(org.mockito.MockitoAnnotations.initMocks) Mock(org.mockito.Mock) Date(java.util.Date) Assert.assertThrows(org.junit.Assert.assertThrows) HashMap(java.util.HashMap) Assertions.assertThat(org.fest.assertions.Assertions.assertThat) Arrays.asList(java.util.Arrays.asList) SearchException(org.icij.datashare.batch.SearchException) User(org.icij.datashare.user.User) Project.project(org.icij.datashare.text.Project.project) DatashareTimeRule(org.icij.datashare.test.DatashareTimeRule) BATCH_SEARCH_MAX_TIME(org.icij.datashare.cli.DatashareCliOptions.BATCH_SEARCH_MAX_TIME) ExecutorService(java.util.concurrent.ExecutorService) Before(org.junit.Before) BATCH_THROTTLE(org.icij.datashare.cli.DatashareCliOptions.BATCH_THROTTLE) PropertiesProvider(org.icij.datashare.PropertiesProvider) DocumentBuilder.createDoc(org.icij.datashare.text.DocumentBuilder.createDoc) Indexer(org.icij.datashare.text.indexing.Indexer) CollectionUtils.asSet(org.icij.datashare.CollectionUtils.asSet) Test(org.junit.Test) MAX_BATCH_RESULT_SIZE(org.icij.datashare.tasks.BatchSearchRunner.MAX_BATCH_RESULT_SIZE) Document(org.icij.datashare.text.Document) Executors(java.util.concurrent.Executors) User.local(org.icij.datashare.user.User.local) TimeUnit(java.util.concurrent.TimeUnit) Matchers.any(org.mockito.Matchers.any) CountDownLatch(java.util.concurrent.CountDownLatch) Mockito(org.mockito.Mockito) List(java.util.List) Rule(org.junit.Rule) BatchSearch(org.icij.datashare.batch.BatchSearch) TerFunction(org.icij.datashare.function.TerFunction) PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) Document(org.icij.datashare.text.Document) Date(java.util.Date) Test(org.junit.Test)

Aggregations

Arrays.asList (java.util.Arrays.asList)1 Date (java.util.Date)1 HashMap (java.util.HashMap)1 List (java.util.List)1 CountDownLatch (java.util.concurrent.CountDownLatch)1 ExecutorService (java.util.concurrent.ExecutorService)1 Executors (java.util.concurrent.Executors)1 TimeUnit (java.util.concurrent.TimeUnit)1 IntStream (java.util.stream.IntStream)1 Assertions.assertThat (org.fest.assertions.Assertions.assertThat)1 CollectionUtils.asSet (org.icij.datashare.CollectionUtils.asSet)1 PropertiesProvider (org.icij.datashare.PropertiesProvider)1 BatchSearch (org.icij.datashare.batch.BatchSearch)1 SearchException (org.icij.datashare.batch.SearchException)1 BATCH_SEARCH_MAX_TIME (org.icij.datashare.cli.DatashareCliOptions.BATCH_SEARCH_MAX_TIME)1 BATCH_THROTTLE (org.icij.datashare.cli.DatashareCliOptions.BATCH_THROTTLE)1 TerFunction (org.icij.datashare.function.TerFunction)1 MAX_BATCH_RESULT_SIZE (org.icij.datashare.tasks.BatchSearchRunner.MAX_BATCH_RESULT_SIZE)1 MAX_SCROLL_SIZE (org.icij.datashare.tasks.BatchSearchRunner.MAX_SCROLL_SIZE)1 DatashareTimeRule (org.icij.datashare.test.DatashareTimeRule)1