Search in sources :

Example 56 with Document

use of org.icij.datashare.text.Document in project datashare by ICIJ.

the class BatchSearchRunnerIntTest method test_search_with_fuzziness.

@Test
public void test_search_with_fuzziness() throws Exception {
    Document mydoc = createDoc("mydoc").build();
    indexer.add(TEST_INDEX, mydoc);
    BatchSearch searchKo1 = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("doc"), User.local(), false, null, null, 1);
    BatchSearch searchKo2 = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("nodoc"), User.local(), false, null, null, 1);
    BatchSearch searchOk = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("hedoc"), User.local(), false, null, null, 2);
    new BatchSearchRunner(indexer, new PropertiesProvider(), searchKo1, resultConsumer).call();
    new BatchSearchRunner(indexer, new PropertiesProvider(), searchKo2, resultConsumer).call();
    new BatchSearchRunner(indexer, new PropertiesProvider(), searchOk, resultConsumer).call();
    verify(resultConsumer, never()).apply(eq(searchKo1.uuid), eq("doc"), anyList());
    verify(resultConsumer, never()).apply(eq(searchKo2.uuid), eq("nodoc"), anyList());
    verify(resultConsumer).apply(searchOk.uuid, "hedoc", singletonList(mydoc));
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) Document(org.icij.datashare.text.Document)

Example 57 with Document

use of org.icij.datashare.text.Document in project datashare by ICIJ.

the class BatchSearchRunnerIntTest method test_search_with_error.

@Test
public void test_search_with_error() throws Exception {
    Document mydoc = createDoc("docId1").with("mydoc").build();
    indexer.add(TEST_INDEX, mydoc);
    BatchSearch search = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("AND mydoc"), User.local());
    SearchException sex = assertThrows(SearchException.class, () -> new BatchSearchRunner(indexer, new PropertiesProvider(), search, resultConsumer).call());
    assertThat(sex.toString()).contains("Failed to parse query [AND mydoc]");
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) SearchException(org.icij.datashare.batch.SearchException) Document(org.icij.datashare.text.Document)

Example 58 with Document

use of org.icij.datashare.text.Document in project datashare by ICIJ.

the class BatchSearchRunnerIntTest method test_search_with_phraseMatches.

@Test
public void test_search_with_phraseMatches() throws Exception {
    Document mydoc = createDoc("docId").with("mydoc to find").build();
    indexer.add(TEST_INDEX, mydoc);
    BatchSearch searchKo = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("to find mydoc"), User.local(), false, null, null, true);
    BatchSearch searchOk = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("mydoc to find"), User.local(), false, null, null, true);
    new BatchSearchRunner(indexer, new PropertiesProvider(), searchKo, resultConsumer).call();
    new BatchSearchRunner(indexer, new PropertiesProvider(), searchOk, resultConsumer).call();
    verify(resultConsumer, never()).apply(eq(searchKo.uuid), eq("to find mydoc"), anyList());
    verify(resultConsumer).apply(searchOk.uuid, "mydoc to find", singletonList(mydoc));
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) Document(org.icij.datashare.text.Document)

Example 59 with Document

use of org.icij.datashare.text.Document in project datashare by ICIJ.

the class BatchSearchRunnerIntTest method test_search_with_file_types_ok.

@Test
public void test_search_with_file_types_ok() throws Exception {
    Document mydoc = createDoc("mydoc").build();
    indexer.add(TEST_INDEX, mydoc);
    BatchSearch search = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("mydoc"), User.local(), false, singletonList("text/plain"), null, 0);
    new BatchSearchRunner(indexer, new PropertiesProvider(), search, resultConsumer).call();
    verify(resultConsumer).apply(search.uuid, "mydoc", singletonList(mydoc));
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) Document(org.icij.datashare.text.Document)

Example 60 with Document

use of org.icij.datashare.text.Document in project datashare by ICIJ.

the class BatchSearchRunnerIntTest method test_search_with_operators.

@Test
public void test_search_with_operators() throws Exception {
    Document mydoc1 = createDoc("docId1").with("mydoc one").build();
    Document mydoc2 = createDoc("docId2").with("mydoc two").build();
    indexer.add(TEST_INDEX, mydoc1);
    indexer.add(TEST_INDEX, mydoc2);
    BatchSearch search = new BatchSearch(project(TEST_INDEX), "name", "desc", asSet("mydoc AND one"), User.local());
    new BatchSearchRunner(indexer, new PropertiesProvider(), search, resultConsumer).call();
    verify(resultConsumer).apply(search.uuid, "mydoc AND one", singletonList(mydoc1));
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) BatchSearch(org.icij.datashare.batch.BatchSearch) Document(org.icij.datashare.text.Document)

Aggregations

Document (org.icij.datashare.text.Document)63 Test (org.junit.Test)48 PropertiesProvider (org.icij.datashare.PropertiesProvider)19 BatchSearch (org.icij.datashare.batch.BatchSearch)15 NamedEntity (org.icij.datashare.text.NamedEntity)11 TikaDocument (org.icij.extract.document.TikaDocument)10 HashMap (java.util.HashMap)9 Path (java.nio.file.Path)6 Date (java.util.Date)5 Indexer (org.icij.datashare.text.indexing.Indexer)5 File (java.io.File)4 IOException (java.io.IOException)4 InputStream (java.io.InputStream)4 IntStream (java.util.stream.IntStream)4 DocumentBuilder.createDoc (org.icij.datashare.text.DocumentBuilder.createDoc)4 Project.project (org.icij.datashare.text.Project.project)4 User (org.icij.datashare.user.User)4 Rule (org.junit.Rule)4 Arrays.asList (java.util.Arrays.asList)3 List (java.util.List)3