Search in sources :

Example 46 with GetRequest

use of org.elasticsearch.action.get.GetRequest in project datashare by ICIJ.

the class ElasticsearchSpewerTest method test_truncated_content_if_document_is_smaller_than_limit.

@Test
public void test_truncated_content_if_document_is_smaller_than_limit() throws Exception {
    ElasticsearchSpewer limitedContentSpewer = new ElasticsearchSpewer(es.client, text -> Language.ENGLISH, new FieldNames(), publisher, new PropertiesProvider(new HashMap<String, String>() {

        {
            put("maxContentLength", "20");
        }
    })).withRefresh(IMMEDIATE).withIndex("test-datashare");
    final TikaDocument document = new DocumentFactory().withIdentifier(new PathIdentifier()).create(get("ok-file.txt"));
    final ParsingReader reader = new ParsingReader(new ByteArrayInputStream("this content is ok".getBytes()));
    document.setReader(reader);
    limitedContentSpewer.write(document);
    GetResponse documentFields = es.client.get(new GetRequest(TEST_INDEX, document.getId()), RequestOptions.DEFAULT);
    assertThat(documentFields.getSourceAsMap()).includes(entry("content", "this content is ok"));
}
Also used : PropertiesProvider(org.icij.datashare.PropertiesProvider) DocumentFactory(org.icij.extract.document.DocumentFactory) FieldNames(org.icij.spewer.FieldNames) HashMap(java.util.HashMap) ParsingReader(org.apache.tika.parser.ParsingReader) ByteArrayInputStream(java.io.ByteArrayInputStream) GetRequest(org.elasticsearch.action.get.GetRequest) PathIdentifier(org.icij.extract.document.PathIdentifier) TikaDocument(org.icij.extract.document.TikaDocument) GetResponse(org.elasticsearch.action.get.GetResponse) Test(org.junit.Test)

Aggregations

GetRequest (org.elasticsearch.action.get.GetRequest)45 GetResponse (org.elasticsearch.action.get.GetResponse)29 Test (org.junit.Test)14 IOException (java.io.IOException)13 IndexRequest (org.elasticsearch.action.index.IndexRequest)9 HashMap (java.util.HashMap)7 TikaDocument (org.icij.extract.document.TikaDocument)7 FetchSourceContext (org.elasticsearch.search.fetch.subphase.FetchSourceContext)6 ArrayList (java.util.ArrayList)5 DocumentFactory (org.icij.extract.document.DocumentFactory)5 ByteArrayInputStream (java.io.ByteArrayInputStream)4 ParsingReader (org.apache.tika.parser.ParsingReader)4 ElasticsearchException (org.elasticsearch.ElasticsearchException)4 BulkItemResponse (org.elasticsearch.action.bulk.BulkItemResponse)4 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)4 SearchRequest (org.elasticsearch.action.search.SearchRequest)4 UpdateRequest (org.elasticsearch.action.update.UpdateRequest)4 PathIdentifier (org.icij.extract.document.PathIdentifier)4 BulkRequest (org.elasticsearch.action.bulk.BulkRequest)3 MultiGetRequest (org.elasticsearch.action.get.MultiGetRequest)3