Search in sources :

Example 1 with DeleteRequestBuilder

use of org.elasticsearch.action.delete.DeleteRequestBuilder in project sonarqube by SonarSource.

the class BulkIndexer method addDeletion.

public void addDeletion(SearchRequestBuilder searchRequest) {
    searchRequest.addSort("_doc", SortOrder.ASC).setScroll(TimeValue.timeValueMinutes(5)).setSize(100).setFetchSource(false);
    // this search is synchronous. An optimization would be to be non-blocking,
    // but it requires to tracking pending requests in close().
    // Same semaphore can't be reused because of potential deadlock (requires to acquire
    // two locks)
    SearchResponse searchResponse = searchRequest.get();
    while (true) {
        SearchHit[] hits = searchResponse.getHits().getHits();
        for (SearchHit hit : hits) {
            SearchHitField routing = hit.field("_routing");
            DeleteRequestBuilder deleteRequestBuilder = client.prepareDelete(hit.index(), hit.type(), hit.getId());
            if (routing != null) {
                deleteRequestBuilder.setRouting(routing.getValue());
            }
            add(deleteRequestBuilder.request());
        }
        String scrollId = searchResponse.getScrollId();
        searchResponse = client.prepareSearchScroll(scrollId).setScroll(TimeValue.timeValueMinutes(5)).get();
        if (hits.length == 0) {
            client.nativeClient().prepareClearScroll().addScrollId(scrollId).get();
            break;
        }
    }
}
Also used : DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder) SearchHit(org.elasticsearch.search.SearchHit) SearchHitField(org.elasticsearch.search.SearchHitField) SearchResponse(org.elasticsearch.action.search.SearchResponse)

Example 2 with DeleteRequestBuilder

use of org.elasticsearch.action.delete.DeleteRequestBuilder in project nifi by apache.

the class TestDeleteElasticsearch5 method testDeleteRetryableException.

@Test
public void testDeleteRetryableException() throws IOException {
    mockDeleteProcessor = new DeleteElasticsearch5() {

        @Override
        protected DeleteRequestBuilder prepareDeleteRequest(String index, String docId, String docType) {
            return null;
        }

        @Override
        protected DeleteResponse doDelete(DeleteRequestBuilder requestBuilder) throws InterruptedException, ExecutionException {
            throw new ElasticsearchTimeoutException("timeout");
        }

        @Override
        public void setup(ProcessContext context) {
        }
    };
    runner = TestRunners.newTestRunner(mockDeleteProcessor);
    runner.setValidateExpressionUsage(true);
    runner.setProperty(AbstractElasticsearch5TransportClientProcessor.CLUSTER_NAME, "elasticsearch");
    runner.setProperty(AbstractElasticsearch5TransportClientProcessor.HOSTS, "127.0.0.1:9300");
    runner.setProperty(AbstractElasticsearch5TransportClientProcessor.PING_TIMEOUT, "5s");
    runner.setProperty(AbstractElasticsearch5TransportClientProcessor.SAMPLER_INTERVAL, "5s");
    runner.setProperty(DeleteElasticsearch5.INDEX, INDEX1);
    runner.setProperty(DeleteElasticsearch5.TYPE, TYPE1);
    runner.setProperty(DeleteElasticsearch5.DOCUMENT_ID, "${documentId}");
    runner.assertValid();
    runner.enqueue(new byte[] {}, new HashMap<String, String>() {

        {
            put("documentId", documentId);
        }
    });
    runner.run(1, true, true);
    runner.assertAllFlowFilesTransferred(DeleteElasticsearch5.REL_RETRY, 1);
    final MockFlowFile out = runner.getFlowFilesForRelationship(DeleteElasticsearch5.REL_RETRY).get(0);
    assertNotNull(out);
    assertEquals("timeout", out.getAttribute(DeleteElasticsearch5.ES_ERROR_MESSAGE));
    out.assertAttributeEquals(DeleteElasticsearch5.ES_REST_STATUS, null);
    out.assertAttributeEquals(DeleteElasticsearch5.ES_FILENAME, documentId);
    out.assertAttributeEquals(DeleteElasticsearch5.ES_INDEX, INDEX1);
    out.assertAttributeEquals(DeleteElasticsearch5.ES_TYPE, TYPE1);
}
Also used : DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder) MockFlowFile(org.apache.nifi.util.MockFlowFile) DeleteResponse(org.elasticsearch.action.delete.DeleteResponse) ElasticsearchTimeoutException(org.elasticsearch.ElasticsearchTimeoutException) ExecutionException(java.util.concurrent.ExecutionException) ProcessContext(org.apache.nifi.processor.ProcessContext) Test(org.junit.Test)

Example 3 with DeleteRequestBuilder

use of org.elasticsearch.action.delete.DeleteRequestBuilder in project fess by codelibs.

the class FessEsClient method delete.

public boolean delete(final String index, final String type, final String id, final long version) {
    try {
        final DeleteRequestBuilder builder = client.prepareDelete(index, type, id).setRefreshPolicy(RefreshPolicy.IMMEDIATE);
        if (version > 0) {
            builder.setVersion(version);
        }
        final DeleteResponse response = builder.execute().actionGet(ComponentUtil.getFessConfig().getIndexDeleteTimeout());
        return response.getResult() == Result.DELETED;
    } catch (final ElasticsearchException e) {
        throw new FessEsClientException("Failed to delete: " + index + "/" + type + "/" + id + "/" + version, e);
    }
}
Also used : DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder) DeleteResponse(org.elasticsearch.action.delete.DeleteResponse) ElasticsearchException(org.elasticsearch.ElasticsearchException)

Example 4 with DeleteRequestBuilder

use of org.elasticsearch.action.delete.DeleteRequestBuilder in project elasticsearch-indexing-proxy by codelibs.

the class RequestSender method processDeleteRequest.

private void processDeleteRequest(final StreamInput streamInput) throws IOException {
    final DeleteRequestBuilder builder = RequestUtils.createDeleteRequest(client, streamInput, index);
    executeDeleteRequest(streamInput, builder);
}
Also used : DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder)

Example 5 with DeleteRequestBuilder

use of org.elasticsearch.action.delete.DeleteRequestBuilder in project elasticsearch-indexing-proxy by codelibs.

the class RequestUtils method createDeleteRequest.

public static DeleteRequestBuilder createDeleteRequest(final Client client, final StreamInput streamInput, final String index) throws IOException {
    final DeleteRequestBuilder builder = client.prepareDelete();
    final DeleteRequest request = builder.request();
    request.readFrom(streamInput);
    if (index != null) {
        request.index(index);
    }
    return builder;
}
Also used : DeleteRequestBuilder(org.elasticsearch.action.delete.DeleteRequestBuilder) DeleteRequest(org.elasticsearch.action.delete.DeleteRequest)

Aggregations

DeleteRequestBuilder (org.elasticsearch.action.delete.DeleteRequestBuilder)10 DeleteResponse (org.elasticsearch.action.delete.DeleteResponse)7 ExecutionException (java.util.concurrent.ExecutionException)3 ProcessContext (org.apache.nifi.processor.ProcessContext)2 MockFlowFile (org.apache.nifi.util.MockFlowFile)2 ElasticsearchException (org.elasticsearch.ElasticsearchException)2 ElasticsearchTimeoutException (org.elasticsearch.ElasticsearchTimeoutException)2 Test (org.junit.Test)2 IOException (java.io.IOException)1 HashMap (java.util.HashMap)1 FlowFile (org.apache.nifi.flowfile.FlowFile)1 ComponentLog (org.apache.nifi.logging.ComponentLog)1 ProcessException (org.apache.nifi.processor.exception.ProcessException)1 ResourceNotFoundException (org.elasticsearch.ResourceNotFoundException)1 DeleteRequest (org.elasticsearch.action.delete.DeleteRequest)1 SearchResponse (org.elasticsearch.action.search.SearchResponse)1 SearchHit (org.elasticsearch.search.SearchHit)1 SearchHitField (org.elasticsearch.search.SearchHitField)1 ReceiveTimeoutTransportException (org.elasticsearch.transport.ReceiveTimeoutTransportException)1 IndexException (org.molgenis.data.index.exception.IndexException)1