Examples with BulkResponse org.elasticsearch.action.bulk.BulkResponse used on opensource projects

Example 96 with BulkResponse

use of org.elasticsearch.action.bulk.BulkResponse in project samza by apache.

the class ElasticsearchSystemProducerTest method testFlushFailedSendFromFailedDocument.

@Test(expected = SamzaException.class)
public void testFlushFailedSendFromFailedDocument() throws Exception {
    ArgumentCaptor<BulkProcessor.Listener> listenerCaptor = ArgumentCaptor.forClass(BulkProcessor.Listener.class);
    when(BULK_PROCESSOR_FACTORY.getBulkProcessor(eq(CLIENT), listenerCaptor.capture())).thenReturn(processorOne);
    producer.register(SOURCE_ONE);
    BulkResponse response = getRespWithFailedDocument(RestStatus.BAD_REQUEST);
    listenerCaptor.getValue().afterBulk(0, null, response);
    producer.flush(SOURCE_ONE);
}

Example 97 with BulkResponse

use of org.elasticsearch.action.bulk.BulkResponse in project bibot by alfintech.

the class ElasticPublisher method publishAnalytics.

@Override
public void publishAnalytics(List<Analytic> analytics) throws IOException {
    BulkRequestBuilder bulkRequest = client.prepareBulk();
    for (Analytic analytic : analytics) {
        System.out.println(analytic);
        XContentBuilder builder = XContentFactory.jsonBuilder().startObject().field("currencyPair", analytic.getCurrencyPair().toString()).field("intervalVolume", BigDecimalUtil.toDouble(analytic.getIntervalVolume())).field("dayVolume", BigDecimalUtil.toDouble(analytic.getDayAverage())).field("percentageVolume", BigDecimalUtil.toDouble(analytic.getPercentageVolume())).field("percentageAllTimeHigh", BigDecimalUtil.toDouble(analytic.getPercentageAllTimeHigh())).field("date", analytic.getDate()).endObject();
        bulkRequest.add(client.prepareIndex("marketdataanalytic", "minutely", analytic.getCurrencyPair().toString() + analytic.getDate().getTime()).setSource(builder));
    }
    BulkResponse response = bulkRequest.get();
    LOG.debug(response.toString());
}

Example 98 with BulkResponse

use of org.elasticsearch.action.bulk.BulkResponse in project stash-codesearch-plugin by palantir.

the class RequestBuffer method flush.

public BulkResponse flush() {
    BulkResponse resp = null;
    if (bulkRequest.numberOfActions() > 0) {
        resp = bulkRequest.get();
        bulkRequest = client.prepareBulk();
    }
    return resp;
}

Example 99 with BulkResponse

use of org.elasticsearch.action.bulk.BulkResponse in project flink by apache.

the class Elasticsearch6SinkBuilder method getBulkProcessorBuilderFactory.

@Override
protected BulkProcessorBuilderFactory getBulkProcessorBuilderFactory() {
    return new BulkProcessorBuilderFactory() {

        @Override
        public BulkProcessor.Builder apply(RestHighLevelClient client, BulkProcessorConfig bulkProcessorConfig, BulkProcessor.Listener listener) {
            BulkProcessor.Builder builder = BulkProcessor.builder(new // This cannot be inlined as a
            BulkRequestConsumerFactory() {

                // lambda because then
                // deserialization fails
                @Override
                public void accept(BulkRequest bulkRequest, ActionListener<BulkResponse> bulkResponseActionListener) {
                    client.bulkAsync(bulkRequest, bulkResponseActionListener);
                }
            }, listener);
            if (bulkProcessorConfig.getBulkFlushMaxActions() != -1) {
                builder.setBulkActions(bulkProcessorConfig.getBulkFlushMaxActions());
            }
            if (bulkProcessorConfig.getBulkFlushMaxMb() != -1) {
                builder.setBulkSize(new ByteSizeValue(bulkProcessorConfig.getBulkFlushMaxMb(), ByteSizeUnit.MB));
            }
            if (bulkProcessorConfig.getBulkFlushInterval() != -1) {
                builder.setFlushInterval(new TimeValue(bulkProcessorConfig.getBulkFlushInterval()));
            }
            BackoffPolicy backoffPolicy;
            final TimeValue backoffDelay = new TimeValue(bulkProcessorConfig.getBulkFlushBackOffDelay());
            final int maxRetryCount = bulkProcessorConfig.getBulkFlushBackoffRetries();
            switch(bulkProcessorConfig.getFlushBackoffType()) {
                case CONSTANT:
                    backoffPolicy = BackoffPolicy.constantBackoff(backoffDelay, maxRetryCount);
                    break;
                case EXPONENTIAL:
                    backoffPolicy = BackoffPolicy.exponentialBackoff(backoffDelay, maxRetryCount);
                    break;
                case NONE:
                    backoffPolicy = BackoffPolicy.noBackoff();
                    break;
                default:
                    throw new IllegalArgumentException("Received unknown backoff policy type " + bulkProcessorConfig.getFlushBackoffType());
            }
            builder.setBackoffPolicy(backoffPolicy);
            return builder;
        }
    };
}

Example 100 with BulkResponse

use of org.elasticsearch.action.bulk.BulkResponse in project elasticsearch-suggest-plugin by spinscale.

the class AbstractSuggestTest method indexProducts.

private void indexProducts(List<Map<String, Object>> products, String index, String routing) throws Exception {
    long currentCount = getCurrentDocumentCount(index);
    BulkRequest bulkRequest = new BulkRequest();
    for (Map<String, Object> product : products) {
        IndexRequest indexRequest = new IndexRequest(index, "product", (String) product.get("ProductId"));
        indexRequest.source(product);
        if (Strings.hasLength(routing)) {
            indexRequest.routing(routing);
        }
        bulkRequest.add(indexRequest);
    }
    bulkRequest.refresh(true);
    BulkResponse response = client().bulk(bulkRequest).actionGet();
    if (response.hasFailures()) {
        fail("Error in creating products: " + response.buildFailureMessage());
    }
    assertDocumentCountAfterIndexing(index, products.size() + currentCount);
}