Search in sources :

Example 21 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class MessagesAdapterES7 method runBulkRequest.

private BulkResponse runBulkRequest(int indexedSuccessfully, List<IndexingRequest> chunk) throws ChunkedBulkIndexer.EntityTooLargeException {
    final BulkRequest bulkRequest = createBulkRequest(chunk);
    final BulkResponse result;
    try {
        result = this.client.execute((c, requestOptions) -> c.bulk(bulkRequest, requestOptions));
    } catch (ElasticsearchException e) {
        for (ElasticsearchException cause : e.guessRootCauses()) {
            if (cause.status().equals(RestStatus.REQUEST_ENTITY_TOO_LARGE)) {
                throw new ChunkedBulkIndexer.EntityTooLargeException(indexedSuccessfully, indexingErrorsFrom(chunk));
            }
        }
        throw new org.graylog2.indexer.ElasticsearchException(e);
    }
    return result;
}
Also used : Iterables(com.google.common.collect.Iterables) Arrays(java.util.Arrays) IndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest) MessagesAdapter(org.graylog2.indexer.messages.MessagesAdapter) ElasticsearchException(org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchException) LoggerFactory(org.slf4j.LoggerFactory) XContentType(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentType) GetResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.get.GetResponse) Function(java.util.function.Function) ArrayList(java.util.ArrayList) Inject(javax.inject.Inject) Meter(com.codahale.metrics.Meter) Indexable(org.graylog2.indexer.messages.Indexable) ResultMessage(org.graylog2.indexer.results.ResultMessage) Locale(java.util.Locale) Map(java.util.Map) Messages(org.graylog2.indexer.messages.Messages) AnalyzeResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.AnalyzeResponse) MetricRegistry(com.codahale.metrics.MetricRegistry) Logger(org.slf4j.Logger) IndexingRequest(org.graylog2.indexer.messages.IndexingRequest) BulkItemResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkItemResponse) AnalyzeRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.AnalyzeRequest) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) GetRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.get.GetRequest) JsonProcessingException(com.fasterxml.jackson.core.JsonProcessingException) RestStatus(org.graylog.shaded.elasticsearch7.org.elasticsearch.rest.RestStatus) IOException(java.io.IOException) Collectors(java.util.stream.Collectors) List(java.util.List) BulkResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkResponse) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) ChunkedBulkIndexer(org.graylog2.indexer.messages.ChunkedBulkIndexer) Collections(java.util.Collections) MetricRegistry.name(com.codahale.metrics.MetricRegistry.name) DocumentNotFoundException(org.graylog2.indexer.messages.DocumentNotFoundException) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) BulkResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkResponse) ElasticsearchException(org.graylog.shaded.elasticsearch7.org.elasticsearch.ElasticsearchException) ChunkedBulkIndexer(org.graylog2.indexer.messages.ChunkedBulkIndexer)

Example 22 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class FieldMappingApi method fieldTypes.

public Map<String, FieldMapping> fieldTypes(String index) {
    final JsonNode result = client.execute((c, requestOptions) -> {
        final Response response = c.getLowLevelClient().performRequest(request(index));
        return objectMapper.readTree(response.getEntity().getContent());
    }, "Unable to retrieve field types of index " + index);
    final JsonNode fields = result.path(index).path("mappings").path("properties");
    // noinspection UnstableApiUsage
    return Streams.stream(fields.fields()).collect(Collectors.toMap(Map.Entry::getKey, entry -> FieldMapping.create(entry.getValue().path("type").asText(), entry.getValue().path("fielddata").asBoolean())));
}
Also used : Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) Inject(javax.inject.Inject) ElasticsearchClient(org.graylog.storage.elasticsearch7.ElasticsearchClient) Map(java.util.Map) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) AutoValue(com.google.auto.value.AutoValue) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) Optional(java.util.Optional) JsonNode(com.fasterxml.jackson.databind.JsonNode) Streams(com.google.common.collect.Streams) Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) Collectors(java.util.stream.Collectors) Nullable(javax.annotation.Nullable) JsonNode(com.fasterxml.jackson.databind.JsonNode) Map(java.util.Map)

Example 23 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class StatsApi method stats.

private JsonNode stats(Collection<String> indices, Collection<String> metrics, Consumer<Request> prepareRequest) {
    final StringBuilder endpoint = new StringBuilder();
    if (!indices.isEmpty()) {
        final String joinedIndices = String.join(",", indices);
        endpoint.append("/");
        endpoint.append(joinedIndices);
    }
    endpoint.append("/_stats");
    if (!metrics.isEmpty()) {
        final String joinedMetrics = String.join(",", metrics);
        endpoint.append("/");
        endpoint.append(joinedMetrics);
    }
    final Request request = new Request("GET", endpoint.toString());
    prepareRequest.accept(request);
    return client.execute((c, requestOptions) -> {
        request.setOptions(requestOptions);
        final Response response = c.getLowLevelClient().performRequest(request);
        return objectMapper.readTree(response.getEntity().getContent());
    }, "Unable to retrieve index stats for " + String.join(",", indices));
}
Also used : Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request)

Example 24 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class ScrollResultES7 method nextSearchResult.

private SearchResponse nextSearchResult() throws IOException {
    final SearchScrollRequest scrollRequest = new SearchScrollRequest(this.scrollId);
    scrollRequest.scroll(TimeValue.parseTimeValue(this.scroll, DEFAULT_SCROLL, "scroll time"));
    return client.executeWithIOException((c, requestOptions) -> c.scroll(scrollRequest, requestOptions), "Unable to retrieve next chunk from search: ");
}
Also used : SearchScrollRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchScrollRequest)

Example 25 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class ScrollResultES7 method cancel.

@Override
public void cancel() throws IOException {
    final ClearScrollRequest request = new ClearScrollRequest();
    request.addScrollId(scrollId);
    client.executeWithIOException((c, requestOptions) -> c.clearScroll(request, requestOptions), "Unable to cancel scrolling search request");
}
Also used : ClearScrollRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.ClearScrollRequest)

Aggregations

IndicesAliasesRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest)9 JsonNode (com.fasterxml.jackson.databind.JsonNode)8 Response (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response)8 CreateIndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest)8 Collectors (java.util.stream.Collectors)7 ClusterHealthRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest)7 Request (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request)7 Inject (javax.inject.Inject)6 DeleteIndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)6 UpdateSettingsRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest)6 DeleteIndexTemplateRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest)6 IndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest)6 PutIndexTemplateRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest)6 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)5 Collection (java.util.Collection)5 List (java.util.List)5 Map (java.util.Map)5 ClusterUpdateSettingsRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest)5 BulkRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest)5 ElasticsearchClient (org.graylog.storage.elasticsearch7.ElasticsearchClient)5