Search in sources :

Example 66 with Response

use of org.elasticsearch.client.Response in project storm by apache.

the class EsState method updateState.

/**
 * Store current state to ElasticSearch.
 *
 * @param tuples list of tuples for storing to ES.
 *               Each tuple should have relevant fields (source, index, type, id) for EsState's tupleMapper to extract ES document.
 */
public void updateState(List<TridentTuple> tuples) {
    try {
        String bulkRequest = buildRequest(tuples);
        Response response = client.performRequest("post", "_bulk", new HashMap<>(), new StringEntity(bulkRequest.toString()));
        BulkIndexResponse bulkResponse = objectMapper.readValue(response.getEntity().getContent(), BulkIndexResponse.class);
        if (bulkResponse.hasErrors()) {
            LOG.warn("failed processing bulk index requests: " + bulkResponse.getFirstError() + ": " + bulkResponse.getFirstResult());
            throw new FailedException();
        }
    } catch (IOException e) {
        LOG.warn("failed processing bulk index requests: " + e.toString());
        throw new FailedException(e);
    }
}
Also used : BulkIndexResponse(org.apache.storm.elasticsearch.response.BulkIndexResponse) Response(org.elasticsearch.client.Response) StringEntity(org.apache.http.entity.StringEntity) BulkIndexResponse(org.apache.storm.elasticsearch.response.BulkIndexResponse) FailedException(org.apache.storm.topology.FailedException) IOException(java.io.IOException)

Example 67 with Response

use of org.elasticsearch.client.Response in project storm by apache.

the class EsPercolateBolt method process.

/**
 * {@inheritDoc}
 * Tuple should have relevant fields (source, index, type) for storeMapper to extract ES document.<br/>
 * If there exists non-empty percolate response, EsPercolateBolt will emit tuple with original source
 * and Percolate.Match for each Percolate.Match in PercolateResponse.
 */
@Override
public void process(Tuple tuple) {
    try {
        String source = tupleMapper.getSource(tuple);
        String index = tupleMapper.getIndex(tuple);
        String type = tupleMapper.getType(tuple);
        Map<String, String> indexParams = new HashMap<>();
        indexParams.put(type, null);
        String percolateDoc = "{\"doc\": " + source + "}";
        Response response = client.performRequest("get", getEndpoint(index, type, "_percolate"), new HashMap<>(), new StringEntity(percolateDoc));
        PercolateResponse percolateResponse = objectMapper.readValue(response.getEntity().getContent(), PercolateResponse.class);
        if (!percolateResponse.getMatches().isEmpty()) {
            for (PercolateResponse.Match match : percolateResponse.getMatches()) {
                collector.emit(new Values(source, match));
            }
        }
        collector.ack(tuple);
    } catch (Exception e) {
        collector.reportError(e);
        collector.fail(tuple);
    }
}
Also used : PercolateResponse(org.apache.storm.elasticsearch.response.PercolateResponse) Response(org.elasticsearch.client.Response) StringEntity(org.apache.http.entity.StringEntity) HashMap(java.util.HashMap) Values(org.apache.storm.tuple.Values) PercolateResponse(org.apache.storm.elasticsearch.response.PercolateResponse)

Example 68 with Response

use of org.elasticsearch.client.Response in project graylog2-server by Graylog2.

the class PlainJsonApi method perform.

public JsonNode perform(Request request, String errorMessage) {
    return client.execute((c, requestOptions) -> {
        request.setOptions(requestOptions);
        final Response response = c.getLowLevelClient().performRequest(request);
        return objectMapper.readTree(response.getEntity().getContent());
    }, errorMessage);
}
Also used : Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response)

Example 69 with Response

use of org.elasticsearch.client.Response in project graylog2-server by Graylog2.

the class ClientES7 method existingIndices.

private String[] existingIndices() {
    final Request request = new Request("GET", "/_cat/indices");
    request.addParameter("format", "json");
    request.addParameter("h", "index");
    final JsonNode jsonResponse = client.execute((c, requestOptions) -> {
        request.setOptions(requestOptions);
        final Response response = c.getLowLevelClient().performRequest(request);
        return objectMapper.readTree(response.getEntity().getContent());
    });
    return Streams.stream(jsonResponse.elements()).map(index -> index.path("index").asText()).distinct().toArray(String[]::new);
}
Also used : Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) GetIndexTemplatesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesResponse) IndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest) WriteRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.WriteRequest) CloseIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CloseIndexRequest) PutIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest) GetIndexTemplatesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesRequest) RefreshRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.refresh.RefreshRequest) GetIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexRequest) BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) DeleteIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) ClusterHealthRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) DeleteIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest) UpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) ClusterUpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest) IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) JsonNode(com.fasterxml.jackson.databind.JsonNode)

Example 70 with Response

use of org.elasticsearch.client.Response in project graylog2-server by Graylog2.

the class ClientES7 method getMapping.

private Map<String, String> getMapping(String index) {
    final Request request = new Request("GET", "/" + index + "/_mapping");
    final JsonNode response = client.execute((c, requestOptions) -> {
        request.setOptions(requestOptions);
        final Response result = c.getLowLevelClient().performRequest(request);
        return objectMapper.readTree(result.getEntity().getContent());
    });
    return extractFieldMappings(index, response).collect(Collectors.toMap(Map.Entry::getKey, entry -> entry.getValue().path("type").asText()));
}
Also used : Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) GetIndexTemplatesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesResponse) ElasticsearchClient(org.graylog.storage.elasticsearch7.ElasticsearchClient) IndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest) Retryer(com.github.rholder.retry.Retryer) RetryerBuilder(com.github.rholder.retry.RetryerBuilder) LoggerFactory(org.slf4j.LoggerFactory) Callable(java.util.concurrent.Callable) WriteRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.WriteRequest) CloseIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CloseIndexRequest) ClusterHealthStatus(org.graylog.shaded.elasticsearch7.org.elasticsearch.cluster.health.ClusterHealthStatus) PutIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest) GetIndexTemplatesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesRequest) RefreshRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.refresh.RefreshRequest) Map(java.util.Map) JsonNode(com.fasterxml.jackson.databind.JsonNode) AliasActions(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest.AliasActions) Response(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response) WaitStrategies(com.github.rholder.retry.WaitStrategies) GetIndexTemplatesResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesResponse) GetIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexRequest) BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) Logger(org.slf4j.Logger) ObjectMapper(com.fasterxml.jackson.databind.ObjectMapper) DeleteIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) Client(org.graylog.testing.elasticsearch.Client) ObjectMapperProvider(org.graylog2.shared.bindings.providers.ObjectMapperProvider) Streams(com.google.common.collect.Streams) ClusterHealthRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) Collectors(java.util.stream.Collectors) DeleteIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest) UpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) TimeUnit(java.util.concurrent.TimeUnit) ClusterUpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest) Stream(java.util.stream.Stream) StopStrategies(com.github.rholder.retry.StopStrategies) Settings(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.settings.Settings) IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) IndexTemplateMetadata(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.IndexTemplateMetadata) Collections(java.util.Collections) IndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest) WriteRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.support.WriteRequest) CloseIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CloseIndexRequest) PutIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest) GetIndexTemplatesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexTemplatesRequest) RefreshRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.refresh.RefreshRequest) GetIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.GetIndexRequest) BulkIndexRequest(org.graylog.testing.elasticsearch.BulkIndexRequest) DeleteIndexTemplateRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest) Request(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request) ClusterHealthRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest) DeleteIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest) CreateIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest) UpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest) ClusterUpdateSettingsRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest) IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest) BulkRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest) JsonNode(com.fasterxml.jackson.databind.JsonNode) Map(java.util.Map)

Aggregations

Response (org.elasticsearch.client.Response)75 IOException (java.io.IOException)24 Request (org.elasticsearch.client.Request)19 BasicHeader (org.apache.http.message.BasicHeader)14 NStringEntity (org.apache.http.nio.entity.NStringEntity)14 HttpEntity (org.apache.http.HttpEntity)13 ResponseException (org.elasticsearch.client.ResponseException)10 JsonNode (com.fasterxml.jackson.databind.JsonNode)9 HashMap (java.util.HashMap)9 RestClient (org.elasticsearch.client.RestClient)8 Map (java.util.Map)7 Response (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response)7 ArrayList (java.util.ArrayList)6 RestBulkItemResponse (org.janusgraph.diskstorage.es.rest.RestBulkResponse.RestBulkItemResponse)6 JSONArray (org.json.simple.JSONArray)6 JSONObject (org.json.simple.JSONObject)6 JSONParser (org.json.simple.parser.JSONParser)6 ParseException (org.json.simple.parser.ParseException)6 InputStream (java.io.InputStream)5 StringEntity (org.apache.http.entity.StringEntity)5