Search in sources :

Example 36 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method indexRangeStatsOfIndex.

@Override
public IndexRangeStats indexRangeStatsOfIndex(String index) {
    final FilterAggregationBuilder builder = AggregationBuilders.filter("agg", QueryBuilders.existsQuery(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.min("ts_min").field(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.max("ts_max").field(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.terms("streams").size(Integer.MAX_VALUE).field(Message.FIELD_STREAMS));
    final SearchSourceBuilder query = SearchSourceBuilder.searchSource().aggregation(builder).size(0);
    final SearchRequest request = new SearchRequest().source(query).indices(index).searchType(SearchType.DFS_QUERY_THEN_FETCH).indicesOptions(IndicesOptions.lenientExpandOpen());
    final SearchResponse result = client.execute((c, requestOptions) -> c.search(request, requestOptions), "Couldn't build index range of index " + index);
    if (result.getTotalShards() == 0 || result.getAggregations() == null) {
        throw new IndexNotFoundException("Couldn't build index range of index " + index + " because it doesn't exist.");
    }
    final Filter f = result.getAggregations().get("agg");
    if (f == null) {
        throw new IndexNotFoundException("Couldn't build index range of index " + index + " because it doesn't exist.");
    } else if (f.getDocCount() == 0L) {
        LOG.debug("No documents with attribute \"timestamp\" found in index <{}>", index);
        return IndexRangeStats.EMPTY;
    }
    final Min minAgg = f.getAggregations().get("ts_min");
    final long minUnixTime = new Double(minAgg.getValue()).longValue();
    final DateTime min = new DateTime(minUnixTime, DateTimeZone.UTC);
    final Max maxAgg = f.getAggregations().get("ts_max");
    final long maxUnixTime = new Double(maxAgg.getValue()).longValue();
    final DateTime max = new DateTime(maxUnixTime, DateTimeZone.UTC);
    // make sure we return an empty list, so we can differentiate between old indices that don't have this information
    // and newer ones that simply have no streams.
    final Terms streams = f.getAggregations().get("streams");
    final List<String> streamIds = streams.getBuckets().stream().map(MultiBucketsAggregation.Bucket::getKeyAsString).collect(toList());
    return IndexRangeStats.create(min, max, streamIds);
}
Also used : SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) FilterAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) Max(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Max) Terms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.Terms) DateTime(org.joda.time.DateTime) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) Min(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Min) Filter(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.Filter) MultiBucketsAggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation) IndexNotFoundException(org.graylog2.indexer.IndexNotFoundException)

Example 37 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method openIndex.

@Override
public void openIndex(String index) {
    final OpenIndexRequest request = new OpenIndexRequest(index);
    client.execute((c, requestOptions) -> c.indices().open(request, requestOptions), "Unable to open index " + index);
}
Also used : OpenIndexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.open.OpenIndexRequest)

Example 38 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method move.

@Override
public void move(String source, String target, Consumer<IndexMoveResult> resultCallback) {
    final ReindexRequest request = new ReindexRequest();
    request.setSourceIndices(source);
    request.setDestIndex(target);
    final BulkByScrollResponse result = client.execute((c, requestOptions) -> c.reindex(request, requestOptions));
    final IndexMoveResult indexMoveResult = IndexMoveResult.create(Math.toIntExact(result.getTotal()), result.getTook().millis(), !result.getBulkFailures().isEmpty());
    resultCallback.accept(indexMoveResult);
}
Also used : ReindexRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.reindex.ReindexRequest) IndexMoveResult(org.graylog2.indexer.indices.IndexMoveResult) BulkByScrollResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.index.reindex.BulkByScrollResponse)

Example 39 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method removeAliases.

@Override
public void removeAliases(Set<String> indices, String alias) {
    final IndicesAliasesRequest indicesAliasesRequest = new IndicesAliasesRequest();
    final IndicesAliasesRequest.AliasActions aliasAction = IndicesAliasesRequest.AliasActions.remove().alias(alias).indices(indices.toArray(new String[0]));
    indicesAliasesRequest.addAliasAction(aliasAction);
    client.execute((c, requestOptions) -> c.indices().updateAliases(indicesAliasesRequest, requestOptions), "Couldn't remove alias " + alias + " from indices " + indices);
}
Also used : IndicesAliasesRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest)

Example 40 with RequestOptions

use of org.graylog.shaded.elasticsearch7.org.elasticsearch.client.RequestOptions in project graylog2-server by Graylog2.

the class IndicesAdapterES7 method flush.

@Override
public void flush(String index) {
    final FlushRequest request = new FlushRequest(index);
    client.execute((c, requestOptions) -> c.indices().flush(request, requestOptions), "Unable to flush index " + index);
}
Also used : FlushRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.flush.FlushRequest)

Aggregations

IndicesAliasesRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.alias.IndicesAliasesRequest)9 JsonNode (com.fasterxml.jackson.databind.JsonNode)8 Response (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Response)8 CreateIndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.CreateIndexRequest)8 Collectors (java.util.stream.Collectors)7 ClusterHealthRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest)7 Request (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.Request)7 Inject (javax.inject.Inject)6 DeleteIndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest)6 UpdateSettingsRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.settings.put.UpdateSettingsRequest)6 DeleteIndexTemplateRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.indices.template.delete.DeleteIndexTemplateRequest)6 IndexRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.index.IndexRequest)6 PutIndexTemplateRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.client.indices.PutIndexTemplateRequest)6 ObjectMapper (com.fasterxml.jackson.databind.ObjectMapper)5 Collection (java.util.Collection)5 List (java.util.List)5 Map (java.util.Map)5 ClusterUpdateSettingsRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.admin.cluster.settings.ClusterUpdateSettingsRequest)5 BulkRequest (org.graylog.shaded.elasticsearch7.org.elasticsearch.action.bulk.BulkRequest)5 ElasticsearchClient (org.graylog.storage.elasticsearch7.ElasticsearchClient)5