Search in sources :

Example 11 with TermsAggregation

use of io.searchbox.core.search.aggregation.TermsAggregation in project graylog2-server by Graylog2.

the class IndicesAdapterES6 method indexRangeStatsOfIndex.

@Override
public IndexRangeStats indexRangeStatsOfIndex(String index) {
    final FilterAggregationBuilder builder = AggregationBuilders.filter("agg", QueryBuilders.existsQuery(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.min("ts_min").field(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.max("ts_max").field(Message.FIELD_TIMESTAMP)).subAggregation(AggregationBuilders.terms("streams").size(Integer.MAX_VALUE).field(Message.FIELD_STREAMS));
    final String query = searchSource().aggregation(builder).size(0).toString();
    final Search request = new Search.Builder(query).addIndex(index).setSearchType(SearchType.DFS_QUERY_THEN_FETCH).ignoreUnavailable(true).build();
    if (LOG.isDebugEnabled()) {
        String data = "{}";
        try {
            data = request.getData(objectMapper.copy().enable(SerializationFeature.INDENT_OUTPUT));
        } catch (IOException e) {
            LOG.debug("Couldn't pretty print request payload", e);
        }
        LOG.debug("Index range query: _search/{}: {}", index, data);
    }
    final SearchResult result = JestUtils.execute(jestClient, request, () -> "Couldn't build index range of index " + index);
    final FilterAggregation f = result.getAggregations().getFilterAggregation("agg");
    if (f == null) {
        throw new IndexNotFoundException("Couldn't build index range of index " + index + " because it doesn't exist.");
    } else if (f.getCount() == 0L) {
        LOG.debug("No documents with attribute \"timestamp\" found in index <{}>", index);
        return IndexRangeStats.EMPTY;
    }
    final MinAggregation minAgg = f.getMinAggregation("ts_min");
    final DateTime min = new DateTime(minAgg.getMin().longValue(), DateTimeZone.UTC);
    final MaxAggregation maxAgg = f.getMaxAggregation("ts_max");
    final DateTime max = new DateTime(maxAgg.getMax().longValue(), DateTimeZone.UTC);
    // make sure we return an empty list, so we can differentiate between old indices that don't have this information
    // and newer ones that simply have no streams.
    final TermsAggregation streams = f.getTermsAggregation("streams");
    final List<String> streamIds = streams.getBuckets().stream().map(TermsAggregation.Entry::getKeyAsString).collect(toList());
    return IndexRangeStats.create(min, max, streamIds);
}
Also used : TermsAggregation(io.searchbox.core.search.aggregation.TermsAggregation) FilterAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) FieldSortBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.sort.FieldSortBuilder) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) FilterAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.bucket.filter.FilterAggregationBuilder) SearchResult(io.searchbox.core.SearchResult) MinAggregation(io.searchbox.core.search.aggregation.MinAggregation) IOException(java.io.IOException) MaxAggregation(io.searchbox.core.search.aggregation.MaxAggregation) DateTime(org.joda.time.DateTime) Search(io.searchbox.core.Search) IndexNotFoundException(org.graylog2.indexer.IndexNotFoundException) FilterAggregation(io.searchbox.core.search.aggregation.FilterAggregation)

Aggregations

TermsAggregation (io.searchbox.core.search.aggregation.TermsAggregation)11 SearchResult (io.searchbox.core.SearchResult)7 MetricAggregation (io.searchbox.core.search.aggregation.MetricAggregation)7 ArrayList (java.util.ArrayList)5 TagTypeIndexSearchResponseDto (org.finra.herd.model.dto.TagTypeIndexSearchResponseDto)4 Search (io.searchbox.core.Search)3 AbstractDaoTest (org.finra.herd.dao.AbstractDaoTest)3 ResultTypeIndexSearchResponseDto (org.finra.herd.model.dto.ResultTypeIndexSearchResponseDto)3 SearchSourceBuilder (org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder)3 Test (org.junit.Test)3 FilterAggregation (io.searchbox.core.search.aggregation.FilterAggregation)2 IOException (java.io.IOException)2 HashMap (java.util.HashMap)2 List (java.util.List)2 Map (java.util.Map)2 TagIndexSearchResponseDto (org.finra.herd.model.dto.TagIndexSearchResponseDto)2 ImmutableMap (com.google.common.collect.ImmutableMap)1 Gson (com.google.gson.Gson)1 JsonElement (com.google.gson.JsonElement)1 JsonObject (com.google.gson.JsonObject)1