Search in sources :

Example 56 with Aggregation

use of org.elasticsearch.search.aggregations.Aggregation in project graylog2-server by Graylog2.

the class ESPivot method processSeries.

private void processSeries(PivotResult.Row.Builder rowBuilder, SearchResponse searchResult, ESGeneratedQueryContext queryContext, Pivot pivot, ArrayDeque<String> columnKeys, HasAggregations aggregation, boolean rollup, String source) {
    pivot.series().forEach(seriesSpec -> {
        final ESPivotSeriesSpecHandler<? extends SeriesSpec, ? extends Aggregation> seriesHandler = seriesHandlers.get(seriesSpec.type());
        final Aggregation series = seriesHandler.extractAggregationFromResult(pivot, seriesSpec, aggregation, queryContext);
        seriesHandler.handleResult(pivot, seriesSpec, searchResult, series, this, queryContext).map(value -> {
            columnKeys.addLast(value.id());
            final PivotResult.Value v = PivotResult.Value.create(columnKeys, value.value(), rollup, source);
            columnKeys.removeLast();
            return v;
        }).forEach(rowBuilder::addValue);
    });
}
Also used : Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) HasAggregations(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.HasAggregations) ESSearchTypeHandler(org.graylog.storage.elasticsearch7.views.searchtypes.ESSearchTypeHandler) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) DateTimeZone(org.joda.time.DateTimeZone) InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Query(org.graylog.plugins.views.search.Query) PivotResult(org.graylog.plugins.views.search.searchtypes.pivot.PivotResult) LoggerFactory(org.slf4j.LoggerFactory) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) AggregationBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilders) Inject(javax.inject.Inject) Tuple2(org.jooq.lambda.tuple.Tuple2) ImmutableList(com.google.common.collect.ImmutableList) SearchType(org.graylog.plugins.views.search.SearchType) AggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilder) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) Map(java.util.Map) ESGeneratedQueryContext(org.graylog.storage.elasticsearch7.views.ESGeneratedQueryContext) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) PivotSpec(org.graylog.plugins.views.search.searchtypes.pivot.PivotSpec) SearchJob(org.graylog.plugins.views.search.SearchJob) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) Logger(org.slf4j.Logger) IdentityHashMap(java.util.IdentityHashMap) Iterator(java.util.Iterator) DateTime(org.joda.time.DateTime) EntryStream(one.util.streamex.EntryStream) Max(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Max) MaxAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.MaxAggregationBuilder) Aggregations(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregations) List(java.util.List) Tuple(org.jooq.lambda.tuple.Tuple) Stream(java.util.stream.Stream) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) MinAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.MinAggregationBuilder) ArrayDeque(java.util.ArrayDeque) Min(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Min)

Example 57 with Aggregation

use of org.elasticsearch.search.aggregations.Aggregation in project graylog2-server by Graylog2.

the class TestMultisearchResponse method getDefaultNamedXContents.

private static List<NamedXContentRegistry.Entry> getDefaultNamedXContents() {
    Map<String, ContextParser<Object, ? extends Aggregation>> map = new HashMap<>();
    map.put("cardinality", (p, c) -> ParsedCardinality.fromXContent(p, (String) c));
    map.put("percentiles_bucket", (p, c) -> ParsedPercentilesBucket.fromXContent(p, (String) c));
    map.put("median_absolute_deviation", (p, c) -> ParsedMedianAbsoluteDeviation.fromXContent(p, (String) c));
    map.put("min", (p, c) -> ParsedMin.fromXContent(p, (String) c));
    map.put("max", (p, c) -> ParsedMax.fromXContent(p, (String) c));
    map.put("sum", (p, c) -> ParsedSum.fromXContent(p, (String) c));
    map.put("avg", (p, c) -> ParsedAvg.fromXContent(p, (String) c));
    map.put("weighted_avg", (p, c) -> ParsedWeightedAvg.fromXContent(p, (String) c));
    map.put("value_count", (p, c) -> ParsedValueCount.fromXContent(p, (String) c));
    map.put("simple_value", (p, c) -> ParsedSimpleValue.fromXContent(p, (String) c));
    map.put("derivative", (p, c) -> ParsedDerivative.fromXContent(p, (String) c));
    map.put("bucket_metric_value", (p, c) -> ParsedBucketMetricValue.fromXContent(p, (String) c));
    map.put("stats", (p, c) -> ParsedStats.fromXContent(p, (String) c));
    map.put("stats_bucket", (p, c) -> ParsedStatsBucket.fromXContent(p, (String) c));
    map.put("extended_stats", (p, c) -> ParsedExtendedStats.fromXContent(p, (String) c));
    map.put("extended_stats_bucket", (p, c) -> ParsedExtendedStatsBucket.fromXContent(p, (String) c));
    map.put("histogram", (p, c) -> ParsedHistogram.fromXContent(p, (String) c));
    map.put("date_histogram", (p, c) -> ParsedDateHistogram.fromXContent(p, (String) c));
    map.put("auto_date_histogram", (p, c) -> ParsedAutoDateHistogram.fromXContent(p, (String) c));
    map.put("sterms", (p, c) -> ParsedStringTerms.fromXContent(p, (String) c));
    map.put("lterms", (p, c) -> ParsedLongTerms.fromXContent(p, (String) c));
    map.put("dterms", (p, c) -> ParsedDoubleTerms.fromXContent(p, (String) c));
    map.put("missing", (p, c) -> ParsedMissing.fromXContent(p, (String) c));
    map.put("nested", (p, c) -> ParsedNested.fromXContent(p, (String) c));
    map.put("reverse_nested", (p, c) -> ParsedReverseNested.fromXContent(p, (String) c));
    map.put("global", (p, c) -> ParsedGlobal.fromXContent(p, (String) c));
    map.put("filter", (p, c) -> ParsedFilter.fromXContent(p, (String) c));
    map.put("sampler", (p, c) -> ParsedSampler.fromXContent(p, (String) c));
    map.put("range", (p, c) -> ParsedRange.fromXContent(p, (String) c));
    map.put("date_range", (p, c) -> ParsedDateRange.fromXContent(p, (String) c));
    map.put("filters", (p, c) -> ParsedFilters.fromXContent(p, (String) c));
    map.put("top_hits", (p, c) -> ParsedTopHits.fromXContent(p, (String) c));
    map.put("composite", (p, c) -> ParsedComposite.fromXContent(p, (String) c));
    List<NamedXContentRegistry.Entry> entries = map.entrySet().stream().map((entry) -> new NamedXContentRegistry.Entry(Aggregation.class, new ParseField(entry.getKey()), entry.getValue())).collect(Collectors.toList());
    entries.add(new NamedXContentRegistry.Entry(Suggest.Suggestion.class, new ParseField("term"), (parser, context) -> TermSuggestion.fromXContent(parser, (String) context)));
    entries.add(new NamedXContentRegistry.Entry(Suggest.Suggestion.class, new ParseField("phrase"), (parser, context) -> PhraseSuggestion.fromXContent(parser, (String) context)));
    entries.add(new NamedXContentRegistry.Entry(Suggest.Suggestion.class, new ParseField("completion"), (parser, context) -> CompletionSuggestion.fromXContent(parser, (String) context)));
    return entries;
}
Also used : Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) ParsedDateHistogram(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.histogram.ParsedDateHistogram) ParsedDateRange(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.range.ParsedDateRange) URL(java.net.URL) ParsedAvg(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedAvg) ParsedMin(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedMin) ParsedSimpleValue(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedSimpleValue) URISyntaxException(java.net.URISyntaxException) ParsedPercentilesBucket(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedPercentilesBucket) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) ParsedRange(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.range.ParsedRange) MultiSearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.MultiSearchResponse) Map(java.util.Map) ParsedValueCount(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedValueCount) Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) ParsedCardinality(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedCardinality) Path(java.nio.file.Path) ParsedSum(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedSum) ParsedNested(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.nested.ParsedNested) ParsedStringTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedStringTerms) ContextParser(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.ContextParser) ParsedFilters(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.ParsedFilters) ParseField(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.ParseField) Collectors(java.util.stream.Collectors) UncheckedIOException(java.io.UncheckedIOException) List(java.util.List) CompletionSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.completion.CompletionSuggestion) ParsedAutoDateHistogram(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.histogram.ParsedAutoDateHistogram) ParsedMax(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedMax) NamedXContentRegistry(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.NamedXContentRegistry) ParsedDerivative(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedDerivative) ParsedReverseNested(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.nested.ParsedReverseNested) ParsedMedianAbsoluteDeviation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedMedianAbsoluteDeviation) ParsedStats(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedStats) ParsedSampler(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.sampler.ParsedSampler) ParsedFilter(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.filter.ParsedFilter) PhraseSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.phrase.PhraseSuggestion) Suggest(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.Suggest) JsonXContent(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.json.JsonXContent) ParsedBucketMetricValue(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedBucketMetricValue) HashMap(java.util.HashMap) ParsedExtendedStatsBucket(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedExtendedStatsBucket) ParsedStatsBucket(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.pipeline.ParsedStatsBucket) LoggingDeprecationHandler(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.LoggingDeprecationHandler) ParsedWeightedAvg(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedWeightedAvg) Resources(com.google.common.io.Resources) Files(java.nio.file.Files) IOException(java.io.IOException) ParsedLongTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedLongTerms) ParsedMissing(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.missing.ParsedMissing) XContentParser(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.XContentParser) ParsedTopHits(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedTopHits) ParsedExtendedStats(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.ParsedExtendedStats) Paths(java.nio.file.Paths) ParsedHistogram(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.histogram.ParsedHistogram) ParsedGlobal(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.global.ParsedGlobal) ParsedDoubleTerms(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.terms.ParsedDoubleTerms) ParsedComposite(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.bucket.composite.ParsedComposite) InputStream(java.io.InputStream) TermSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.term.TermSuggestion) CompletionSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.completion.CompletionSuggestion) PhraseSuggestion(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.suggest.phrase.PhraseSuggestion) ContextParser(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.ContextParser) HashMap(java.util.HashMap) NamedXContentRegistry(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.xcontent.NamedXContentRegistry) ParseField(org.graylog.shaded.elasticsearch7.org.elasticsearch.common.ParseField)

Example 58 with Aggregation

use of org.elasticsearch.search.aggregations.Aggregation in project graylog2-server by Graylog2.

the class ElasticsearchBackendGeneratedRequestTestBase method setUpSUT.

@Before
public void setUpSUT() {
    this.elasticSearchTypeHandlers = new HashMap<>();
    final Map<String, ESPivotBucketSpecHandler<? extends BucketSpec, ? extends Aggregation>> bucketHandlers = Collections.emptyMap();
    final Map<String, ESPivotSeriesSpecHandler<? extends SeriesSpec, ? extends Aggregation>> seriesHandlers = new HashMap<>();
    seriesHandlers.put(Average.NAME, new ESAverageHandler());
    seriesHandlers.put(Max.NAME, new ESMaxHandler());
    elasticSearchTypeHandlers.put(Pivot.NAME, () -> new ESPivot(bucketHandlers, seriesHandlers));
    this.elasticsearchBackend = new ElasticsearchBackend(elasticSearchTypeHandlers, client, indexLookup, new QueryStringDecorators.Fake(), (elasticsearchBackend, ssb, job, query) -> new ESGeneratedQueryContext(elasticsearchBackend, ssb, job, query, fieldTypesLookup), false);
}
Also used : ESSearchTypeHandler(org.graylog.storage.elasticsearch7.views.searchtypes.ESSearchTypeHandler) ArgumentMatchers.any(org.mockito.ArgumentMatchers.any) ElasticsearchClient(org.graylog.storage.elasticsearch7.ElasticsearchClient) InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Provider(javax.inject.Provider) Query(org.graylog.plugins.views.search.Query) Mock(org.mockito.Mock) HashMap(java.util.HashMap) Captor(org.mockito.Captor) Max(org.graylog.plugins.views.search.searchtypes.pivot.series.Max) ESPivotSeriesSpecHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivotSeriesSpecHandler) ArgumentCaptor(org.mockito.ArgumentCaptor) SearchType(org.graylog.plugins.views.search.SearchType) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) Map(java.util.Map) FieldTypesLookup(org.graylog.plugins.views.search.elasticsearch.FieldTypesLookup) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) MockitoJUnit(org.mockito.junit.MockitoJUnit) Search(org.graylog.plugins.views.search.Search) Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) QueryResult(org.graylog.plugins.views.search.QueryResult) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) Before(org.junit.Before) SearchJob(org.graylog.plugins.views.search.SearchJob) ImmutableSet(com.google.common.collect.ImmutableSet) SearchRequest(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchRequest) ESMaxHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.series.ESMaxHandler) ESPivotBucketSpecHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivotBucketSpecHandler) Set(java.util.Set) Mockito.times(org.mockito.Mockito.times) ESAverageHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.series.ESAverageHandler) Mockito.verify(org.mockito.Mockito.verify) List(java.util.List) QueryStringDecorators(org.graylog.plugins.views.search.elasticsearch.QueryStringDecorators) Rule(org.junit.Rule) ESPivot(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivot) MockitoRule(org.mockito.junit.MockitoRule) Average(org.graylog.plugins.views.search.searchtypes.pivot.series.Average) IndexLookup(org.graylog.plugins.views.search.elasticsearch.IndexLookup) Collections(java.util.Collections) ESPivotSeriesSpecHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivotSeriesSpecHandler) HashMap(java.util.HashMap) ESMaxHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.series.ESMaxHandler) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) ESAverageHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.series.ESAverageHandler) ESPivotBucketSpecHandler(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivotBucketSpecHandler) ESPivot(org.graylog.storage.elasticsearch7.views.searchtypes.pivot.ESPivot) Before(org.junit.Before)

Aggregations

Aggregation (org.elasticsearch.search.aggregations.Aggregation)53 HashMap (java.util.HashMap)24 SearchResponse (org.elasticsearch.action.search.SearchResponse)24 Terms (org.elasticsearch.search.aggregations.bucket.terms.Terms)19 List (java.util.List)18 ArrayList (java.util.ArrayList)17 MultiBucketsAggregation (org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation)17 ElasticsearchAssertions.assertSearchResponse (org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertSearchResponse)16 Script (org.elasticsearch.script.Script)13 ScriptedMetric (org.elasticsearch.search.aggregations.metrics.scripted.ScriptedMetric)13 Map (java.util.Map)10 StringTerms (org.elasticsearch.search.aggregations.bucket.terms.StringTerms)7 Aggregation (org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation)5 VertexiumException (org.vertexium.VertexiumException)5 FacetDefinition (io.vertigo.dynamo.collections.metamodel.FacetDefinition)4 IOException (java.io.IOException)4 SearchHit (org.elasticsearch.search.SearchHit)4 SearchHits (org.elasticsearch.search.SearchHits)4 Bucket (org.elasticsearch.search.aggregations.bucket.MultiBucketsAggregation.Bucket)4 BucketSpec (org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec)4