Search in sources :

Example 1 with Pivot

use of org.graylog.plugins.views.search.searchtypes.pivot.Pivot in project graylog2-server by Graylog2.

the class ESPivot method processSeries.

private void processSeries(PivotResult.Row.Builder rowBuilder, SearchResult searchResult, ESGeneratedQueryContext queryContext, Pivot pivot, ArrayDeque<String> columnKeys, MetricAggregation aggregation, boolean rollup, String source) {
    pivot.series().forEach(seriesSpec -> {
        final ESPivotSeriesSpecHandler<? extends SeriesSpec, ? extends Aggregation> seriesHandler = seriesHandlers.get(seriesSpec.type());
        final Aggregation series = seriesHandler.extractAggregationFromResult(pivot, seriesSpec, aggregation, queryContext);
        seriesHandler.handleResult(pivot, seriesSpec, searchResult, series, this, queryContext).map(value -> {
            columnKeys.addLast(value.id());
            final PivotResult.Value v = PivotResult.Value.create(columnKeys, value.value(), rollup, source);
            columnKeys.removeLast();
            return v;
        }).forEach(rowBuilder::addValue);
    });
}
Also used : Aggregation(io.searchbox.core.search.aggregation.Aggregation) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) ESGeneratedQueryContext(org.graylog.storage.elasticsearch6.views.ESGeneratedQueryContext) ESSearchTypeHandler(org.graylog.storage.elasticsearch6.views.searchtypes.ESSearchTypeHandler) MinAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.metrics.min.MinAggregationBuilder) DateTimeZone(org.joda.time.DateTimeZone) InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Query(org.graylog.plugins.views.search.Query) PivotResult(org.graylog.plugins.views.search.searchtypes.pivot.PivotResult) LoggerFactory(org.slf4j.LoggerFactory) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) Inject(javax.inject.Inject) Tuple2(org.jooq.lambda.tuple.Tuple2) ImmutableList(com.google.common.collect.ImmutableList) SearchType(org.graylog.plugins.views.search.SearchType) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) Map(java.util.Map) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) MaxAggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.metrics.max.MaxAggregationBuilder) AggregationBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilder) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) PivotSpec(org.graylog.plugins.views.search.searchtypes.pivot.PivotSpec) SearchJob(org.graylog.plugins.views.search.SearchJob) Logger(org.slf4j.Logger) IdentityHashMap(java.util.IdentityHashMap) Iterator(java.util.Iterator) DateTime(org.joda.time.DateTime) Aggregation(io.searchbox.core.search.aggregation.Aggregation) EntryStream(one.util.streamex.EntryStream) List(java.util.List) Tuple(org.jooq.lambda.tuple.Tuple) Stream(java.util.stream.Stream) AggregationBuilders(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.aggregations.AggregationBuilders) MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) ArrayDeque(java.util.ArrayDeque) SearchSourceBuilder(org.graylog.shaded.elasticsearch6.org.elasticsearch.search.builder.SearchSourceBuilder) SearchResult(io.searchbox.core.SearchResult)

Example 2 with Pivot

use of org.graylog.plugins.views.search.searchtypes.pivot.Pivot in project graylog2-server by Graylog2.

the class PivotAggregationSearch method getSourceStreamsQuery.

/**
 * Returns the query to compute the sources streams for the aggregation.
 *
 * @param parameters processor parameters
 * @return source streams query
 */
private Query getSourceStreamsQuery(AggregationEventProcessorParameters parameters) {
    final Pivot pivot = Pivot.builder().id(STREAMS_PIVOT_ID).rollup(true).rowGroups(ImmutableList.of(Values.builder().limit(Integer.MAX_VALUE).field("streams").build())).series(ImmutableList.of(Count.builder().id(STREAMS_PIVOT_COUNT_ID).build())).build();
    final Set<SearchType> searchTypes = Collections.singleton(pivot);
    final Query.Builder queryBuilder = Query.builder().id(STREAMS_QUERY_ID).searchTypes(searchTypes).query(ElasticsearchQueryString.of(config.query())).timerange(parameters.timerange());
    final Set<String> streams = getStreams(parameters);
    if (!streams.isEmpty()) {
        queryBuilder.filter(filteringForStreamIds(streams));
    }
    return queryBuilder.build();
}
Also used : Query(org.graylog.plugins.views.search.Query) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) SearchType(org.graylog.plugins.views.search.SearchType) ElasticsearchQueryString(org.graylog.plugins.views.search.elasticsearch.ElasticsearchQueryString)

Example 3 with Pivot

use of org.graylog.plugins.views.search.searchtypes.pivot.Pivot in project graylog2-server by Graylog2.

the class CommandFactoryTest method throwsIfSearchTypeIsNotMessageList.

@Test
void throwsIfSearchTypeIsNotMessageList() {
    Pivot p = Pivot.builder().id("pivot-id").series(newArrayList()).rollup(false).build();
    Query q = org.graylog.plugins.views.search.TestData.validQueryBuilder().searchTypes(ImmutableSet.of(p)).build();
    Search s = searchWithQueries(q);
    assertThatExceptionOfType(ExportException.class).isThrownBy(() -> sut.buildWithMessageList(s, p.id(), ResultFormat.builder().build())).withMessageContaining("supported");
}
Also used : Query(org.graylog.plugins.views.search.Query) Search(org.graylog.plugins.views.search.Search) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) Test(org.junit.jupiter.api.Test)

Example 4 with Pivot

use of org.graylog.plugins.views.search.searchtypes.pivot.Pivot in project graylog2-server by Graylog2.

the class ESPivotTest method includesCustomNameinResultIfPresent.

@Test
public void includesCustomNameinResultIfPresent() throws InvalidRangeParametersException {
    final ESPivot esPivot = new ESPivot(Collections.emptyMap(), Collections.emptyMap());
    final Pivot pivot = Pivot.builder().id("somePivotId").name("customPivot").series(Collections.emptyList()).rollup(false).build();
    final long documentCount = 424242;
    when(queryResult.getTotal()).thenReturn(documentCount);
    final MetricAggregation mockMetricAggregation = createTimestampRangeAggregations((double) new Date().getTime(), (double) new Date().getTime());
    when(queryResult.getAggregations()).thenReturn(mockMetricAggregation);
    when(query.effectiveTimeRange(pivot)).thenReturn(RelativeRange.create(300));
    final SearchType.Result result = esPivot.doExtractResult(null, query, pivot, queryResult, null, null);
    assertThat(result.name()).contains("customPivot");
}
Also used : MetricAggregation(io.searchbox.core.search.aggregation.MetricAggregation) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) ESPivot(org.graylog.storage.elasticsearch6.views.searchtypes.pivot.ESPivot) SearchType(org.graylog.plugins.views.search.SearchType) ESPivot(org.graylog.storage.elasticsearch6.views.searchtypes.pivot.ESPivot) Date(java.util.Date) Test(org.junit.Test)

Example 5 with Pivot

use of org.graylog.plugins.views.search.searchtypes.pivot.Pivot in project graylog2-server by Graylog2.

the class ESPivot method processSeries.

private void processSeries(PivotResult.Row.Builder rowBuilder, SearchResponse searchResult, ESGeneratedQueryContext queryContext, Pivot pivot, ArrayDeque<String> columnKeys, HasAggregations aggregation, boolean rollup, String source) {
    pivot.series().forEach(seriesSpec -> {
        final ESPivotSeriesSpecHandler<? extends SeriesSpec, ? extends Aggregation> seriesHandler = seriesHandlers.get(seriesSpec.type());
        final Aggregation series = seriesHandler.extractAggregationFromResult(pivot, seriesSpec, aggregation, queryContext);
        seriesHandler.handleResult(pivot, seriesSpec, searchResult, series, this, queryContext).map(value -> {
            columnKeys.addLast(value.id());
            final PivotResult.Value v = PivotResult.Value.create(columnKeys, value.value(), rollup, source);
            columnKeys.removeLast();
            return v;
        }).forEach(rowBuilder::addValue);
    });
}
Also used : Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) HasAggregations(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.HasAggregations) ESSearchTypeHandler(org.graylog.storage.elasticsearch7.views.searchtypes.ESSearchTypeHandler) SearchResponse(org.graylog.shaded.elasticsearch7.org.elasticsearch.action.search.SearchResponse) DateTimeZone(org.joda.time.DateTimeZone) InvalidRangeParametersException(org.graylog2.plugin.indexer.searches.timeranges.InvalidRangeParametersException) Query(org.graylog.plugins.views.search.Query) PivotResult(org.graylog.plugins.views.search.searchtypes.pivot.PivotResult) LoggerFactory(org.slf4j.LoggerFactory) RelativeRange(org.graylog2.plugin.indexer.searches.timeranges.RelativeRange) AggregationBuilders(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilders) Inject(javax.inject.Inject) Tuple2(org.jooq.lambda.tuple.Tuple2) ImmutableList(com.google.common.collect.ImmutableList) SearchType(org.graylog.plugins.views.search.SearchType) AggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.AggregationBuilder) BucketSpec(org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec) SeriesSpec(org.graylog.plugins.views.search.searchtypes.pivot.SeriesSpec) Map(java.util.Map) ESGeneratedQueryContext(org.graylog.storage.elasticsearch7.views.ESGeneratedQueryContext) AbsoluteRange(org.graylog2.plugin.indexer.searches.timeranges.AbsoluteRange) Aggregation(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregation) Pivot(org.graylog.plugins.views.search.searchtypes.pivot.Pivot) TimeRange(org.graylog2.plugin.indexer.searches.timeranges.TimeRange) PivotSpec(org.graylog.plugins.views.search.searchtypes.pivot.PivotSpec) SearchJob(org.graylog.plugins.views.search.SearchJob) SearchSourceBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.builder.SearchSourceBuilder) Logger(org.slf4j.Logger) IdentityHashMap(java.util.IdentityHashMap) Iterator(java.util.Iterator) DateTime(org.joda.time.DateTime) EntryStream(one.util.streamex.EntryStream) Max(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Max) MaxAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.MaxAggregationBuilder) Aggregations(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.Aggregations) List(java.util.List) Tuple(org.jooq.lambda.tuple.Tuple) Stream(java.util.stream.Stream) Optional(java.util.Optional) Preconditions(com.google.common.base.Preconditions) MinAggregationBuilder(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.MinAggregationBuilder) ArrayDeque(java.util.ArrayDeque) Min(org.graylog.shaded.elasticsearch7.org.elasticsearch.search.aggregations.metrics.Min)

Aggregations

Pivot (org.graylog.plugins.views.search.searchtypes.pivot.Pivot)6 SearchType (org.graylog.plugins.views.search.SearchType)5 Query (org.graylog.plugins.views.search.Query)4 Preconditions (com.google.common.base.Preconditions)2 ImmutableList (com.google.common.collect.ImmutableList)2 MetricAggregation (io.searchbox.core.search.aggregation.MetricAggregation)2 ArrayDeque (java.util.ArrayDeque)2 Date (java.util.Date)2 IdentityHashMap (java.util.IdentityHashMap)2 Iterator (java.util.Iterator)2 List (java.util.List)2 Map (java.util.Map)2 Optional (java.util.Optional)2 Stream (java.util.stream.Stream)2 Inject (javax.inject.Inject)2 EntryStream (one.util.streamex.EntryStream)2 SearchJob (org.graylog.plugins.views.search.SearchJob)2 BucketSpec (org.graylog.plugins.views.search.searchtypes.pivot.BucketSpec)2 PivotResult (org.graylog.plugins.views.search.searchtypes.pivot.PivotResult)2 PivotSpec (org.graylog.plugins.views.search.searchtypes.pivot.PivotSpec)2