Search in sources :

Example 1 with InternalGlobal

use of org.opensearch.search.aggregations.bucket.global.InternalGlobal in project OpenSearch by opensearch-project.

the class TermsAggregatorTests method testGlobalAggregationWithScore.

public void testGlobalAggregationWithScore() throws IOException {
    try (Directory directory = newDirectory()) {
        try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) {
            Document document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("a")));
            indexWriter.addDocument(document);
            document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("c")));
            indexWriter.addDocument(document);
            document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("e")));
            indexWriter.addDocument(document);
            try (IndexReader indexReader = maybeWrapReaderEs(indexWriter.getReader())) {
                IndexSearcher indexSearcher = newIndexSearcher(indexReader);
                String executionHint = randomFrom(TermsAggregatorFactory.ExecutionMode.values()).toString();
                Aggregator.SubAggCollectionMode collectionMode = randomFrom(Aggregator.SubAggCollectionMode.values());
                GlobalAggregationBuilder globalBuilder = new GlobalAggregationBuilder("global").subAggregation(new TermsAggregationBuilder("terms").userValueTypeHint(ValueType.STRING).executionHint(executionHint).collectMode(collectionMode).field("keyword").order(BucketOrder.key(true)).subAggregation(new TermsAggregationBuilder("sub_terms").userValueTypeHint(ValueType.STRING).executionHint(executionHint).collectMode(collectionMode).field("keyword").order(BucketOrder.key(true)).subAggregation(new TopHitsAggregationBuilder("top_hits").storedField("_none_"))));
                MappedFieldType fieldType = new KeywordFieldMapper.KeywordFieldType("keyword");
                InternalGlobal result = searchAndReduce(indexSearcher, new MatchAllDocsQuery(), globalBuilder, fieldType);
                InternalMultiBucketAggregation<?, ?> terms = result.getAggregations().get("terms");
                assertThat(terms.getBuckets().size(), equalTo(3));
                for (MultiBucketsAggregation.Bucket bucket : terms.getBuckets()) {
                    InternalMultiBucketAggregation<?, ?> subTerms = bucket.getAggregations().get("sub_terms");
                    assertThat(subTerms.getBuckets().size(), equalTo(1));
                    MultiBucketsAggregation.Bucket subBucket = subTerms.getBuckets().get(0);
                    InternalTopHits topHits = subBucket.getAggregations().get("top_hits");
                    assertThat(topHits.getHits().getHits().length, equalTo(1));
                    for (SearchHit hit : topHits.getHits()) {
                        assertThat(hit.getScore(), greaterThan(0f));
                    }
                }
            }
        }
    }
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) InternalGlobal(org.opensearch.search.aggregations.bucket.global.InternalGlobal) SearchHit(org.opensearch.search.SearchHit) TopHitsAggregationBuilder(org.opensearch.search.aggregations.metrics.TopHitsAggregationBuilder) Aggregator(org.opensearch.search.aggregations.Aggregator) Document(org.apache.lucene.document.Document) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) GlobalAggregationBuilder(org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder) InternalTopHits(org.opensearch.search.aggregations.metrics.InternalTopHits) MultiBucketsAggregation(org.opensearch.search.aggregations.bucket.MultiBucketsAggregation) SortedDocValuesField(org.apache.lucene.document.SortedDocValuesField) IndexReader(org.apache.lucene.index.IndexReader) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) BytesRef(org.apache.lucene.util.BytesRef) Directory(org.apache.lucene.store.Directory)

Example 2 with InternalGlobal

use of org.opensearch.search.aggregations.bucket.global.InternalGlobal in project OpenSearch by opensearch-project.

the class MinAggregatorTests method testGetProperty.

public void testGetProperty() throws IOException {
    GlobalAggregationBuilder globalBuilder = new GlobalAggregationBuilder("global").subAggregation(new MinAggregationBuilder("min").field("number"));
    MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType("number", NumberFieldMapper.NumberType.INTEGER);
    testCase(globalBuilder, new MatchAllDocsQuery(), iw -> {
        iw.addDocument(singleton(new NumericDocValuesField("number", 7)));
        iw.addDocument(singleton(new NumericDocValuesField("number", 1)));
    }, (Consumer<InternalGlobal>) global -> {
        assertEquals(2, global.getDocCount());
        assertTrue(AggregationInspectionHelper.hasValue(global));
        assertNotNull(global.getAggregations().asMap().get("min"));
        InternalMin min = (InternalMin) global.getAggregations().asMap().get("min");
        assertEquals(1.0, min.getValue(), 0);
        assertThat(global.getProperty("min"), equalTo(min));
        assertThat(global.getProperty("min.value"), equalTo(1.0));
        assertThat(min.getProperty("value"), equalTo(1.0));
    }, fieldType);
}
Also used : MultiReader(org.apache.lucene.index.MultiReader) Query(org.apache.lucene.search.Query) NoMergePolicy(org.apache.lucene.index.NoMergePolicy) Arrays(java.util.Arrays) InternalHistogram(org.opensearch.search.aggregations.bucket.histogram.InternalHistogram) ScriptModule(org.opensearch.script.ScriptModule) ScriptEngine(org.opensearch.script.ScriptEngine) BiFunction(java.util.function.BiFunction) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) Term(org.apache.lucene.index.Term) ScriptType(org.opensearch.script.ScriptType) DoublePoint(org.apache.lucene.document.DoublePoint) Document(org.apache.lucene.document.Document) AggregationBuilder(org.opensearch.search.aggregations.AggregationBuilder) MapperService(org.opensearch.index.mapper.MapperService) Collections.singleton(java.util.Collections.singleton) Directory(org.apache.lucene.store.Directory) Map(java.util.Map) LeafReaderContext(org.apache.lucene.index.LeafReaderContext) MockScriptEngine(org.opensearch.script.MockScriptEngine) KeywordFieldMapper(org.opensearch.index.mapper.KeywordFieldMapper) AggregationInspectionHelper(org.opensearch.search.aggregations.support.AggregationInspectionHelper) QueryBuilders.termQuery(org.opensearch.index.query.QueryBuilders.termQuery) FilterAggregationBuilder(org.opensearch.search.aggregations.bucket.filter.FilterAggregationBuilder) ScriptService(org.opensearch.script.ScriptService) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) IpFieldMapper(org.opensearch.index.mapper.IpFieldMapper) SortedSetDocValuesField(org.apache.lucene.document.SortedSetDocValuesField) InetAddressPoint(org.apache.lucene.document.InetAddressPoint) BytesRef(org.apache.lucene.util.BytesRef) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) DirectoryReader(org.apache.lucene.index.DirectoryReader) Script(org.opensearch.script.Script) LongTerms(org.opensearch.search.aggregations.bucket.terms.LongTerms) GlobalAggregationBuilder(org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder) Settings(org.opensearch.common.settings.Settings) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) Terms(org.opensearch.search.aggregations.bucket.terms.Terms) Tuple(org.opensearch.common.collect.Tuple) InternalTerms(org.opensearch.search.aggregations.bucket.terms.InternalTerms) IndexWriter(org.apache.lucene.index.IndexWriter) List(java.util.List) Matchers.equalTo(org.hamcrest.Matchers.equalTo) IndexSettings(org.opensearch.index.IndexSettings) QueryShardContext(org.opensearch.index.query.QueryShardContext) BigArrays(org.opensearch.common.util.BigArrays) IndexWriterConfig(org.apache.lucene.index.IndexWriterConfig) IndexReader(org.apache.lucene.index.IndexReader) IndexSearcher(org.apache.lucene.search.IndexSearcher) BucketOrder(org.opensearch.search.aggregations.BucketOrder) LongPoint(org.apache.lucene.document.LongPoint) HistogramAggregationBuilder(org.opensearch.search.aggregations.bucket.histogram.HistogramAggregationBuilder) StringField(org.apache.lucene.document.StringField) CheckedConsumer(org.opensearch.common.CheckedConsumer) HashMap(java.util.HashMap) Function(java.util.function.Function) Supplier(java.util.function.Supplier) NumberFieldMapper(org.opensearch.index.mapper.NumberFieldMapper) ArrayList(java.util.ArrayList) IntPoint(org.apache.lucene.document.IntPoint) CoreValuesSourceType(org.opensearch.search.aggregations.support.CoreValuesSourceType) ValuesSourceType(org.opensearch.search.aggregations.support.ValuesSourceType) FloatPoint(org.apache.lucene.document.FloatPoint) AggregatorTestCase(org.opensearch.search.aggregations.AggregatorTestCase) Filter(org.opensearch.search.aggregations.bucket.filter.Filter) IOException(java.io.IOException) DocValuesFieldExistsQuery(org.apache.lucene.search.DocValuesFieldExistsQuery) InternalGlobal(org.opensearch.search.aggregations.bucket.global.InternalGlobal) Consumer(java.util.function.Consumer) Field(org.apache.lucene.document.Field) CircuitBreakerService(org.opensearch.indices.breaker.CircuitBreakerService) TermsAggregationBuilder(org.opensearch.search.aggregations.bucket.terms.TermsAggregationBuilder) LeafDocLookup(org.opensearch.search.lookup.LeafDocLookup) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) Comparator(java.util.Comparator) Collections(java.util.Collections) InternalGlobal(org.opensearch.search.aggregations.bucket.global.InternalGlobal) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) GlobalAggregationBuilder(org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder)

Example 3 with InternalGlobal

use of org.opensearch.search.aggregations.bucket.global.InternalGlobal in project OpenSearch by opensearch-project.

the class GlobalAggregatorTests method testCase.

// Note that `global`'s fancy support for ignoring the query comes from special code in AggregationPhase. We don't test that here.
private void testCase(CheckedConsumer<RandomIndexWriter, IOException> buildIndex, BiConsumer<InternalGlobal, InternalMin> verify) throws IOException {
    Directory directory = newDirectory();
    RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory);
    buildIndex.accept(indexWriter);
    indexWriter.close();
    IndexReader indexReader = DirectoryReader.open(directory);
    IndexSearcher indexSearcher = newSearcher(indexReader, true, true);
    GlobalAggregationBuilder aggregationBuilder = new GlobalAggregationBuilder("_name");
    aggregationBuilder.subAggregation(new MinAggregationBuilder("in_global").field("number"));
    MappedFieldType fieldType = new NumberFieldMapper.NumberFieldType("number", NumberFieldMapper.NumberType.LONG);
    GlobalAggregator aggregator = createAggregator(aggregationBuilder, indexSearcher, fieldType);
    aggregator.preCollection();
    indexSearcher.search(new MatchAllDocsQuery(), aggregator);
    aggregator.postCollection();
    InternalGlobal result = (InternalGlobal) aggregator.buildTopLevel();
    verify.accept(result, (InternalMin) result.getAggregations().asMap().get("in_global"));
    indexReader.close();
    directory.close();
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) InternalGlobal(org.opensearch.search.aggregations.bucket.global.InternalGlobal) IndexReader(org.apache.lucene.index.IndexReader) MinAggregationBuilder(org.opensearch.search.aggregations.metrics.MinAggregationBuilder) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) Directory(org.apache.lucene.store.Directory) GlobalAggregationBuilder(org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder) GlobalAggregator(org.opensearch.search.aggregations.bucket.global.GlobalAggregator)

Example 4 with InternalGlobal

use of org.opensearch.search.aggregations.bucket.global.InternalGlobal in project OpenSearch by opensearch-project.

the class RareTermsAggregatorTests method testGlobalAggregationWithScore.

public void testGlobalAggregationWithScore() throws IOException {
    try (Directory directory = newDirectory()) {
        try (RandomIndexWriter indexWriter = new RandomIndexWriter(random(), directory)) {
            Document document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("a")));
            indexWriter.addDocument(document);
            document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("c")));
            indexWriter.addDocument(document);
            document = new Document();
            document.add(new SortedDocValuesField("keyword", new BytesRef("e")));
            indexWriter.addDocument(document);
            try (IndexReader indexReader = maybeWrapReaderEs(indexWriter.getReader())) {
                IndexSearcher indexSearcher = newIndexSearcher(indexReader);
                GlobalAggregationBuilder globalBuilder = new GlobalAggregationBuilder("global").subAggregation(new RareTermsAggregationBuilder("terms").field("keyword").subAggregation(new RareTermsAggregationBuilder("sub_terms").field("keyword").subAggregation(new TopHitsAggregationBuilder("top_hits").storedField("_none_"))));
                MappedFieldType fieldType = new KeywordFieldMapper.KeywordFieldType("keyword");
                InternalGlobal result = searchAndReduce(indexSearcher, new MatchAllDocsQuery(), globalBuilder, fieldType);
                InternalMultiBucketAggregation<?, ?> terms = result.getAggregations().get("terms");
                assertThat(terms.getBuckets().size(), equalTo(3));
                for (MultiBucketsAggregation.Bucket bucket : terms.getBuckets()) {
                    InternalMultiBucketAggregation<?, ?> subTerms = bucket.getAggregations().get("sub_terms");
                    assertThat(subTerms.getBuckets().size(), equalTo(1));
                    MultiBucketsAggregation.Bucket subBucket = subTerms.getBuckets().get(0);
                    InternalTopHits topHits = subBucket.getAggregations().get("top_hits");
                    assertThat(topHits.getHits().getHits().length, equalTo(1));
                    for (SearchHit hit : topHits.getHits()) {
                        assertThat(hit.getScore(), greaterThan(0f));
                    }
                }
            }
        }
    }
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) InternalGlobal(org.opensearch.search.aggregations.bucket.global.InternalGlobal) SearchHit(org.opensearch.search.SearchHit) TopHitsAggregationBuilder(org.opensearch.search.aggregations.metrics.TopHitsAggregationBuilder) Document(org.apache.lucene.document.Document) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) GlobalAggregationBuilder(org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder) InternalTopHits(org.opensearch.search.aggregations.metrics.InternalTopHits) MultiBucketsAggregation(org.opensearch.search.aggregations.bucket.MultiBucketsAggregation) SortedDocValuesField(org.apache.lucene.document.SortedDocValuesField) IndexReader(org.apache.lucene.index.IndexReader) MappedFieldType(org.opensearch.index.mapper.MappedFieldType) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) BytesRef(org.apache.lucene.util.BytesRef) Directory(org.apache.lucene.store.Directory)

Aggregations

IndexReader (org.apache.lucene.index.IndexReader)4 RandomIndexWriter (org.apache.lucene.index.RandomIndexWriter)4 IndexSearcher (org.apache.lucene.search.IndexSearcher)4 MatchAllDocsQuery (org.apache.lucene.search.MatchAllDocsQuery)4 Directory (org.apache.lucene.store.Directory)4 MappedFieldType (org.opensearch.index.mapper.MappedFieldType)4 GlobalAggregationBuilder (org.opensearch.search.aggregations.bucket.global.GlobalAggregationBuilder)4 InternalGlobal (org.opensearch.search.aggregations.bucket.global.InternalGlobal)4 Document (org.apache.lucene.document.Document)3 BytesRef (org.apache.lucene.util.BytesRef)3 SortedDocValuesField (org.apache.lucene.document.SortedDocValuesField)2 SearchHit (org.opensearch.search.SearchHit)2 MultiBucketsAggregation (org.opensearch.search.aggregations.bucket.MultiBucketsAggregation)2 InternalTopHits (org.opensearch.search.aggregations.metrics.InternalTopHits)2 TopHitsAggregationBuilder (org.opensearch.search.aggregations.metrics.TopHitsAggregationBuilder)2 IOException (java.io.IOException)1 ArrayList (java.util.ArrayList)1 Arrays (java.util.Arrays)1 Collections (java.util.Collections)1 Collections.singleton (java.util.Collections.singleton)1