Search in sources :

Example 1 with NestedQueryBuilder

use of org.opensearch.index.query.NestedQueryBuilder in project bw-calendar-engine by Bedework.

the class ESQueryFilter method makeQuery.

/* TODO we need to provide a chain of filters when we have deep paths,
       e.g. entity[key1].entity[key2].value = "something"
   */
public QueryBuilder makeQuery(final List<PropertyInfoIndex> pis, final Object val, final Integer intKey, final String strKey, final OperationType opType, final boolean negate) throws CalFacadeException {
    /* Work backwards through the property list building a path.
       When the head of the path is a nested type:
         If it's the first we found:
            generate a match or term query based on the leaf
         otherwise:
            we already have a nested query to push inside a new one
    
       If the top entry has a keyindex we expect a String or Numeric 
       key value we generate a bool query with 2 must match terms.
     */
    QueryBuilder qb = null;
    // current nested level
    QueryBuilder nqb = null;
    PropertyInfoIndex leafPii = null;
    /* See if we need to build a nested query */
    final BwIcalPropertyInfoEntry rootPie = BwIcalPropertyInfo.getPinfo(pis.get(0));
    final boolean isNested = rootPie.getNested();
    for (int plistIndex = pis.size() - 1; plistIndex >= 0; plistIndex--) {
        final PropertyInfoIndex pii = pis.get(plistIndex);
        if (leafPii == null) {
            leafPii = pii;
        }
        final BwIcalPropertyInfoEntry bwPie = BwIcalPropertyInfo.getPinfo(pii);
        final String fullTermPath;
        if (bwPie.getTermsField() == null) {
            fullTermPath = null;
        } else {
            fullTermPath = makePropertyRef(pis, plistIndex, bwPie.getTermsField());
        }
        if (isNested) {
            final QueryBuilder nested;
            String path = makePropertyRef(pis, plistIndex, null);
            if (nqb != null) {
                if (plistIndex == 0) {
                    // TODO Temp fix this
                    path = "event." + path;
                }
                nested = new NestedQueryBuilder(path, nqb, ScoreMode.Avg);
            } else {
                qb = makeQuery(leafPii, makePropertyRef(pis, null), fullTermPath, val, opType, 1, false);
                /* Is the parent indexed? */
                final BwIcalPropertyInfoEntry parentPie;
                if (plistIndex == 0) {
                    // No parent
                    parentPie = null;
                } else {
                    parentPie = BwIcalPropertyInfo.getPinfo(pis.get(plistIndex - 1));
                }
                if ((parentPie != null) && (parentPie.getKeyindex() != PropertyInfoIndex.UNKNOWN_PROPERTY)) {
                    final BoolQueryBuilder bqb = new BoolQueryBuilder();
                    if (qb == null) {
                        error("No nested query for " + pii);
                        return null;
                    }
                    bqb.must(qb);
                    final List<PropertyInfoIndex> indexPis = new ArrayList<>();
                    indexPis.add(pis.get(plistIndex - 1));
                    indexPis.add(parentPie.getKeyindex());
                    final String indexPath = makePropertyRef(indexPis, null);
                    if (intKey != null) {
                        bqb.must(new TermQueryBuilder(indexPath, intKey));
                    } else if (strKey != null) {
                        bqb.must(new TermQueryBuilder(indexPath, strKey));
                    } else {
                        error("Missing key for index for " + pii);
                        return null;
                    }
                    qb = bqb;
                }
                nested = qb;
            }
            nqb = nested;
        } else if (plistIndex == 0) {
            // No nested types found
            qb = makeQuery(leafPii, makePropertyRef(pis, null), fullTermPath, val, opType, 1, false);
        }
    }
    if (nqb != null) {
        qb = nqb;
    }
    if (negate) {
        return not(qb);
    }
    return qb;
}
Also used : PropertyInfoIndex(org.bedework.util.calendar.PropertyIndex.PropertyInfoIndex) BwIcalPropertyInfoEntry(org.bedework.calfacade.ical.BwIcalPropertyInfo.BwIcalPropertyInfoEntry) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) ArrayList(java.util.ArrayList) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) MatchNoneQueryBuilder(org.opensearch.index.query.MatchNoneQueryBuilder) QueryBuilder(org.opensearch.index.query.QueryBuilder) TermsQueryBuilder(org.opensearch.index.query.TermsQueryBuilder) BoolQueryBuilder(org.opensearch.index.query.BoolQueryBuilder) MultiMatchQueryBuilder(org.opensearch.index.query.MultiMatchQueryBuilder) RangeQueryBuilder(org.opensearch.index.query.RangeQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) MatchQueryBuilder(org.opensearch.index.query.MatchQueryBuilder) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder)

Example 2 with NestedQueryBuilder

use of org.opensearch.index.query.NestedQueryBuilder in project OpenSearch by opensearch-project.

the class MetadataFetchingIT method testInnerHits.

public void testInnerHits() {
    assertAcked(prepareCreate("test").addMapping("_doc", "nested", "type=nested"));
    ensureGreen();
    client().prepareIndex("test").setId("1").setSource("field", "value", "nested", Collections.singletonMap("title", "foo")).get();
    refresh();
    SearchResponse response = client().prepareSearch("test").storedFields("_none_").setFetchSource(false).setQuery(new NestedQueryBuilder("nested", new TermQueryBuilder("nested.title", "foo"), ScoreMode.Total).innerHit(new InnerHitBuilder().setStoredFieldNames(Collections.singletonList("_none_")).setFetchSourceContext(new FetchSourceContext(false)))).get();
    assertThat(response.getHits().getTotalHits().value, equalTo(1L));
    assertThat(response.getHits().getAt(0).getId(), nullValue());
    assertThat(response.getHits().getAt(0).getSourceAsString(), nullValue());
    assertThat(response.getHits().getAt(0).getInnerHits().size(), equalTo(1));
    SearchHits hits = response.getHits().getAt(0).getInnerHits().get("nested");
    assertThat(hits.getTotalHits().value, equalTo(1L));
    assertThat(hits.getAt(0).getId(), nullValue());
    assertThat(hits.getAt(0).getSourceAsString(), nullValue());
}
Also used : FetchSourceContext(org.opensearch.search.fetch.subphase.FetchSourceContext) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) InnerHitBuilder(org.opensearch.index.query.InnerHitBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) SearchHits(org.opensearch.search.SearchHits) SearchResponse(org.opensearch.action.search.SearchResponse)

Example 3 with NestedQueryBuilder

use of org.opensearch.index.query.NestedQueryBuilder in project OpenSearch by opensearch-project.

the class NestedHelperTests method testNested.

public void testNested() throws IOException {
    QueryShardContext context = indexService.newQueryShardContext(0, new IndexSearcher(new MultiReader()), () -> 0, null);
    NestedQueryBuilder queryBuilder = new NestedQueryBuilder("nested1", new MatchAllQueryBuilder(), ScoreMode.Avg);
    OpenSearchToParentBlockJoinQuery query = (OpenSearchToParentBlockJoinQuery) queryBuilder.toQuery(context);
    Query expectedChildQuery = new BooleanQuery.Builder().add(new MatchAllDocsQuery(), Occur.MUST).add(new TermQuery(new Term("_type", "__nested1")), Occur.FILTER).build();
    assertEquals(expectedChildQuery, query.getChildQuery());
    assertFalse(new NestedHelper(mapperService).mightMatchNestedDocs(query));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested1"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested2"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested3"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested_missing"));
    queryBuilder = new NestedQueryBuilder("nested1", new TermQueryBuilder("nested1.foo", "bar"), ScoreMode.Avg);
    query = (OpenSearchToParentBlockJoinQuery) queryBuilder.toQuery(context);
    // this time we do not add a filter since the inner query only matches inner docs
    expectedChildQuery = new TermQuery(new Term("nested1.foo", "bar"));
    assertEquals(expectedChildQuery, query.getChildQuery());
    assertFalse(new NestedHelper(mapperService).mightMatchNestedDocs(query));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested1"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested2"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested3"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested_missing"));
    queryBuilder = new NestedQueryBuilder("nested2", new TermQueryBuilder("nested2.foo", "bar"), ScoreMode.Avg);
    query = (OpenSearchToParentBlockJoinQuery) queryBuilder.toQuery(context);
    // we need to add the filter again because of include_in_parent
    expectedChildQuery = new BooleanQuery.Builder().add(new TermQuery(new Term("nested2.foo", "bar")), Occur.MUST).add(new TermQuery(new Term("_type", "__nested2")), Occur.FILTER).build();
    assertEquals(expectedChildQuery, query.getChildQuery());
    assertFalse(new NestedHelper(mapperService).mightMatchNestedDocs(query));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested1"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested2"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested3"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested_missing"));
    queryBuilder = new NestedQueryBuilder("nested3", new TermQueryBuilder("nested3.foo", "bar"), ScoreMode.Avg);
    query = (OpenSearchToParentBlockJoinQuery) queryBuilder.toQuery(context);
    // we need to add the filter again because of include_in_root
    expectedChildQuery = new BooleanQuery.Builder().add(new TermQuery(new Term("nested3.foo", "bar")), Occur.MUST).add(new TermQuery(new Term("_type", "__nested3")), Occur.FILTER).build();
    assertEquals(expectedChildQuery, query.getChildQuery());
    assertFalse(new NestedHelper(mapperService).mightMatchNestedDocs(query));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested1"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested2"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested3"));
    assertTrue(new NestedHelper(mapperService).mightMatchNonNestedDocs(query, "nested_missing"));
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) BooleanQuery(org.apache.lucene.search.BooleanQuery) TermQuery(org.apache.lucene.search.TermQuery) Query(org.apache.lucene.search.Query) MatchNoDocsQuery(org.apache.lucene.search.MatchNoDocsQuery) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) TermQuery(org.apache.lucene.search.TermQuery) BooleanQuery(org.apache.lucene.search.BooleanQuery) MultiReader(org.apache.lucene.index.MultiReader) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) Term(org.apache.lucene.index.Term) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) MatchAllDocsQuery(org.apache.lucene.search.MatchAllDocsQuery) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) QueryShardContext(org.opensearch.index.query.QueryShardContext) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder)

Example 4 with NestedQueryBuilder

use of org.opensearch.index.query.NestedQueryBuilder in project OpenSearch by opensearch-project.

the class NestedSortingTests method testMultiLevelNestedSorting.

public void testMultiLevelNestedSorting() throws IOException {
    XContentBuilder mapping = XContentFactory.jsonBuilder();
    mapping.startObject();
    {
        mapping.startObject("_doc");
        {
            mapping.startObject("properties");
            {
                {
                    mapping.startObject("title");
                    mapping.field("type", "text");
                    mapping.endObject();
                }
                {
                    mapping.startObject("genre");
                    mapping.field("type", "keyword");
                    mapping.endObject();
                }
                {
                    mapping.startObject("chapters");
                    mapping.field("type", "nested");
                    {
                        mapping.startObject("properties");
                        {
                            mapping.startObject("title");
                            mapping.field("type", "text");
                            mapping.endObject();
                        }
                        {
                            mapping.startObject("read_time_seconds");
                            mapping.field("type", "integer");
                            mapping.endObject();
                        }
                        {
                            mapping.startObject("paragraphs");
                            mapping.field("type", "nested");
                            {
                                mapping.startObject("properties");
                                {
                                    {
                                        mapping.startObject("header");
                                        mapping.field("type", "text");
                                        mapping.endObject();
                                    }
                                    {
                                        mapping.startObject("content");
                                        mapping.field("type", "text");
                                        mapping.endObject();
                                    }
                                    {
                                        mapping.startObject("word_count");
                                        mapping.field("type", "integer");
                                        mapping.endObject();
                                    }
                                }
                                mapping.endObject();
                            }
                            mapping.endObject();
                        }
                        mapping.endObject();
                    }
                    mapping.endObject();
                }
            }
            mapping.endObject();
        }
        mapping.endObject();
    }
    mapping.endObject();
    IndexService indexService = createIndex("nested_sorting", Settings.EMPTY, "_doc", mapping);
    List<List<Document>> books = new ArrayList<>();
    {
        List<Document> book = new ArrayList<>();
        Document document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "Paragraph 1", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 743));
        document.add(new IntPoint("chapters.paragraphs.word_count", 743));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "chapter 3", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 400));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 400));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "Paragraph 1", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 234));
        document.add(new IntPoint("chapters.paragraphs.word_count", 234));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "chapter 2", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 200));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 200));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "Paragraph 2", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 478));
        document.add(new IntPoint("chapters.paragraphs.word_count", 478));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "Paragraph 1", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 849));
        document.add(new IntPoint("chapters.paragraphs.word_count", 849));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "chapter 1", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 1400));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 1400));
        book.add(document);
        document = new Document();
        document.add(new StringField("genre", "science fiction", Field.Store.NO));
        document.add(new StringField("_type", "_doc", Field.Store.NO));
        document.add(new StringField("_id", "1", Field.Store.YES));
        document.add(new NumericDocValuesField(PRIMARY_TERM_NAME, 0));
        book.add(document);
        books.add(book);
    }
    {
        List<Document> book = new ArrayList<>();
        Document document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "Introduction", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 76));
        document.add(new IntPoint("chapters.paragraphs.word_count", 76));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "chapter 1", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 20));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 20));
        book.add(document);
        document = new Document();
        document.add(new StringField("genre", "romance", Field.Store.NO));
        document.add(new StringField("_type", "_doc", Field.Store.NO));
        document.add(new StringField("_id", "2", Field.Store.YES));
        document.add(new NumericDocValuesField(PRIMARY_TERM_NAME, 0));
        book.add(document);
        books.add(book);
    }
    {
        List<Document> book = new ArrayList<>();
        Document document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "A bad dream", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 976));
        document.add(new IntPoint("chapters.paragraphs.word_count", 976));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "The beginning of the end", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 1200));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 1200));
        book.add(document);
        document = new Document();
        document.add(new StringField("genre", "horror", Field.Store.NO));
        document.add(new StringField("_type", "_doc", Field.Store.NO));
        document.add(new StringField("_id", "3", Field.Store.YES));
        document.add(new NumericDocValuesField(PRIMARY_TERM_NAME, 0));
        book.add(document);
        books.add(book);
    }
    {
        List<Document> book = new ArrayList<>();
        Document document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "macaroni", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 180));
        document.add(new IntPoint("chapters.paragraphs.word_count", 180));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "hamburger", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 150));
        document.add(new IntPoint("chapters.paragraphs.word_count", 150));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "tosti", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 120));
        document.add(new IntPoint("chapters.paragraphs.word_count", 120));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "easy meals", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 800));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 800));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.paragraphs.header", "introduction", Field.Store.NO));
        document.add(new StringField("_type", "__chapters.paragraphs", Field.Store.NO));
        document.add(new TextField("chapters.paragraphs.text", "some text...", Field.Store.NO));
        document.add(new SortedNumericDocValuesField("chapters.paragraphs.word_count", 87));
        document.add(new IntPoint("chapters.paragraphs.word_count", 87));
        book.add(document);
        document = new Document();
        document.add(new TextField("chapters.title", "introduction", Field.Store.NO));
        document.add(new StringField("_type", "__chapters", Field.Store.NO));
        document.add(new IntPoint("chapters.read_time_seconds", 10));
        document.add(new NumericDocValuesField("chapters.read_time_seconds", 10));
        book.add(document);
        document = new Document();
        document.add(new StringField("genre", "cooking", Field.Store.NO));
        document.add(new StringField("_type", "_doc", Field.Store.NO));
        document.add(new StringField("_id", "4", Field.Store.YES));
        document.add(new NumericDocValuesField(PRIMARY_TERM_NAME, 0));
        book.add(document);
        books.add(book);
    }
    {
        List<Document> book = new ArrayList<>();
        Document document = new Document();
        document.add(new StringField("genre", "unknown", Field.Store.NO));
        document.add(new StringField("_type", "_doc", Field.Store.NO));
        document.add(new StringField("_id", "5", Field.Store.YES));
        document.add(new NumericDocValuesField(PRIMARY_TERM_NAME, 0));
        book.add(document);
        books.add(book);
    }
    Collections.shuffle(books, random());
    for (List<Document> book : books) {
        writer.addDocuments(book);
        if (randomBoolean()) {
            writer.commit();
        }
    }
    DirectoryReader reader = DirectoryReader.open(writer);
    reader = OpenSearchDirectoryReader.wrap(reader, new ShardId(indexService.index(), 0));
    IndexSearcher searcher = new IndexSearcher(reader);
    QueryShardContext queryShardContext = indexService.newQueryShardContext(0, searcher, () -> 0L, null);
    FieldSortBuilder sortBuilder = new FieldSortBuilder("chapters.paragraphs.word_count");
    sortBuilder.setNestedSort(new NestedSortBuilder("chapters").setNestedSort(new NestedSortBuilder("chapters.paragraphs")));
    QueryBuilder queryBuilder = new MatchAllQueryBuilder();
    TopFieldDocs topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
    assertThat(topFields.totalHits.value, equalTo(5L));
    assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2"));
    assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76L));
    assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("4"));
    assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(87L));
    assertThat(searcher.doc(topFields.scoreDocs[2].doc).get("_id"), equalTo("1"));
    assertThat(((FieldDoc) topFields.scoreDocs[2]).fields[0], equalTo(234L));
    assertThat(searcher.doc(topFields.scoreDocs[3].doc).get("_id"), equalTo("3"));
    assertThat(((FieldDoc) topFields.scoreDocs[3]).fields[0], equalTo(976L));
    assertThat(searcher.doc(topFields.scoreDocs[4].doc).get("_id"), equalTo("5"));
    assertThat(((FieldDoc) topFields.scoreDocs[4]).fields[0], equalTo(Long.MAX_VALUE));
    // Specific genre
    {
        queryBuilder = new TermQueryBuilder("genre", "romance");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76L));
        queryBuilder = new TermQueryBuilder("genre", "science fiction");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(234L));
        queryBuilder = new TermQueryBuilder("genre", "horror");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976L));
        queryBuilder = new TermQueryBuilder("genre", "cooking");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87L));
    }
    // reverse sort order
    {
        sortBuilder.order(SortOrder.DESC);
        queryBuilder = new MatchAllQueryBuilder();
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(5L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976L));
        assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("1"));
        assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(849L));
        assertThat(searcher.doc(topFields.scoreDocs[2].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[2]).fields[0], equalTo(180L));
        assertThat(searcher.doc(topFields.scoreDocs[3].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[3]).fields[0], equalTo(76L));
        assertThat(searcher.doc(topFields.scoreDocs[4].doc).get("_id"), equalTo("5"));
        assertThat(((FieldDoc) topFields.scoreDocs[4]).fields[0], equalTo(Long.MIN_VALUE));
    }
    // Specific genre and reverse sort order
    {
        queryBuilder = new TermQueryBuilder("genre", "romance");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76L));
        queryBuilder = new TermQueryBuilder("genre", "science fiction");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(849L));
        queryBuilder = new TermQueryBuilder("genre", "horror");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(976L));
        queryBuilder = new TermQueryBuilder("genre", "cooking");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(180L));
    }
    // Nested filter + query
    {
        queryBuilder = new RangeQueryBuilder("chapters.read_time_seconds").to(50L);
        sortBuilder = new FieldSortBuilder("chapters.paragraphs.word_count");
        sortBuilder.setNestedSort(new NestedSortBuilder("chapters").setFilter(queryBuilder).setNestedSort(new NestedSortBuilder("chapters.paragraphs")));
        topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(2L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76L));
        assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(87L));
        sortBuilder.order(SortOrder.DESC);
        topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(2L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87L));
        assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(76L));
    }
    // Multiple Nested filters + query
    {
        queryBuilder = new RangeQueryBuilder("chapters.read_time_seconds").to(50L);
        sortBuilder = new FieldSortBuilder("chapters.paragraphs.word_count");
        sortBuilder.setNestedSort(new NestedSortBuilder("chapters").setFilter(queryBuilder).setNestedSort(new NestedSortBuilder("chapters.paragraphs").setFilter(new RangeQueryBuilder("chapters.paragraphs.word_count").from(80L))));
        topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(2L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87L));
        assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(Long.MAX_VALUE));
        sortBuilder.order(SortOrder.DESC);
        topFields = search(new NestedQueryBuilder("chapters", queryBuilder, ScoreMode.None), sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(2L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87L));
        assertThat(searcher.doc(topFields.scoreDocs[1].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[1]).fields[0], equalTo(Long.MIN_VALUE));
    }
    // Nested filter + Specific genre
    {
        sortBuilder = new FieldSortBuilder("chapters.paragraphs.word_count");
        sortBuilder.setNestedSort(new NestedSortBuilder("chapters").setFilter(new RangeQueryBuilder("chapters.read_time_seconds").to(50L)).setNestedSort(new NestedSortBuilder("chapters.paragraphs")));
        queryBuilder = new TermQueryBuilder("genre", "romance");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("2"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(76L));
        queryBuilder = new TermQueryBuilder("genre", "science fiction");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("1"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(Long.MAX_VALUE));
        queryBuilder = new TermQueryBuilder("genre", "horror");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("3"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(Long.MAX_VALUE));
        queryBuilder = new TermQueryBuilder("genre", "cooking");
        topFields = search(queryBuilder, sortBuilder, queryShardContext, searcher);
        assertThat(topFields.totalHits.value, equalTo(1L));
        assertThat(searcher.doc(topFields.scoreDocs[0].doc).get("_id"), equalTo("4"));
        assertThat(((FieldDoc) topFields.scoreDocs[0]).fields[0], equalTo(87L));
    }
    searcher.getIndexReader().close();
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) FieldDoc(org.apache.lucene.search.FieldDoc) IndexService(org.opensearch.index.IndexService) ArrayList(java.util.ArrayList) TopFieldDocs(org.apache.lucene.search.TopFieldDocs) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) QueryBuilder(org.opensearch.index.query.QueryBuilder) RangeQueryBuilder(org.opensearch.index.query.RangeQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder) TermQueryBuilder(org.opensearch.index.query.TermQueryBuilder) Document(org.apache.lucene.document.Document) RangeQueryBuilder(org.opensearch.index.query.RangeQueryBuilder) ShardId(org.opensearch.index.shard.ShardId) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NumericDocValuesField(org.apache.lucene.document.NumericDocValuesField) NestedQueryBuilder(org.opensearch.index.query.NestedQueryBuilder) TextField(org.apache.lucene.document.TextField) QueryShardContext(org.opensearch.index.query.QueryShardContext) List(java.util.List) ArrayList(java.util.ArrayList) DirectoryReader(org.apache.lucene.index.DirectoryReader) OpenSearchDirectoryReader(org.opensearch.common.lucene.index.OpenSearchDirectoryReader) FieldSortBuilder(org.opensearch.search.sort.FieldSortBuilder) IntPoint(org.apache.lucene.document.IntPoint) SortedNumericDocValuesField(org.apache.lucene.document.SortedNumericDocValuesField) NestedSortBuilder(org.opensearch.search.sort.NestedSortBuilder) StringField(org.apache.lucene.document.StringField) XContentBuilder(org.opensearch.common.xcontent.XContentBuilder) MatchAllQueryBuilder(org.opensearch.index.query.MatchAllQueryBuilder)

Aggregations

NestedQueryBuilder (org.opensearch.index.query.NestedQueryBuilder)4 TermQueryBuilder (org.opensearch.index.query.TermQueryBuilder)4 MatchAllQueryBuilder (org.opensearch.index.query.MatchAllQueryBuilder)3 ArrayList (java.util.ArrayList)2 IndexSearcher (org.apache.lucene.search.IndexSearcher)2 XContentBuilder (org.opensearch.common.xcontent.XContentBuilder)2 QueryBuilder (org.opensearch.index.query.QueryBuilder)2 QueryShardContext (org.opensearch.index.query.QueryShardContext)2 RangeQueryBuilder (org.opensearch.index.query.RangeQueryBuilder)2 List (java.util.List)1 Document (org.apache.lucene.document.Document)1 IntPoint (org.apache.lucene.document.IntPoint)1 NumericDocValuesField (org.apache.lucene.document.NumericDocValuesField)1 SortedNumericDocValuesField (org.apache.lucene.document.SortedNumericDocValuesField)1 StringField (org.apache.lucene.document.StringField)1 TextField (org.apache.lucene.document.TextField)1 DirectoryReader (org.apache.lucene.index.DirectoryReader)1 MultiReader (org.apache.lucene.index.MultiReader)1 Term (org.apache.lucene.index.Term)1 BooleanQuery (org.apache.lucene.search.BooleanQuery)1