Search in sources :

Example 16 with TotalHits

use of org.apache.lucene.search.TotalHits in project Anserini by castorini.

the class SearchCollection method search.

public <K> ScoredDocuments search(IndexSearcher searcher, K qid, String queryString, RerankerCascade cascade, ScoredDocuments queryQrels, boolean hasRelDocs) throws IOException {
    Query query = null;
    if (args.sdm) {
        query = new SdmQueryGenerator(args.sdm_tw, args.sdm_ow, args.sdm_uw).buildQuery(IndexArgs.CONTENTS, analyzer, queryString);
    } else {
        QueryGenerator generator;
        try {
            generator = (QueryGenerator) Class.forName("io.anserini.search.query." + args.queryGenerator).getConstructor().newInstance();
        } catch (Exception e) {
            e.printStackTrace();
            throw new IllegalArgumentException("Unable to load QueryGenerator: " + args.topicReader);
        }
        query = generator.buildQuery(IndexArgs.CONTENTS, analyzer, queryString);
    }
    TopDocs rs = new TopDocs(new TotalHits(0, TotalHits.Relation.EQUAL_TO), new ScoreDoc[] {});
    if (!isRerank || (args.rerankcutoff > 0 && args.rf_qrels == null) || (args.rf_qrels != null && !hasRelDocs)) {
        if (args.arbitraryScoreTieBreak) {
            // Figure out how to break the scoring ties.
            rs = searcher.search(query, (isRerank && args.rf_qrels == null) ? args.rerankcutoff : args.hits);
        } else {
            rs = searcher.search(query, (isRerank && args.rf_qrels == null) ? args.rerankcutoff : args.hits, BREAK_SCORE_TIES_BY_DOCID, true);
        }
    }
    List<String> queryTokens = AnalyzerUtils.analyze(analyzer, queryString);
    queries.put(qid.toString(), queryTokens);
    RerankerContext context = new RerankerContext<>(searcher, qid, query, null, queryString, queryTokens, null, args);
    ScoredDocuments scoredFbDocs;
    if (isRerank && args.rf_qrels != null) {
        if (hasRelDocs) {
            scoredFbDocs = queryQrels;
        } else {
            // if no relevant documents, only perform score based tie breaking next
            LOG.info("No relevant documents for " + qid.toString());
            scoredFbDocs = ScoredDocuments.fromTopDocs(rs, searcher);
            cascade = new RerankerCascade();
            cascade.add(new ScoreTiesAdjusterReranker());
        }
    } else {
        scoredFbDocs = ScoredDocuments.fromTopDocs(rs, searcher);
    }
    return cascade.run(scoredFbDocs, context);
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) Query(org.apache.lucene.search.Query) TermInSetQuery(org.apache.lucene.search.TermInSetQuery) BooleanQuery(org.apache.lucene.search.BooleanQuery) ScoredDocuments(io.anserini.rerank.ScoredDocuments) QueryNodeException(org.apache.lucene.queryparser.flexible.core.QueryNodeException) IOException(java.io.IOException) CompletionException(java.util.concurrent.CompletionException) CmdLineException(org.kohsuke.args4j.CmdLineException) AtomicMoveNotSupportedException(java.nio.file.AtomicMoveNotSupportedException) TopDocs(org.apache.lucene.search.TopDocs) RerankerCascade(io.anserini.rerank.RerankerCascade) QueryGenerator(io.anserini.search.query.QueryGenerator) SdmQueryGenerator(io.anserini.search.query.SdmQueryGenerator) ScoreTiesAdjusterReranker(io.anserini.rerank.lib.ScoreTiesAdjusterReranker) SdmQueryGenerator(io.anserini.search.query.SdmQueryGenerator) RerankerContext(io.anserini.rerank.RerankerContext)

Example 17 with TotalHits

use of org.apache.lucene.search.TotalHits in project sonarqube by SonarSource.

the class IssueIndexTest method paging.

@Test
public void paging() {
    ComponentDto project = newPrivateProjectDto();
    ComponentDto file = newFileDto(project, null);
    for (int i = 0; i < 12; i++) {
        indexIssues(newDoc("I" + i, file));
    }
    IssueQuery.Builder query = IssueQuery.builder();
    // There are 12 issues in total, with 10 issues per page, the page 2 should only contain 2 elements
    SearchResponse result = underTest.search(query.build(), new SearchOptions().setPage(2, 10));
    assertThat(result.getHits().getHits()).hasSize(2);
    assertThat(result.getHits().getTotalHits()).isEqualTo(new TotalHits(12, TotalHits.Relation.EQUAL_TO));
    result = underTest.search(IssueQuery.builder().build(), new SearchOptions().setOffset(0).setLimit(5));
    assertThat(result.getHits().getHits()).hasSize(5);
    assertThat(result.getHits().getTotalHits()).isEqualTo(new TotalHits(12, TotalHits.Relation.EQUAL_TO));
    result = underTest.search(IssueQuery.builder().build(), new SearchOptions().setOffset(2).setLimit(10));
    assertThat(result.getHits().getHits()).hasSize(10);
    assertThat(result.getHits().getTotalHits()).isEqualTo(new TotalHits(12, TotalHits.Relation.EQUAL_TO));
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) ComponentDto(org.sonar.db.component.ComponentDto) SearchOptions(org.sonar.server.es.SearchOptions) SearchResponse(org.elasticsearch.action.search.SearchResponse) Test(org.junit.Test)

Example 18 with TotalHits

use of org.apache.lucene.search.TotalHits in project sonarqube by SonarSource.

the class EsUtilsTest method convertToDocs_empty.

@Test
public void convertToDocs_empty() {
    SearchHits hits = new SearchHits(new SearchHit[] {}, new TotalHits(0, TotalHits.Relation.EQUAL_TO), 0);
    List<BaseDoc> docs = EsUtils.convertToDocs(hits, IssueDoc::new);
    assertThat(docs).isEmpty();
}
Also used : TotalHits(org.apache.lucene.search.TotalHits) IssueDoc(org.sonar.server.issue.index.IssueDoc) SearchHits(org.elasticsearch.search.SearchHits) Test(org.junit.Test)

Example 19 with TotalHits

use of org.apache.lucene.search.TotalHits in project snow-owl by b2ihealthcare.

the class EsDocumentSearcher method toHits.

private <T> Hits<T> toHits(Class<T> select, List<Class<?>> from, final List<String> fields, final boolean fetchSource, final int limit, final int totalHits, final SortBy sortBy, final Iterable<SearchHit> hits) throws IOException {
    final HitConverter<T> hitConverter = HitConverter.getConverter(mapper, select, from, fetchSource, fields);
    Object[] searchAfterSortValues = null;
    final ImmutableList.Builder<T> result = ImmutableList.builder();
    for (Iterator<SearchHit> iterator = hits.iterator(); iterator.hasNext(); ) {
        SearchHit hit = iterator.next();
        // if this was the last value then collect the sort values for searchAfter
        final T value = hitConverter.convert(hit);
        if (value instanceof WithScore) {
            ((WithScore) value).setScore(Float.isNaN(hit.getScore()) ? 0.0f : hit.getScore());
        }
        result.add(value);
        if (!iterator.hasNext()) {
            searchAfterSortValues = hit.getSortValues();
        }
    }
    return new Hits<T>(result.build(), toSearchAfterToken(searchAfterSortValues), limit, totalHits);
}
Also used : SearchHits(org.elasticsearch.search.SearchHits) TopHits(org.elasticsearch.search.aggregations.metrics.TopHits) TotalHits(org.apache.lucene.search.TotalHits) SearchHit(org.elasticsearch.search.SearchHit)

Aggregations

TotalHits (org.apache.lucene.search.TotalHits)18 Test (org.junit.Test)14 SearchHits (org.elasticsearch.search.SearchHits)13 SearchHit (org.elasticsearch.search.SearchHit)12 TestSupport (com.hazelcast.jet.core.test.TestSupport)9 ParallelJVMTest (com.hazelcast.test.annotation.ParallelJVMTest)9 QuickTest (com.hazelcast.test.annotation.QuickTest)9 SearchRequest (org.elasticsearch.action.search.SearchRequest)8 SearchResponse (org.elasticsearch.action.search.SearchResponse)7 BytesArray (org.elasticsearch.common.bytes.BytesArray)3 Text (org.elasticsearch.common.text.Text)3 SliceBuilder (org.elasticsearch.search.slice.SliceBuilder)3 FunctionEx (com.hazelcast.function.FunctionEx)2 TestOutbox (com.hazelcast.jet.core.test.TestOutbox)2 Prirep (com.hazelcast.jet.elastic.impl.Shard.Prirep)2 HazelcastParallelClassRunner (com.hazelcast.test.HazelcastParallelClassRunner)2 RerankerCascade (io.anserini.rerank.RerankerCascade)2 RerankerContext (io.anserini.rerank.RerankerContext)2 ScoredDocuments (io.anserini.rerank.ScoredDocuments)2 ScoreTiesAdjusterReranker (io.anserini.rerank.lib.ScoreTiesAdjusterReranker)2