Search in sources :

Example 6 with LTRScoringModel

use of org.apache.solr.ltr.model.LTRScoringModel in project lucene-solr by apache.

the class TestLTRReRankingPipeline method testRescorer.

@Ignore
@Test
public void testRescorer() throws IOException {
    final Directory dir = newDirectory();
    final RandomIndexWriter w = new RandomIndexWriter(random(), dir);
    Document doc = new Document();
    doc.add(newStringField("id", "0", Field.Store.YES));
    doc.add(newTextField("field", "wizard the the the the the oz", Field.Store.NO));
    doc.add(new FloatDocValuesField("final-score", 1.0f));
    w.addDocument(doc);
    doc = new Document();
    doc.add(newStringField("id", "1", Field.Store.YES));
    // 1 extra token, but wizard and oz are close;
    doc.add(newTextField("field", "wizard oz the the the the the the", Field.Store.NO));
    doc.add(new FloatDocValuesField("final-score", 2.0f));
    w.addDocument(doc);
    final IndexReader r = w.getReader();
    w.close();
    // Do ordinary BooleanQuery:
    final BooleanQuery.Builder bqBuilder = new BooleanQuery.Builder();
    bqBuilder.add(new TermQuery(new Term("field", "wizard")), BooleanClause.Occur.SHOULD);
    bqBuilder.add(new TermQuery(new Term("field", "oz")), BooleanClause.Occur.SHOULD);
    final IndexSearcher searcher = getSearcher(r);
    // first run the standard query
    TopDocs hits = searcher.search(bqBuilder.build(), 10);
    assertEquals(2, hits.totalHits);
    assertEquals("0", searcher.doc(hits.scoreDocs[0].doc).get("id"));
    assertEquals("1", searcher.doc(hits.scoreDocs[1].doc).get("id"));
    final List<Feature> features = makeFieldValueFeatures(new int[] { 0, 1, 2 }, "final-score");
    final List<Normalizer> norms = new ArrayList<Normalizer>(Collections.nCopies(features.size(), IdentityNormalizer.INSTANCE));
    final List<Feature> allFeatures = makeFieldValueFeatures(new int[] { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 }, "final-score");
    final LTRScoringModel ltrScoringModel = TestLinearModel.createLinearModel("test", features, norms, "test", allFeatures, null);
    final LTRRescorer rescorer = new LTRRescorer(new LTRScoringQuery(ltrScoringModel));
    hits = rescorer.rescore(searcher, hits, 2);
    // rerank using the field final-score
    assertEquals("1", searcher.doc(hits.scoreDocs[0].doc).get("id"));
    assertEquals("0", searcher.doc(hits.scoreDocs[1].doc).get("id"));
    r.close();
    dir.close();
}
Also used : IndexSearcher(org.apache.lucene.search.IndexSearcher) BooleanQuery(org.apache.lucene.search.BooleanQuery) TermQuery(org.apache.lucene.search.TermQuery) Normalizer(org.apache.solr.ltr.norm.Normalizer) IdentityNormalizer(org.apache.solr.ltr.norm.IdentityNormalizer) ArrayList(java.util.ArrayList) FloatDocValuesField(org.apache.lucene.document.FloatDocValuesField) Term(org.apache.lucene.index.Term) Document(org.apache.lucene.document.Document) FieldValueFeature(org.apache.solr.ltr.feature.FieldValueFeature) Feature(org.apache.solr.ltr.feature.Feature) TopDocs(org.apache.lucene.search.TopDocs) IndexReader(org.apache.lucene.index.IndexReader) RandomIndexWriter(org.apache.lucene.index.RandomIndexWriter) LTRScoringModel(org.apache.solr.ltr.model.LTRScoringModel) Directory(org.apache.lucene.store.Directory) Ignore(org.junit.Ignore) Test(org.junit.Test)

Example 7 with LTRScoringModel

use of org.apache.solr.ltr.model.LTRScoringModel in project lucene-solr by apache.

the class ManagedModelStore method addModelFromMap.

private void addModelFromMap(Map<String, Object> modelMap) {
    try {
        final LTRScoringModel algo = fromLTRScoringModelMap(solrResourceLoader, modelMap, managedFeatureStore);
        addModel(algo);
    } catch (final ModelException e) {
        throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e);
    }
}
Also used : ModelException(org.apache.solr.ltr.model.ModelException) LTRScoringModel(org.apache.solr.ltr.model.LTRScoringModel) SolrException(org.apache.solr.common.SolrException)

Aggregations

LTRScoringModel (org.apache.solr.ltr.model.LTRScoringModel)7 ArrayList (java.util.ArrayList)5 Feature (org.apache.solr.ltr.feature.Feature)5 IdentityNormalizer (org.apache.solr.ltr.norm.IdentityNormalizer)5 Normalizer (org.apache.solr.ltr.norm.Normalizer)5 Test (org.junit.Test)5 Document (org.apache.lucene.document.Document)4 FloatDocValuesField (org.apache.lucene.document.FloatDocValuesField)4 IndexReader (org.apache.lucene.index.IndexReader)4 RandomIndexWriter (org.apache.lucene.index.RandomIndexWriter)4 Term (org.apache.lucene.index.Term)4 BooleanQuery (org.apache.lucene.search.BooleanQuery)4 IndexSearcher (org.apache.lucene.search.IndexSearcher)4 TermQuery (org.apache.lucene.search.TermQuery)4 TopDocs (org.apache.lucene.search.TopDocs)4 Directory (org.apache.lucene.store.Directory)4 ValueFeature (org.apache.solr.ltr.feature.ValueFeature)3 ModelException (org.apache.solr.ltr.model.ModelException)3 FieldValueFeature (org.apache.solr.ltr.feature.FieldValueFeature)2 Ignore (org.junit.Ignore)2