Search in sources :

Example 1 with MeanReciprocalRank

use of org.opensearch.index.rankeval.MeanReciprocalRank in project OpenSearch by opensearch-project.

the class RankEvalIT method testMetrics.

/**
 * Test case checks that the default metrics are registered and usable
 */
public void testMetrics() throws IOException {
    List<RatedRequest> specifications = createTestEvaluationSpec();
    List<Supplier<EvaluationMetric>> metrics = Arrays.asList(PrecisionAtK::new, RecallAtK::new, MeanReciprocalRank::new, DiscountedCumulativeGain::new, () -> new ExpectedReciprocalRank(1));
    double[] expectedScores = new double[] { 0.4285714285714286, 1.0, 0.75, 1.6408962261063627, 0.4407738095238095 };
    int i = 0;
    for (Supplier<EvaluationMetric> metricSupplier : metrics) {
        RankEvalSpec spec = new RankEvalSpec(specifications, metricSupplier.get());
        RankEvalRequest rankEvalRequest = new RankEvalRequest(spec, new String[] { "index", "index2" });
        RankEvalResponse response = execute(rankEvalRequest, highLevelClient()::rankEval, highLevelClient()::rankEvalAsync);
        assertEquals(expectedScores[i], response.getMetricScore(), Double.MIN_VALUE);
        i++;
    }
}
Also used : EvaluationMetric(org.opensearch.index.rankeval.EvaluationMetric) RankEvalSpec(org.opensearch.index.rankeval.RankEvalSpec) DiscountedCumulativeGain(org.opensearch.index.rankeval.DiscountedCumulativeGain) RecallAtK(org.opensearch.index.rankeval.RecallAtK) PrecisionAtK(org.opensearch.index.rankeval.PrecisionAtK) ExpectedReciprocalRank(org.opensearch.index.rankeval.ExpectedReciprocalRank) RankEvalResponse(org.opensearch.index.rankeval.RankEvalResponse) RankEvalRequest(org.opensearch.index.rankeval.RankEvalRequest) MeanReciprocalRank(org.opensearch.index.rankeval.MeanReciprocalRank) RatedRequest(org.opensearch.index.rankeval.RatedRequest) Supplier(java.util.function.Supplier)

Aggregations

Supplier (java.util.function.Supplier)1 DiscountedCumulativeGain (org.opensearch.index.rankeval.DiscountedCumulativeGain)1 EvaluationMetric (org.opensearch.index.rankeval.EvaluationMetric)1 ExpectedReciprocalRank (org.opensearch.index.rankeval.ExpectedReciprocalRank)1 MeanReciprocalRank (org.opensearch.index.rankeval.MeanReciprocalRank)1 PrecisionAtK (org.opensearch.index.rankeval.PrecisionAtK)1 RankEvalRequest (org.opensearch.index.rankeval.RankEvalRequest)1 RankEvalResponse (org.opensearch.index.rankeval.RankEvalResponse)1 RankEvalSpec (org.opensearch.index.rankeval.RankEvalSpec)1 RatedRequest (org.opensearch.index.rankeval.RatedRequest)1 RecallAtK (org.opensearch.index.rankeval.RecallAtK)1