Package org.elasticsearch.index.rankeval

Interface EvaluationMetric

All Superinterfaces:
NamedWriteable, org.elasticsearch.xcontent.ToXContent, org.elasticsearch.xcontent.ToXContentObject, Writeable
All Known Implementing Classes:
DiscountedCumulativeGain, ExpectedReciprocalRank, MeanReciprocalRank, PrecisionAtK, RecallAtK
public interface EvaluationMetric extends org.elasticsearch.xcontent.ToXContentObject, NamedWriteable
Implementations of EvaluationMetric need to provide a way to compute the quality metric for a result list returned by some search (@link SearchHits) and a list of rated documents.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.elasticsearch.xcontent.ToXContent

    org.elasticsearch.xcontent.ToXContent.DelegatingMapParams, org.elasticsearch.xcontent.ToXContent.MapParams, org.elasticsearch.xcontent.ToXContent.Params

    Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

    Writeable.Reader<V extends Object>, Writeable.Writer<V extends Object>

  • Field Summary

    Fields inherited from interface org.elasticsearch.xcontent.ToXContent

    EMPTY_PARAMS

  • Method Summary

    Modifier and Type
    Method
    Description
    default double
    combine(Collection<EvalQueryQuality> partialResults)
    Combine several EvalQueryQuality results into the overall evaluation score.
    EvalQueryQuality
    evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs)
    Evaluates a single ranking evaluation case.
    static List<org.elasticsearch.index.rankeval.RatedDocument.DocumentKey>
    filterUnratedDocuments(List<RatedSearchHit> ratedHits)
    Filter RatedSearchHits that do not have a rating.
    default OptionalInt
    forcedSearchSize()
    Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
    static List<RatedSearchHit>
    joinHitsWithRatings(SearchHit[] hits, List<RatedDocument> ratedDocs)
    Joins hits with rated documents using the joint _index/_id document key.

    Methods inherited from interface org.elasticsearch.common.io.stream.NamedWriteable

    getWriteableName

    Methods inherited from interface org.elasticsearch.xcontent.ToXContent

    toXContent

    Methods inherited from interface org.elasticsearch.xcontent.ToXContentObject

    isFragment

    Methods inherited from interface org.elasticsearch.common.io.stream.Writeable

    writeTo

  • Method Details

    • evaluate

      EvalQueryQuality evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs)
      Evaluates a single ranking evaluation case.
      Parameters:
      taskId - an identifier of the query for which the search ranking is evaluated
      hits - the search result hits
      ratedDocs - the documents that contain the document rating for this query case
      Returns:
      an EvalQueryQuality instance that contains the metric score with respect to the provided search hits and ratings

    • joinHitsWithRatings

      static List<RatedSearchHit> joinHitsWithRatings(SearchHit[] hits, List<RatedDocument> ratedDocs)
      Joins hits with rated documents using the joint _index/_id document key.

    • filterUnratedDocuments

      static List<org.elasticsearch.index.rankeval.RatedDocument.DocumentKey> filterUnratedDocuments(List<RatedSearchHit> ratedHits)
      Filter RatedSearchHits that do not have a rating.

    • combine

      default double combine(Collection<EvalQueryQuality> partialResults)
      Combine several EvalQueryQuality results into the overall evaluation score. This defaults to averaging over the partial results, but can be overwritten to obtain a different behavior.

    • forcedSearchSize

      default OptionalInt forcedSearchSize()
      Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.
      Returns:
      the number of search hits this metrics requests