Using hit curves to compare search algorithm performance

Jorge R. Herskovic, M. Sriram Iyengar, Elmer V. Bernstam

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


Databases continue to grow but the metrics available to evaluate information retrieval systems have not changed. Large collections such as MEDLINE and the World Wide Web contain many relevant documents for common queries. Ranking is therefore increasingly important and successful information retrieval systems, such as Google, have emphasized ranking. However, existing evaluation metrics such as precision and recall, do not directly account for ranking. This paper describes a novel way of measuring information retrieval performance using weighted hit curves adapted from the field of statistical detection to reflect multiple desirable characteristics such as relevance, importance, and methodologic quality. In statistical detection, hit curves have been proposed to represent occurrence of interesting events during a detection process. Similarly, hit curves can be used to study the position of relevant documents within large result sets. We describe hit curves in light of a formal model of information retrieval, show how hit curves represent system performance including ranking, and define ways to statistically compare performance of multiple systems using hit curves. We provide example scenarios where traditional measures are less suitable than hit curves and conclude that hit curves may be useful for evaluating retrieval from large collections where ranking performance is crucial.

Original languageEnglish (US)
Pages (from-to)93-99
Number of pages7
JournalJournal of Biomedical Informatics
Issue number2
StatePublished - Apr 2007
Externally publishedYes


  • Hit curves
  • Information storage and retrieval
  • Precision
  • Ranking
  • Recall
  • Retrieval evaluation
  • Statistical detection

ASJC Scopus subject areas

  • Computer Science Applications
  • Health Informatics


Dive into the research topics of 'Using hit curves to compare search algorithm performance'. Together they form a unique fingerprint.

Cite this