Robert M. Losee,
When Information Retrieval Measures Agree about the Relative Quality of Document Rankings,
Journal of the American Society for Information Science, 51(9), pp. 834-840, 2000.
(Won JASIS Best Paper of the Year Award, 2000)
(pdf of full article)

Abstract:

The variety of performance measures available for information retrieval systems, search engines, and network filtering agents can be confusing to both practitioners and scholars. Most discussions about these measures address their theoretical foundations and the characteristics of a measure that make it desirable for a particular application. In this work, we consider how the preferences shown by measures of performance at two points in a search may be formally compared. Criteria are developed that allow one to determine the percent of time or conditions under which two different performance measures suggest that one document ordering is superior to another ordering, or when the two measures disagree about the relative value of document orderings. As an example, graphs provide illustrations of the relationships between precision and E and F measures.

Return to Losee home page at http://www.ils.unc.edu/~losee