2004/058 - A probabilistic interpretation of precision, recall and F-score, with implication for evaluation
Cyril Goutte, Eric Gaussier
ECIR 27th European Conference on Information Retrieval,Santiago de Compostela, Spain, 21-23 March, 2005.
We address the problem of 1/assessing the confidence of the standard point estimates, precision, recall and F-score, and 2/ comparing the results, in terms of precision, recall and F-score, obtained using two different methods. To do so, we use a probabilistic setting which allows us to obtain posterior distributions on these performance indicators, rather than point estimates. This framework is applied to the case where different methods are run on different datasets from the same source, as well as the standard situation where competing results are obtained on the same data.