Total Cost Ratio

Wiki MarkupA scheme for [MeasuringAccuracy] developed by Ion Androutsopoulos et al, and described in \[http://www.aueb.gr/users/ion/docs/mlnet_paper.pdf _An Evaluation of Naive Bayesian Anti-Spam Filtering_\] (Ion Androutsopoulos, John Koutsias, Konstantinos V. Chandrinos, George Paliouras and Constantine D. Spyropoulos, published in the Proceedings of the workshop on Machine Learning in the New Information Age, G. Potamias, V. Moustakis and M. van Someren (eds.), 11th European Conference on Machine Learning, Barcelona, Spain, pp. 9-17, 2000.)

Quoting that paper:

  To compare easily with the baseline, we introduce the total cost ratio (TCR):

 \[...\]
  Greater TCR indicates better performance. For TCR < 1, not using the filter
  is better. If cost is proportional to wasted time, TCR measures how much time
  is wasted to delete manually all spam messages when no filter is present,
  compared to the time wasted to delete manually any spam messages that passed
  the filter plus the time needed to recover from mistakenly blocked legitimate
  messages.

TCR uses weighted costs – it introduces a lambda value, indicating how much more important (costly) a non-spam mail is, when compared to a spam mail, based on how much effort a user would have to expend to recover from a misclassification.
The lambda value is intended to be set to a value based on how the filter deals with classified messages; for example, if the filter simply displays spam messages in a different colour in the same inbox, the lambda is 1; if the filter blocks the message and asks the sender to re-send, the lambda is 9; if the filter deletes the message silently, the lambda is 999.

...