next up previous contents
Next: Detection Evaluation Up: Evaluation of Classifiers Previous: Confusion Matrices   Contents


ROC Analysis

ROC analysis proceeds from the analysis of a special case of confusion matrix when there are only two classes: the instances can only be positive or negative. Table [*] shows graphically a general confusion matrix for this special case. The entries in the confusion matrix have the following meaning:


Table C.3: Example of confusion matrix with only two classes.
  Automatic
 
  Positive Negative
Positive  $ a$    $ b$
Negative  $ c$  


For this $ 2$ x$ 2$ confusion matrix a set of parameters [44] are typically extracted in order to evaluate the result:

A ROC graph is a plot with the false positive rate on the $ X$ -axis and the sensitivity (the true positive rate) on the $ Y$ -axis. Thus, each axis ranges from 0 to $ 1$ . The point $ (x=0, y=1)$ is the perfect classifier: it classifies all positive cases and negative cases correctly. The point $ (x=0, y=0)$ represents a classifier that predicts all cases to be negative, while the point $ (x=1, y=1)$ corresponds to a classifier that predicts every case to be positive. Point $ (x=1, y=0)$ is the classifier that is incorrect for all classifications. When no useful discrimination is achieved the true positive rate is always equal to the false positive rate, obtaining thus a point in the diagonal line from point $ (x=0, y=0)$ to point $ (x=1, y=1)$ .

However, a ROC graph has more information that a single confusion matrix. In many cases, a classifier has a parameter that can be adjusted to increase true positive rate at the cost of an increased false positive rate. Therefore, each parameter setting provides a point on the graph, and varying the parameter a curve is achieved.

Figure [*] shows an example of a ROC graph with two ROC curves labeled $ C1$ and $ C2$ , and the probability obtained by chance. Curve $ C2$ obtains better performance than curve $ C1$ , as it goes closer to the point $ (x=0, y=1)$ , the perfect classifier. A measure commonly derived form a ROC curve is the area under the curve [19], which is an indication for the overall sensitivity and specificity of the observer, commonly called $ Az$ . As closest to the upper-left-hand corner of the graph, the area increases until a maximum area of $ 1$ .

Figure C.1: Two ROC curves and the diagonal line marking the chance classifier.
\includegraphics[width=10 cm]{images/roc.eps}


next up previous contents
Next: Detection Evaluation Up: Evaluation of Classifiers Previous: Confusion Matrices   Contents
Arnau Oliver 2008-06-17