1

I've been looking into using ROC curves as a evaluation tool of a multi-class classification. The only data I have about this classification is in form of 7-by-7 confusion matrix.

Visualisation of the ROC curves is not important, result in form of AUC is sufficient.

Is this possible?

Edit: Simplified the question, omitted unnecessary presumption I have made.

famargar
  • 789
  • 1
  • 5
  • 24
Vladimir M
  • 13
  • 5
  • Welcome to CV. First, this site is not intended to be a resource for software specific questions. Then, it appears that you have assumed everyone is familiar with Hand and Till's paper. This is definitely not the case. Please elaborate and explain the key points from that paper relevant to your question. – Mike Hunter Jul 14 '16 at 11:08
  • Do you have one 7-by-7 confusion matrix or all possible confusion matrices at the different thresholds? – Calimo Jul 14 '16 at 14:05
  • @Calimo Currently I have one one matrix. Assuming that I could produce all possible confusion matrices at different thresholds, how would I go about computing the estimated AUC in multi-class case? – Vladimir M Jul 14 '16 at 14:40
  • If you go for the Hand & Till method, I'd say you'd need to take all the 2x2 sub-matrices (should be 21 of them if I can count), calculate sensitivity and specificity, compute all the 21 AUCs and then average... – Calimo Jul 14 '16 at 15:10

1 Answers1

9

You cannot construct a ROC curve from the confusion matrix alone. A confusion matrix represents a single point in the ROC space, and you need all possible confusion matrices at all thresholds to build a full curve and compute the AUC.

This holds true for multi-class ROC analysis.

Calimo
  • 2,829
  • 17
  • 26