1

I'm learning statistics but don't really understand the result of kappa value posted in the first sample here.

Is there anyone help me explain in English what the value 0.915 means? (observed proportionate agreement) and the kappa value 0.801. Does 0.915 is the value that both A and B agree with the totally 94 proposals? Or something?

amoeba
  • 93,463
  • 28
  • 275
  • 317
Peter Sm
  • 11
  • 4
  • Welcome to [stats.se]! Please take a moment to view our [tour]. This appears to be a [tag:self-study] question. If that is the case, please read our [wiki](http://stats.stackexchange.com/tags/self-study/info) and tag your question as well. – Tavrock Mar 14 '17 at 09:58
  • Thank you for your suggestion. I just added "self-study" tag. – Peter Sm Mar 14 '17 at 10:04
  • @JeffreyGirard This is asking about the meaning of various Kappa values, which although a simple question, is not covered in the link to another question provided. – Carl Mar 19 '17 at 18:55
  • 1
    @usεr11852 I suppose, but I don't like the answers there either; too much wind for too little in the sails. If you want to close this question, I think my simple answer would need porting over, see below. – Carl Mar 19 '17 at 22:10
  • 1
    @usεr11852 Oh, bother. So fully go there and vote please: https://stats.meta.stackexchange.com/questions/4690/proposal-to-merge-cohens-kappa-q-a/4692#4692 – Carl Mar 19 '17 at 23:16
  • @Tavrock This is NOT a self-study question. Self-study tag is for routine textbook/lecture exercises. This is not an exercise. I removed the tag. – amoeba Mar 20 '17 at 15:01

1 Answers1

1

$$\kappa=\frac{p_o-p_c}{1-p_c}$$ where $p_o$ is the percent observed agreement and $p_c$ is the percent chance agreement (estimated).

The interpretation of a kappa coefficient (Cohen's or otherwise) is the amount of observed non-chance agreement divided by the possible amount of non-chance agreement. So if you correctly classified 80% of the items, but we would expect you to have gotten 20% of them correct by chance alone, then you would have a kappa value of $0.75$ which is smaller than observed agreement (i.e., 80%).

$$\kappa=\frac{0.80-0.20}{1-0.20}=\frac{0.60}{0.80}=0.75$$

Jeffrey Girard
  • 3,922
  • 1
  • 13
  • 36