5

The Kappa ($\kappa$) test is a Z-test kind of test. If I am not very wrong, to compute the $\kappa$ test, we can just estimate the appropriate variance $\hat {var}(\hat\kappa)$ for the kappa statistic $\hat\kappa$ and then feed it to a z-test by taking $\mu$ = $\hat\kappa$ and $\sigma^2$ = $var(\hat\kappa)$.

To compute the power of a z-test one can use the relation $1 - \beta = \phi(Z_{a} - \sqrt n * (\mu-\mu_0)/\sigma)$

Would the power of this underlying z-test be also the power of the original $\kappa$ test? If not, why?

Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
Cesar
  • 984
  • 1
  • 9
  • 21

1 Answers1

2

The kappa statistic does not have a normal distribution. It is asymptotically normal which means that the normal gives a good approximation in large samples. In large samples the approximate power of the test can be based on the normal distribution. Note that the variance of kappa has a special form that should be used when estimating var(κˆ) just like using p^(1-p^)/n when using the normal approximation to the binomial. For kappa The asymptotic variance of the simple kappa coefficient is computed as

(A+B-C)/(1-Pe) $^2$) n

where

A=∑ pii (1-(pi.+p.i)(1-κ^)) $^2$

i

B=(1-κ^))$^2$ ∑∑ pij (p.i+pj.)$^2$

           i≠j

C=( κ^-Pe(1-κ^))$^2$ )

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Thanks, this makes sense. I am glad to see the asymptotic variance you posted. It is the same I found in other stats books - and the one I am using. However, I am a little worried that the Kappa variance traditionally referred in the remote sensing literature is very different; would you by the way also know the reason for that? The remote sensing version is the one reported by Congalton, page 106: http://www.scribd.com/doc/53393988/60/Kappa – Cesar Jun 23 '12 at 22:53
  • There are various forms of Kappa. i was referring to Cohen's. There is also one dur to Fliess. – Michael R. Chernick Jun 23 '12 at 23:03
  • Thanks. But that one is due to Cohen's as well... Still don't know why the difference. – Cesar Jun 23 '12 at 23:08