Internal reliability for an ordinal scale

Question

My teacher is asking whether it is possible to look at the Cronbachs's alpha when looking at the internal reliability of an ordinal scale. She thinks, because you use the mean of it, it is not possible, but I've seen it in previous research as well. I think it is possible, but I cannot give a technical explanation for this. Who can help me?

Answer 1

From a practical perspective, I don't see any obvious reason to not use Cronbach's alpha with ordinal items (e.g., Likert-type items), as is commonly done in most of the studies. It is a lower bound for reliability, and is essentially used as an indicator of internal consistency of a test or questionnaire. The usual assumptions pertaining to a correct interpretation of its value are as follows: (i) no residual correlations, (ii) items have identical loadings, and (iii) the scale is unidimensional. In fact, the sole case where alpha will be essentially the same as reliability is the case of uniformly high factor loadings, no error covariances, and unidimensional instrument (1).

However, we can speak of an ordinal reliability alpha. For instance, Zumbo et coll. (2) use a polychoric correlation matrix input to calculate alpha parallel to Cronbach. Their simulation studies lead them to conclude that ordinal reliability alpha provides "consistently suitable estimates of the theoretical reliability, regardless of the magnitude of the theoretical reliability, the number of scale points, and the skewness of the scale point distributions. In contrast, coefficient alpha is in general a negatively biased estimate of reliability" for ordinal data (p. 21). Ordinal reliability alpha will normally be higher than the corresponding Cronbach’s alpha.

Otherwise, the usual Cronbach's $\alpha$ is influenced by the number of items in the test and interitem correlations (for a fixed sample size $N=300$, even with modest--albeit perfect--correlation between items, e.g. $\rho = 0.35$, Cronbach’s $\alpha$ would still be at 0.943 with 30 items, and 0.910 with 20 items). There're subtle issues with Cronbach's $\alpha$ and departure from the unidimensionality assumption (systematic errors can greatly inflate the estimate of alpha, especially with large sample sizes) or the presence of inconsistent responses (random responses may inflate Cronbach’s alpha when their mean differ from that of the true responses). If the variables being tested are all dichotomous, Cronbach’s alpha is the same as Kuder-Richardson coefficient (3).

Of note, there are alternative ways to estimate the reliability of test scores, see e.g., Zinbarg et al. (4).

A good review is

Bruce Thompson. Score Reliability. Contemporary Thinking on Reliability issues. Sage Publications, 2003.

References

1. T Raykov. Scale reliability, Cronbach’s coefficient alpha, and violations of essential tau-equivalence for fixed congeneric components. Multivariate Behavioral Research, 32: 329-254, 1997.
2. B D Zumbo, A M Gadermann, and C Zeisser. Ordinal versions of coefficients alpha and theta for likert rating scales. Journal of Modern Applied Statistical Methods, 6: 21-29, 2007.
3. G F Kuder and M W Richardson. The theory of the estimation of test reliability. Psychometrika, 2: 151-160, 1937.
4. R E Zinbarg, W Revelle, I Yovel, and W Li. Cronbach’s $\alpha$, Revelle’s $\beta$, and McDonald’s $\omega_h$: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1): 123-133, 2005.