Agreement is the degree to which two raters, instruments, etc, give the same value when applied to the same object. Special statistical methods have been designed for this task.
Agreement is the degree to which two raters, instruments, etc., give the same value (rating / measurement) when applied to the same object. Agreement can be assessed to determine if one measurement can be substituted for another, the reliability of a measurement, etc. Trying to assess agreement using a correlation coefficient (or perhaps a chi-squared test for categorical variables) is a very common and intuitive mistake. Special statistical methods have been designed for this task.
Some references:
- The Wikipedia entry on inter-rater agreement.
- Jon Uebersax's website on agreement statistics.
- Robinson, W.S. (1057). The statistical measurement of agreement. American Sociological Review, 22, 1, pp. 17-25.
- Bland J.M. & Altman D.G. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. Lancet, 327, 8476, pp. 307–310.
