Can I use ICC and bland altman plot to study agreement of small data?

Question

I have data set with small sample size, like 10 pairs of continuous data. I checked out kappa and percentage agreement, both seems not appropriate for agreement analysis. Can I stick to use ICC and Bland Altman plot? Any opinions are welcome.

Answer 1

Certainly you can. Of course, with your small sample size, you will be able to detect only very large deviations from agreement, e.g., in the bland-altman-plot, your horizontal confidence band around zero will be very wide.

Answer 2

With small numbers, it is hard to proove anything in terms of significance. That's where visual methods like the Bland-Altman-Plot make a lot of sense to give you a feeling, where there is no significance. Beware though, that the "Limits of Agreement", i. e. the two lines above and below the mean of the differences, are spaced 2 times (or 1.96 times) the standard deviation from the mean, which is derived from the normal distribution and not the t-distribution, which makes a difference only in small datasets. Additional Information: a fast way to produce Bland-Altman-Plots in R is through the package BlandAltmanLeh at CRAN but of course you can do it by Hand almost as quickly.