Many randomised controlled trial (RCT) papers report significance tests on baseline parameters just after/before randomisation to show that the groups are indeed similar. This is often part of a "baseline characteristics" table. However, significance tests measure the probability of getting the observed (or a stronger) difference by chance, aren't they? And if the test is significant we conclude that there is a true difference because a random difference of that extent would be unlikely. Does a significance test make any sense after randomisation when we know that any difference must be due to chance?
Asked
Active
Viewed 321 times
11
-
1Readers here may also be interested in reading: [homogeneity testing of baseline characteristics in medical trials](http://stats.stackexchange.com/questions/58384/). – gung - Reinstate Monica May 10 '13 at 17:38
1 Answers
6
A hypothesis test would be nonsensical, but a significance test may be useful.
The hypothesis test would be testing a null hypothesis that is already known to be true, as your question makes clear. It is silly to apply a statistical test to any hypothesis that has a truth value already known via completely reliable information.
A significance test provides a P value that, again as you already say, indicates the probability of getting data as extreme or more extreme given the null hypothesis. However, it seems to me that such a P value can be interpreted in a manner that equates to an answer to the question "How often might I expect to see a difference in baseline values as large as this time, or larger?" The answer might be useful even if it is not clear for what purpose.
Michael Lew
- 10,995
- 2
- 29
- 47