1

Let's say we are asking if a vaccine is effective in mice. We test the vaccine vs control in groups of 10 mice, and the vaccine does confer a significant immune response (based on a linear mixed model with repeated measures, if that makes any difference).

We repeat the experiment twice more, keeping as much as possible the same between experiments, and in these two replicate experiments there is not a significant level of protection.

What conditions (if any) would have to be true for pooling data to be appropriate? By pooling data, I mean that instead of considering three separate groups of 10 mice each, we would treat the data as if they came from a single group of 30 mice.

Intuitively, it doesn't seem to me that three separate 10-mouse experiments should be the same as a single 30-mouse experiment, but whether that's right or not I'd like to understand the explanation.

amoeba
  • 93,463
  • 28
  • 275
  • 317
iayork
  • 107
  • 8

1 Answers1

1

"What conditions would have to be true for pooling data to be appropriate?" - Not having done these separate analyses, i.e. if it were a priori decided that this would be the experimental design, but even then blocking by test would be prudent because 'as much as possible' is still never the same. The latter point, accounting for test run as a random factor, addresses questions such as possible across-experiment differences e.g. seasonality/slight differences in vaccine preparations, etc. The much larger issue of having done the analyses to an unsatisfactory outcome followed by changing the game plan post factum simply cannot be addressed, and both sets of analyses must be reported.

katya
  • 2,084
  • 8
  • 11
  • Is there an argument *against* keeping the replicates separate, and reporting them as such? Aside from the point that analyzing them separately is less likely to show significance (which I take as an advantage, since it is a more conservative approach), do we gain anything by pooling? (As for changing the game plan *post factum*, it has turned out that half our research team thought we would be pooling data, half thought that we would not; so it's not quite clear which direction is *post factum*. Obviously this is something we will explicitly spell out in the future.) – iayork Aug 12 '15 at 18:50
  • Your question seems to justify them as separate, so no argument against presenting them as such. I think pooling is a misnomer, because even if you analyze all data, accounting for separate tests is necessary. As for "analyzing them separately is less likely to show significance", that is actually hard to predict without seeing the data. – katya Aug 12 '15 at 19:28