0

I have two groups with small number of n (less than 10) where some observations are replicates for each other, some are paired difference observations, and some are completely independent. For example:

Group I        Group II
A <--paired--> B
C (unpaired)
D <--paired--> E
F <-| replicate of D

Is there any known way to test the hypothesis that the mean of Group I != mean of Group II?

rook p
  • 1

1 Answers1

1

It sounds like you have an undesigned experiment that presents serious problems for interpretation of the results. First, it is usual to have either paired or unpaired observations and a mixture is awkward (as you apparently know). Second, an important assumption in many significance tests is that the data are a random sample and, given the undesigned appearance of the data, I suspect that may not be the situation in this case. Thus a conventional hypothesis test is not appropriate.

A couple of strategies are worthy of consideration.

  1. Segregate the unpaired and paired observations and analyse them separately. Their results should agree with respect to the size of the effect. 'Agree' here is a bit vague, and you should not expect very close agreement with small samples--the same direction of effect in both sets is probably as much as you can expect unless the effect is very large relative to the relevant variances.

  2. Treat all of the observations as unpaired (use the average of the replicates D and F as a single observation).

It is vital to note that neither approach is reliable, and before making any strong or important conclusion you really will need to either (i) make valid arguments supporting your conclusions that almost need no statistical support from these data or (ii) to repeat the experiment using a better design. Statistical analysis cannot do much to repair 'faulty' data like yours.

Michael Lew
  • 10,995
  • 2
  • 29
  • 47
  • Well, for what it's worth, my employer was the one who "undesigned" the experiment and left me to pick up the pieces. Is there maybe a Monte Carlo simulation that approximates a conventional t-test that I could generalize to tolerate poorly designed data? – rook p Jan 17 '13 at 16:35
  • +1, nice points. It reminds me of [this quote](http://stats.stackexchange.com/questions/726/famous-statistician-quotes#739). – gung - Reinstate Monica Jan 17 '13 at 23:32