6

Which test should I use, if I want to compare the mean of the whole sample with the mean of the fraction of the sample?

So for example the mean of the whole classroom taking the test and the mean of the girls from that classroom.

Thank you very much!

Marlene
  • 61
  • 1
  • 3
    Are you sure you'd want to let the subsample influence the mean of the overall sample for the purpose of this comparison? If not, you could more easily perform an independent samples *t*-test or a robust alternative that compared the subsample with the rest of the sample (excluding the subsample). – Nick Stauner Jul 22 '14 at 08:47
  • Yes, Im sure, but maybe my example above was not the best, so I give another: I would like to see, if the mean of one demographic region differs from the mean of the whole country. I used one sample t-test for that, but Im not sure I did it right. – Marlene Jul 22 '14 at 08:54
  • I think for the demographic example I would tend to use one-sample statistics, like chi-square goodness of fit, one-sample wilcoxon, or one sample t-test. These assume that the values for the country are something like known theoretical expectations. – Sal Mangiafico Jun 10 '18 at 12:24

1 Answers1

7

The problem is that the subset is correlated with the whole.

This makes them dependent. Normally the right thing to do is compare two distinct sets - the subset with what's not in the subset.

Logically, the comparison of the whole with the subset is identical to comparing the subsets --

If they behave alike, then the subset behaves like the whole.

This is fairly simple logic. If girls have the same distribution as boys (the null in the test) and girls obviously have the same distribution as themselves(!), then girls have the same distribution as girls+boys.

Compare the answer here (almost identical in content, but about a different test):

How to compare two samples of frequencies with categorical x values where one is subset of the other

So you can just compare girls vs boys; you are automatically comparing girls vs girls+boys when you do.

So if you would say something like a t-test would be suitable, use an ordinary t-test.

If something akin to a gamma GLM seems suitable for your data, you might use that. If you would use a permutation test to compare the means, do that. If you want to assume a location-shift alternative and want to use a rank-test, the Wilcoxon-Mann-Whitney would work, and so on.

Let's look in more detail at what happens if you nevertheless insist on comparing the part with the whole.

Consider a statistic like the numerator of a two-sample t-test - the difference in means (then we'll standardize it on the denominator like with a t-test once we figure out what the numerator is).

That is, we start with the overall mean - mean of the girls:

$$\bar{y}_{boys+girls} - \bar{y}_{girls} $$

$$=\frac{1}{n_b + n_g}[\sum_{boys} y_i + \sum_{girls} y_i ] - \frac{1}{n_g}\sum_{girls} y_i\,.$$

What happens if you work through the algebra is that you just end up with a scaled version of mean of the boys - mean of the girls:

$$= \frac{n_g \sum_{boys} y_i + n_g \sum_{girls} y_i - (n_b + n_g)\sum_{girls} y_i }{(n_b+n_g)n_g}$$

$$=\frac{\frac{1}{n_b}\sum_{boys} y_i - \frac{1}{n_g}\sum_{girls} y_i}{1+\frac{n_g}{n_b}} $$

$$=\frac{\bar{y}_{boys} - \bar{y}_{girls}}{k} $$

where $k=1+\frac{n_g}{n_b}$.

If you properly standardize that difference for that scaling, you'll just end up with an ordinary t-test of boys vs girls (because the thing you divide this numerator by will also be divided by a factor of $k$).

So in fact, there's not only no logical difference, but if you go ahead and try it anyway (as long as you do that correctly!), you'll end up with exactly the same test statistic.

If instead you do a permutation test of difference in means (overall-girls), you have the same issue - the test statistic will only be a scaled version of the usual one (boys-girls) and you will again have the same p-value.

Glen_b
  • 257,508
  • 32
  • 553
  • 939