Can I do a t test if I have little to no variance in one group?

Question

I have 4 groups that I am comparing to a criterion. In one of my groups, all the participants answered the same on every item, i.e. there is no variance.

How do I deal with that in my ANOVA?

Also, what do I make of that in a t test I am running comparing it to a criterion since I will get no error term? If I do include one participant that I am not sure I am including in my student, the variance is not totally uniform with 1 different observation out of 37 but when I run it, it is not significant because the variance is too small.

I understand that there is nothing I can do computation-wise. I am asking how one handles it conceptually.

Answer 1

Here are a few observations to add to the existing answers. I think it's important to think through conceptually why you are getting a group with zero variance.

Floor and ceiling effects

In my experience in psychology, this example comes up most often when there is a floor or ceiling on a scale, and you have some groups that fall in the middle of the scale and others who fall on the extreme. For example, If your dependent variable is proportion of items correct out of five questions, then you might find that your "smart" group gets 100% correct or that your "clinical group" gets 0% correct.

In this case:

• You might want to fall back on ordinal non-parametric tests if you have no variance in one of your groups.
• Although it may not help you after the fact, you may also want to think conceptually about whether a different measure that did not have floor or ceiling effects would have been better to use. In some cases it wont matter. For example, the point of the analysis may have been to show that one group could perform a task and another could not. In other cases, you may want to model individual differences in all groups, in which case you may need a scale that does not suffer from floor or ceiling effects.

Very small group size

Another case where you can get no group variance is where you have a group with a really small sample size (e.g., $n\lt5$), usually in combination with a dependent variable that is fairly discrete.

In this case, you may be more inclined to put the lack of variance down to chance, and proceed with a standard t-test.

Answer 2

A couple of years ago, I would have fully subscribed to @Michael Chernick's answer.

However, I realized recently that some implementations of the t-test are extremely robust to inequality of variances. In particular, in R the function t.test has a default parameter var.equal=FALSE, which means that it does not simply rely on a pooled estimate of the variance. Instead, it uses the Welch-Satterthwaite approximate degrees of freedom, which compensates for unequal variances.

Let's see an example.

set.seed(123)
x <- rnorm(100)
y <- rnorm(100, sd=0.00001)
# x and y have 0 mean, but very different variance.
t.test(x,y)
Welch Two Sample t-test

data:  x and y 
t = 0.9904, df = 99, p-value = 0.3244
alternative hypothesis: true difference in means is not equal to 0 
95 percent confidence interval:
 -0.09071549  0.27152946 
sample estimates:
    mean of x     mean of y 
 9.040591e-02 -1.075468e-06

You can see that R claims to perform Welch's t-test and not Student's t-test. Here the degree of freedom is claimed to be 99, even though each sample has size 100, so here the function essentially tests the first sample against the fixed value 0.

You can verify yourself that this implementation gives correct (i.e. uniform) p-values for two samples with very different variances.

Now, this was for a two-sample t-test. My own experience with ANOVA is that it is much more sensitive to inequality of variances. In that case, I fully agree with @Michael Chernick.

Answer 3

If you assume that the variances are the same for each group you can get a pooled variance estimate and work with it in constructing t tests for pairwise differences. But that would not be a good assumption unless all the variances were small and the one with all identical values was just a chance occurrence. If you cannot do that then you have no way to estimate the variance for that one group and cannot do the analysis of variance or any t test involving that group as one of the pairs being compared.

Answer 4

Under certain circumstances it may be possible to calculate an upper bound on what the variance for the population could be, and then use that variance in something such as a t-test with unequal variances.

For example, if you asked 10 randomly chosen students in a school of 100 students what is their favorite day in March and they all answered the 15th, you know that the largest variance you could possibly have for the student population is the variance for 10 values of 15, 45 values of 1, and 45 values of 31, which is 204.6364.

A larger variance should make detecting a difference more difficult, so that a t-test using this upper bound on the variance would be conservative in detecting a difference. That means you would be sure of a significant difference resulting from a t-test using the upper bound on the variance, but if you did not find a significant difference, you wouldn't know much, because a significant difference would still be consistent with some of the smaller variances that are possible.

Of course there may not be many situations where you can actually figure this out, but it might be possible.