12

I have run Levene's and Bartlett's test on groups of data from one of my experiments to validate that I am not violating ANOVA's assumption of homogeneity of variances. I'd like to check with you guys that I'm not making any wrong assumptions, if you don't mind :D

The p-value returned by both of those tests is the probability that my data, if it were generated again using equal variances, would be the same. Thus, using those tests, to be able to say that I do not violate ANOVA's assumption of homogeneity of variances, I would only need a p-value that is higher than a chosen alpha level (say 0.05)?

E.g., with the data I am currently using, the Bartlett's test returns p=0.57, while the Levene's test (well they call it a Brown-Forsythe Levene-type test) gives a p=0.95. That means, no matter which test I use, I can say that the data I meet the assumption. Am I making any mistake?

Thanks.

levesque
  • 784
  • 3
  • 10
  • 18

3 Answers3

8

The p-value of your significance test can be interpreted as the probability of observing the value of the relevant statistic as or more extreme than the value you actually observed, given that the null hypothesis is true. (note that the p-value makes no reference to what values of the statistic are likely under the alternative hypothesis)

EDIT: in mathematical terminology, this can be written as: $$p-value = Pr(T > T_{obs} | H_{0})$$ where $T$ is some function of the data (the "statistic") and $T_{obs}$ is the actual value of $T$ observed; $H_{0}$ denotes the conditions implied by the null hypothesis on the sampling distribution of $T$.

You can never be sure that you're assumptions hold true, only whether or not the data you observed is consistent with your assumptions. A p-value gives a rough measure of this consistency.

A p-value does not give the probability that the same data will be observed, only the probability that the value of the statistic is as or more extreme to the value observed, given the null hypothesis.

probabilityislogic
  • 22,555
  • 4
  • 76
  • 97
  • Just one note on p-values (in relation to my bracketed comment), it could very well be that in the case that you have "unusual" data (say p-value of 0.0001). BUT it may also be the case that it is *even more unusual* under the alternative hypothesis (say a p-value of $10^{-30}$ when you switch the null and alternative hypothesis around). I believe this can happen when the statistic $T$ is not a sufficient statistic for the hypothesis test. – probabilityislogic Jan 25 '11 at 05:18
  • ..continuing...It may also be the case that you have very "good" data (say p-value of 0.5). BUT the alternative hypothesis may be *better* (or more consistent) with this data (say p-value of 0.99999 when the null and alternative hypothesis are switched around). – probabilityislogic Jan 25 '11 at 05:21
5

You're on "the right side of the p-value." I'd just adjust your statement slightly to say that, IF the groups had equal variances in their populations, this result of p=0.95 indicates that random sampling using these n-sizes would produce variances this far apart or farther 95% of the time. In other words, strictly speaking it's correct to phrase the result in terms of what it says about the null hypothersis, but not in terms of what it says about the future.

rolando2
  • 11,645
  • 1
  • 39
  • 60
  • I remember the interpretation of the p-value as (in this case): when assuming that the null-hypotheses (i.e., homogeneity of variances) is correct, then the probability of obtaining this or a more extreme result (i.e., contrary the null) is 57% or 95%. But whatsoever, the conclusion is the same and correct. – Henrik Jan 24 '11 at 08:52
3

While the previous comments are 100% correct, the plots produced for model objects in R provide a graphic summary of this question. Personally, i always find the plots much more useful than the p value, as one can transform the data afterward and spot the changes immediately in the plot.

richiemorrisroe
  • 2,666
  • 17
  • 16
  • 2
    Well said, another thing is that a p-value tells you nothing about *what to do* if the null hypothesis is "rejected", but a plot of the data gives you a clue as to the problem – probabilityislogic Jan 24 '11 at 16:13