7

I'm trying to understand the confidence interval returned by the function var.test() in R. More specifically, the confidence interval returned by var.test() is not the one I find when doing the calculation of the F-test by myself. For example :

> s1 <- 10:12 ; s2 <- 13:16
> var(s1)
[1] 1
> var(s2)
[1] 1.666667
> var.test(s1,s2)

    F test to compare two variances

data:  s1 and s2 
F = 0.6, num df = 2, denom df = 3, p-value = 0.7926
alternative hypothesis: true ratio of variances is not equal to 1 
95 percent confidence interval:
  0.03739691 23.49929674 
sample estimates:
ratio of variances 
               0.6

The 95% confidence interval here is [0.037,23.499]. I interpret "confidence interval" as "rejection region", i.e. if the test statistic F is inside this interval, the null hypothesis should be accepted, for a given statistical level (95% here). However, when I try to calculate this, I find :

> qf(c(0.025,0.975),length(s1)-1,length(s2)-1)
[1]  0.02553268 16.04410643

So I guess I'm wrong when I interpret var.test()'s "confidence interval" as the "rejection region".

So my question is : what does this confidence interval represent?

chl
  • 50,972
  • 18
  • 205
  • 364
ehmicky
  • 295
  • 1
  • 3
  • 10
  • 1
    Off the top of my head - `qf()` is giving you the quantiles of an F distribution where the variances are EQUAL. The confidence interval in `var.test()` is drawing a confidence region around the estimated variance ratio. I imagine that uses a non-central F distribution, but have no idea how you derive the non-centrality parameter. – atiretoo Jun 30 '12 at 22:55
  • Thanks a lot ! Reading your reply, I'm guessing what the "confidence region" might stand for. For example here, it would mean, assuming s1 is a sample of the population p1, and s2 of p2 "the ratio between the sample variance of s1 and s2 is 0.6, but the ratio between the real variance of p1 and p2 has 95% odds to lie between 0.037 and 23.499" ? I.e. the confidence interval of F, and not the rejection region. As for the way I calculate the rejection region, I'm probably wrong by using the central F-distribution, I'm still learning statistics and not comfortable with it. – ehmicky Jun 30 '12 at 23:17
  • 2
    You certainly shouldn't interpret "confidence interval" as "rejection region". Take the usual t interval for a regression coefficient estimate - the $100(1-\alpha)\%$ confidence interval comprises the range of values that would _not_ be rejected as null values for the $\alpha$-level $t$-test. So, in that situation, "confidence interval" and "rejection region" are disjoint (and complementary) sets. In other situations, where the confidence interval is produced by an entirely different method from that used to conduct the hypothesis test, the two may not have such an obvious relationship. – Macro Jul 01 '12 at 05:26

2 Answers2

6

Your method using qf computes the rejection region that you would compare the ratio of the 2 variance to. Using the central F is the correct thing to do because we calculate the rejection region assuming the null hypothesis is true and if the null (that the variances are equal) is true then we have a central F distribution.

The derivation for the formula for the confidence interval goes along these lines (I only show the lower limit, some slight modifications give the upper limit)

$ \frac{v1}{\sigma_1^2} / \frac{v2}{\sigma_2^2} \sim f(df_1,df_2) $

$ \frac{v1}{v2} \times \frac{\sigma_1^2}{\sigma_2^2} \sim f(df_1,df_2)$

$prob( \frac{v1}{v2} \times \frac{\sigma_2^2}{\sigma_1^2} > f_{0.975} ) = 0.025 $

$prob( \frac{\sigma_1^2}{\sigma_2^2} < \frac{v1}{v2}/f_{0.975} )= 0.025$

So the confidence interval is also based on the central F since it is estimating the ratio of the 2 true variances.

And in R to get the value "manually" you do

vr <- var(s1)/var(s2)
vr/qf(.975,2,3)

which matches the result of var.test()

atiretoo
  • 1,458
  • 14
  • 29
Greg Snow
  • 46,563
  • 2
  • 90
  • 159
  • So the value that `var.test()` reports for the lower limit is the value of v1/v2 that makes that last equation true? Very nice, thank you. – atiretoo Jul 01 '12 at 16:26
5

The rejection region for a test and a confidence interval are different things. The rejection region is the region where you reject the null hypothesis that the ratio of variances equals $1$. A $100(1-α)\%$ confidence interval is an interval that has the property that in repeated sampling would include the true parameter value in $100(1-α)\%$ of the cases. There is a 1-1 correspondence between a confidence interval and the related hypothesis test but that does not mean that the rejection region equals the confidence region even when the confidence level is the same as the significance level for the test.

chl
  • 50,972
  • 18
  • 205
  • 364
Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Thanks for your reply. By "true parameter", do you mean here the ratio of variances ? If so, how useful is this confidence interval ? Indeed, should one use the actual F computed from a given sample to determine if the null hypothesis is wrong, or use the F ratios that would probably be computed for other samples (confidence interval) ? For example, instead of computing the p-value for a given F, compute the minimum and maximum p-value for a given confidence interval for F ? – ehmicky Jul 01 '12 at 17:53
  • 1
    In this example the parameter would be the ratio of the two variances. Hypothesis tests and confidence intervals serve different purposes. A confidence interval tells you what the pluasible values for the parameter are. The p-value for the hypothesis test just tells how unlikely it would be to observe a value for the test statistic that is as extreme or more extreme if the null hypothesis were true. The conventional thing to do when testing that the variances are equal is to compute the p-value based on the test statistic for the given sample. I see no reason to do what you are suggesting. – Michael R. Chernick Jul 01 '12 at 18:24
  • So I understand the practical use of confidence interval for statistics like mean or variance, but not for a test statistic like F here. For example, I want to test whether heights of Englishmen are more concentrated around the mean than Frenchmen. I use a variance F-test with n = 200 for both samples to do so, and with a F of 1.01, I see it's unlikely to be the case. Now, I know F is 1.01, but has a 95% interval of [0.8,1.2]. How does that help me understanding my problem ? Is it a measure of the reliability of my F-test ? – ehmicky Jul 01 '12 at 20:02
  • I don't think you would normally use confidence intervals for ratios of variances. The F statistic would most likely be used for testing equality of the variances. – Michael R. Chernick Jul 01 '12 at 20:07
  • So you suggest confidence intervals are more useful for practical use with other tests than variance equality F-test ? – ehmicky Jul 01 '12 at 20:11
  • Confidence intervals are useful when you want to estimate parameters expressing the uncertainty of the estimate in addition to the point estimate. That could even be true for the ratio of variances but is not likely to be so. – Michael R. Chernick Jul 01 '12 at 20:36