0

I have been wondering about the F test that is provided by many statistical packages along with the standard regression output. As I understand it, F can be computed by

$$ F_{df_{reg},df_{res}} = \frac{R^2/df_{reg}}{(1-R^2)/df_{res}}. $$

The hypothesis tested by this test can be formulated in two different ways:

$H_{0}$: P$^2 = 0$

$H_{1}$: P$^2 > 0$

or

$H_{0}$: All $\beta_{i} = 0$

$H_{1}$: One or more $\beta_{i} \neq 0$

The first two hypotheses seem to suggest that the F test is one-tailed, which seems to be inline with my intuition since $R^2$ can not take negative values.

The second set of hypotheses, however, suggest two-tailed testing. They, to me, also seem to suggest a direct correspondence between the outcome of the F test for the entire model ($R^2$) and the t tests for the individual coefficients (which I know may not always be the case).

So my question comes down to this: Is the F test, testing whether a model explains a significant amount of variance compared to the null model, a one-tailed or a two-tailed test and why?

Community
  • 1
crsh
  • 368
  • 5
  • 11
  • 2
    what is P$^2$ ? – Stéphane Laurent Apr 23 '14 at 10:43
  • As far as I know, F-tests are always two-tailed. I might be wrong of course, but I am pretty sure I am not. The reason is that they can only test equality restrictions, i.e. only $H_0: \hat\beta=\beta^0$ vs $H_1: \hat\beta \neq \beta^0$. In your example, only the second pair of hypotheses makes sense. – coffeinjunky Apr 23 '14 at 11:06
  • 6
    @coffee F-tests are almost always one-tailed. The reason is that the null hypothesis of equal coefficients means that all variation is random, whereas under the alternate hypothesis there is an added component of variation arising from differences among the coefficients. That would cause the numerator only to *increase.* The other (left) tail corresponds to unusual consistency among the data. Fisher famously used it to suggest that Mendel's pea data were a little too good. – whuber Apr 23 '14 at 14:03
  • @Stéphane Laurent P$^2$ is capital greek rho representing the population statistic of $R^2$. – crsh Apr 23 '14 at 18:53
  • I stand corrected! Thanks @whuber. Note though that while we only use the upper tail of the distribution for the F-test, the hypothesis we are testing is two-sided. That is, at least in standard F-tests, we are testing $\hat\beta = 0$ vs $\hat\beta \neq 0$, i.e. we test whether the estimated coefficients are either smaller or larger than zero (or whatever your null is), i.e. we do not use the F-test to test $\hat\beta > 0$ or $\hat\beta < 0$ (at least in standard F-tests, special cases may exist). – coffeinjunky Apr 24 '14 at 21:37
  • @coffee Thanks for clarifying. We refer to "tails" as a visual analogy. The F-test is based on a single tail of the distribution of the test statistic $F$. Confusion can arise when we try to visualize the alternative hypothesis in a plot of the parameter space $\{\beta_1,\ldots,\beta_p\}$. With $p=1$ parameter, the sets $\beta_1\gt 0$ and $\beta_1\lt 0$ indeed look like two separate "tails" attached to the point $\beta_1=0$. With $p=2$, however, that sense of tails vanishes. The null $\beta_1=\beta_2=0$ is still a point but there's only *one* possible tail for $H_A$: the punctured plane. – whuber Apr 24 '14 at 21:44
  • @coffee (This is a separate comment.) Please note that tests concern the *parameters* $\beta$, not their estimates. Here we are testing $\beta=0$ versus $\beta\ne 0$. (There is no uncertainty about the values of the estimates $\hat\beta$: the computer gives them to us to 17 significant figures if we want!) – whuber Apr 24 '14 at 21:45
  • You seem to have it reversed: one halves the significance level $\alpha$ only when using a (symmetric) *two*-tailed test, splitting the probability evenly across the two tails. – whuber Apr 29 '14 at 15:28

2 Answers2

0

I agree with using one sided test, the intuition is: If the model does correctly explain the variations (that is, rejection H0) then the R-Squared should be high with an obvious low 1 - R-Squared, therefore a pretty high F statistic (big number divided by small number = bigger number). However, if we were to be using a two sided test, when the F statistic is really low (that is a really small R-Squared), you might also reject H0, implying falsely the conclusion stated above. With a one sided test you avoid falling for that false conclusion.

Stingher
  • 1
0

The first time I learned the F-test, it was in the context of comparing the variances of two groups, analogous to how a two-sample t-test compares the means of two groups. If we wanted to know if the two groups had unequal variances, regardless of which variance was higher, we would do a two-sided test. If we only wanted to know if one group had higher variance than the other, we would do a one-sided test.

The F-test of nested regression models does the same, comparing the variances of the error terms in each model. With that in mind, we are not interested in detecting if the complex model has a larger variance, so we focus all of our attention to the other tail to maximize our power to detect the difference of interest: if the complex model has a smaller variance.

Thus, the usual F-test of nested regression models is one-sided.

Dave
  • 28,473
  • 4
  • 52
  • 104