1

Background: in the full-rank model let $\mathbf{b}$ be a $p\times 1$ matrix of independent standard normal random variables and $\mathbf{X}$ a $n\times p$ matrix where $\mathbf{y} = \mathbf{Xb} + \mathbf{e}$, where $\mathbf{b}$ is obtained by solving the Normal Equation.

I want to test if some combination of the elements of $\mathbf{b}$ are significant., and in a course I took the F-statistic was derived as shown below. I can't reconcile this with the F-statistic in the book Elements of Statistical Learning (eqn 3.13 p48).

F-statistic version 1

I start with $H_0: C\mathbf{b} = \boldsymbol{\delta}^*$ where $C$ is an $r \times p$ matrix of rank $r \le p$ and $\boldsymbol{\delta}^*$ is a vector of constants.

From a course I took on linear regression an F-statistic was derived by combining the expectation and variance of $\mathbf{b}$ to make a quadratic form, essentially if the null hypothesis is true $C\mathbf{b} = \boldsymbol{\delta}^*$ (where $\boldsymbol{\delta}^*$ is an $r\times 1$ vector of constants) and the statistic is

$F_{r,n-p} \sim \frac{(C\mathbf{b}-\boldsymbol{\delta}^*)^T \left[ C(X^TX)^{-1}C^T \right]^{-1}(C\mathbf{b}-\boldsymbol{\delta}^*)/r}{SS_{res}/(n-p)}$

with $SS_res = (y-Xb)^T(y-Xb)$ (residual sum of squares)

F-statistic version 2 Reading Elements of Statistical Learning I'm given

$F_{r,n-p}\sim\frac{RSS_0 - RSS_1/r}{RSS_1/(n-p)}$, where $RSS_1$ is the residual sum-of-squares for the least squares fit of the bigger model and $RSS_0$ the same for the nested smaller model.

Translating the above into matrix notation I get:

$\frac{(y-XCb)^T(y-XCb) - (y-Xb)^T(y-Xb)/r}{SS_{res}/(n-p)}$

However I cannot get this to equal F-statistic version 1. I know version 2 is a special case of version 1, but I'm stuck.

I found this post useful, but I don't quite follow the notation: Proof that F-statistic follows F-distribution

0 Answers0