2

Consider the linear model $Y = X \beta + e$, $e \sim N(0, V(\theta))$, where $Y$ is a $n \times 1$ vector, $X$ is the $n \times p$ full rank design matrix, $V(\theta)$ is the covariance matrix. I drop $\theta$ below for convenient expression.

B is an $n \times (n-p)$ matrix whose columns are orthonormal basis of $C(X)^{\perp}$. REML maximizes the likelihood of $B^T Y$, which can be expressed as

$$ L(\theta) = (2 \pi)^{-(n-p)/2} |B^T V B|^{-1/2} \text{exp} \{- \frac{1}{2} Y^T B (B^T V B)^{-1} B^T Y \} \tag{1} $$ where $|A|$ is the determinant of matrix $A$.

This can be proved to be equivalent as

$$L(\theta) = (2 \pi)^{-(n-p)/2} |X^T X|^{1/2} |V|^{-1/2} |X^T V^{-1} X|^{-1/2} \text{exp} \{- \frac{1}{2} Y^T V^{-1} (I - Q) Y \} \tag{2} $$ where $Q = X (X^T V^{-1} X)^{-1}X^T V^{-1}$.

I am able to prove that $$I - Q = V B (B^T V B)^{-1} B^T \tag{3} $$ by showing that they are the same projection operator onto $C(VB)$ along $C(X)$, and thus prove the $\text{exp}$ part.

However, I don't know how to prove

$$|B^T V B| = |X^T X|^{-1} |V| |X^T V^{-1} X| \tag{4} $$

Any suggestion would help.

GZ1995
  • 88
  • 7
  • Can you please add the source of these derivations? – usεr11852 Dec 28 '17 at 00:17
  • I use my notation in the derivation, but there are some sources. For example, page 20 of this [paper](http://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=9681&context=rtd). Also, [this](https://stats.stackexchange.com/questions/205485/how-are-calculations-done-for-reml) is another relevant post, citing "Harville 1974 showed that". – GZ1995 Dec 28 '17 at 03:02
  • Note that it is obvious when $V=$ identity since $B^\top V B = B^\top B = I_{n-p}$ then. So this is a question about dot product. – Yves Dec 28 '17 at 19:56
  • @Yves Sorry I still fail to get the point. Could you please elaborate on how to use dot product to show the equality? Thanks. – GZ1995 Dec 28 '17 at 20:53
  • Sorry I thought that orthonormality related to the dot product $ := y^\top V^{-1}y'$ (which is not the case) and that the result would come by transformation. It seems to be quite tricky, and I will try to understand ``Harville 1974'' or later references that might be more detailed. – Yves Dec 29 '17 at 11:11
  • This article by Lynn R. LaMotte https://link.springer.com/article/10.1007/s00362-006-0335-6 can be relevant (I do not have access to it). You could also compute the derivative of $\log\{|V||X^\top V^{-1} X||B^\top VB|^{-1}\}$ w.r.t. the matrix $V$ to show that this is a constant. – Yves Dec 30 '17 at 08:46

1 Answers1

2

Thanks to the comment by @Yves, the proof of the identity is listed as Proposition 2 in the paper a direct derivation of the reml likelihood function by Lynn R. LaMotte. Because the paper may not be open accessed, I show the proof here with slight change of notation.

Because $\begin{pmatrix} X & B \end{pmatrix}$ is a n by n matrix with $B^T X = 0$, we have

\begin{align} |B^T B| |X^T X| |V| &= |V| | \begin{pmatrix} X^TX & X^TB \\ B^TX & B^TB \end{pmatrix} | \tag{5} \\ &= | \begin{pmatrix} X^T \\ B^T \end{pmatrix} V \begin{pmatrix} X & B \end{pmatrix} | \tag{6} \\ &= |\begin{pmatrix} X^T V X & X^T V B \\ B^T V X & B^T V B \end{pmatrix}| \\ &= |B^T V B| |X^T V X - X^T V B (B^T V B)^{-1} B^T V X| \tag{7} \\ &= |B^T V B | |X^T [V - V B (B^T V B)^{-1} B^T V] X| \\ &= |B^T V B | |X^T X (X^T V^{-1} X)^{-1} X^T X| \tag{8} \\ &= |B^T V B | |X^T X|^2 |X^T V^{-1} X|^{-1} \tag{9} \end{align}

where (5) and (7) follows from rule of determinant of block matrix, (6) and (9) follows from the rule of determinant of product of square matrix, and (8) follows from (3). Finally, using the fact that $B^T B = I$ (orthonormal basis), (4) is proved.

GZ1995
  • 88
  • 7