1

Suppose $X$ is a $p$-dimensional vector following $N_p(\mu,\Sigma)$ distribution, where $\mu$ is $p$-dimensional and $\Sigma$ is $p\times p$. Let $X=\left(\begin{array}{ccc}X_1\\X_2\end{array} \right)$ and $\Sigma=\left( \begin{array}{ccc} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21}&\Sigma_{22} \end{array} \right)$ where $X_1$ is $p_1\times1$ and $X_2$ is $p_2\times1$ with $p_1+p_2=p$. Also we have by usual notations $\Sigma_{12}=\text{Cov}(X_1,X_2)=\Sigma_{21}^T$. Show that $$Q:=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1$$follows $\chi^2(p_2)$.

I tried to proceed making the changes $Y=\Sigma^{-1/2}X$ and $Z=\Sigma_{11}^{-1/2}X_1$ but am not really sure if it is in the right direction. Also, I tried to proceed by considering the transformation $X_2'=X_2-\Sigma_{21}\Sigma_{11}^{-1}X_1$ but things are getting too messy and I don't know how to proceed.

Please prove some hint(s) only and not a complete solution.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Landon Carter
  • 1,295
  • 11
  • 21
  • A hint about how to _think_ of the problem: $X$ is the result of a _nonsingular_ linear transformation of $p$ _independent_ standard normal random variables $(Y_1,Y_2,\ldots, Y_p)$. $X^T\Sigma^{-1}X$ is essentially giving you $\sum_{i=1}^p Y_i^2$ while $X_1^T\Sigma_{11}^{-1}X_1$ is giving you $\sum_{i=1}^{p_1} Y_i^2$. The difference is thus $\sum_{i=p_1+1}^{p} Y_i^2$ which is a $\chi^2$ random variable with $p_2= p - p_1$ degrees of freedom – Dilip Sarwate Jun 14 '15 at 16:02
  • Yes, I could gather this information from the problem statement itself. But this does not help in solving the problem, does it? Or am I missing anything? – Landon Carter Jun 14 '15 at 17:34
  • Good! Mark Stone's hint should set you on the right track. – Dilip Sarwate Jun 14 '15 at 18:04
  • @DilipSarwate May I ask you any book/pdf for learning the Multivariate Normal distribution and the various properties, theorems, etc. on it? Also, it would be great if it is available on the internet for free. I read Rao but I don't like the presentation. – Landon Carter Jun 14 '15 at 18:14
  • 2
    C.R. Rao "Linear Statistical Inference And Its Applications" is a classic. It is tedious in places, but there's a lot packed in there. – Mark L. Stone Jun 14 '15 at 18:27
  • @DilipSarwate It also has to be shown that the quadratic forms are independent. Only then are the degrees of freedom additive. – JohnK Jun 14 '15 at 20:39

2 Answers2

2

Hint (really, really good hint):

The Schur Complement is your friend.

https://en.wikipedia.org/wiki/Schur_complement

Section A.5.5, section C.4 and numerous other occurrences in http://stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf .

Mark L. Stone
  • 12,546
  • 1
  • 31
  • 51
  • Hi! So I did as follows: Suppose the Schur Decomposition is $S=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$. Using the formula for $\Sigma^{-1}$ which I found in Wiki, I finally have $Q=X_2^T\Sigma_{22}^{-1}X_2-(2X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2)^TS^{-1}\Sigma_{12}\Sigma_{22}^{-1}X_2$. Now I know that $X_2^T\Sigma_{22}^{-1}X_2$ follows $\chi^2(p_2)$, but what can be done with the other term? It is quite weird. Will it help to consider $X_1'=X_1-\Sigma_{12}\Sigma_{22}^{-1}X_2$ so that $X_1'$ is independent of $X_2$? – Landon Carter Jun 14 '15 at 13:48
  • Actually I am not quite sure how bilinear forms like $X_1AX_2$ are distributed where $X_1$ and $X_2$ are two different vectors, maybe not of same length even. – Landon Carter Jun 14 '15 at 14:31
  • Last and final hint (otherwise I would be completely doing your homework for you, and I'm old-fashioned, because I always did all of my homework (late 70s/early 80s) by myself with no help from anyone, including T.A.s). The Schur Complement can be used to determine the conditional distribution X | X1 . – Mark L. Stone Jun 14 '15 at 15:48
  • Actually this is not a Homework problem, it came in a Semester exam. This topic will be taught to us next year, so you can say I am a beginner. Thank you for your answer. Although the final comment seemed a lot alien to me. I was thinking this can be solved using simple elementary techniques like idempotence etc. but am wrong, it seems. – Landon Carter Jun 14 '15 at 17:30
  • Schur Complement is "elementary". Anyhow, you need not know anything of the Schur Complement to solve it - there's more than one way to skin a cat, so to speak.. You could just do it yourself. But using Schur Complement, someone has already done the hard work for you, so it's easy as pie. – Mark L. Stone Jun 14 '15 at 18:29
1

There is a simple solution that was perhaps not considered.

We have

$$Q=X^T\Sigma^{-1}X-X_1^T\Sigma_{11}^{-1}X_1=X^TAX\,,$$

where $A=\Sigma^{-1}-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}$ is a symmetric matrix.

$Q$ has a $\chi^2$ distribution if and only if $A\Sigma$ is idempotent (the 'if part' is discussed here).

Now, $$A\Sigma=I_p-\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma$$

But $$\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\Sigma=\begin{pmatrix}\Sigma_{11}^{-1} & 0 \\ 0 & 0\end{pmatrix}\begin{pmatrix}\Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22}\end{pmatrix}=\begin{pmatrix}I_{p_1} & \Sigma_{11}^{-1}\Sigma_{12} \\ 0 & 0\end{pmatrix}\,,$$ which is symmetric idempotent. Hence $A\Sigma$ is also symmetric idempotent.

Degrees of freedom of $Q$ would be $\operatorname{rank}(A\Sigma)=\operatorname{tr}(A\Sigma)$. There would also be a noncentrality parameter if $\mu\ne 0$.

StubbornAtom
  • 8,662
  • 1
  • 21
  • 67