4

This is part of problem 5.23 in A First Course in Linear Model Theory, Dey and Ravishanker. It was on a previous midterm and I didn't know how to do it, but now I am studying for the final and would like to figure it out.

Suppose $\mathbf{x} \sim N_{m}(\mu, \Sigma)$ and $A$ is a symmetric idempotent matrix with rank $p<m$.

Edit: Show that $Cov(\mathbf{x}, \mathbf{x}^\prime A \mathbf{x})=2\Sigma A \mu$

I can find $E(\mathbf{x}^\prime A \mathbf{x})$ and $Var(\mathbf{x}^\prime A \mathbf{x})$ using the formula for the cumulants of $\mathbf{x}^\prime A \mathbf{x}$,

$$ \kappa (\mathbf{x}^\prime A \mathbf{x})=2^{r-1}(r-1)![tr(A\Sigma)^r+r\mu^\prime A(\Sigma A)^{r-1}\mu] $$

but I don't see how I can get the covariance above using the cumulants without more information. Somehow I need to calculate $E(\mathbf{x} \mathbf{x}^\prime A \mathbf{x})$ but I can't figure out how to find this quantity. Hints at how to get started would be great.

caburke
  • 1,342
  • 10
  • 12

2 Answers2

5

The standard normal distribution makes everything OK. If $\newcommand{\Cov}{\mathrm{Cov}}\mathbf{x}\sim N_m(\mu , \Sigma)$, there exists a matrix $V$ such that $VV^\prime=\Sigma$ and $\mathbf{x}=\mu+V\mathbf{z}$, where $\mathbf{z}\sim N_m(\mathbf{0}, \mathbf{I})$. Then my problem reduces to finding,

$$ \begin{equation} \begin{split} \Cov(\mu+V\mathbf{z},(\mu+V\mathbf{z})^\prime A (\mu+V\mathbf{z}))&=V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})+V\Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)\\ &\quad+V\Cov(\mathbf{z},\mu^\prime AV\mathbf{z}) \end{split} \end{equation} $$

Note that $\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z} $ is equal in distribution to $(-\mathbf{z})(-\mathbf{z})^\prime V^\prime AV(-\mathbf{z})=-\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z} $, so

$$ \Cov(\mathbf{z},\mathbf{z}^\prime V^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime AV\mathbf{z})=\mathbf{0}. $$

The term $\mathbf{z}^\prime V^\prime A \mu$ is a scalar and is equal to its transpose, giving

$$ \Cov(\mathbf{z},\mathbf{z}^\prime V^\prime A \mu)=\Cov(\mathbf{z},\mu^\prime AV\mathbf{z})=E(\mathbf{z}\mathbf{z}^\prime V^\prime A \mu)=E(\mathbf{z}\mathbf{z}^\prime)V^\prime A\mu=V^\prime A\mu. $$

The result follows since $VV^\prime A \mu=\Sigma A\mu$.

caburke
  • 1,342
  • 10
  • 12
  • 2
    (+1) With a minor note: $\newcommand{\z}{\mathbf z}\z \z^\prime V^\prime AV\z$ is not symmetric about the origin, however $\z \z^\prime V^\prime AV\z$ and $-\z (-\z)^\prime V^\prime AV(-\z) = -\z \z^\prime V^\prime AV\z$ have the same distribution, which is all you need. – cardinal Dec 12 '12 at 13:59
  • 1
    Good point. I was thinking one dimensionally. – caburke Dec 12 '12 at 15:31
0

By the multivariate form of Stein's Lemma, if you let $h(X) = X'AX$, then you immediately have $$ Cov\left(X,h(X)\right)=\Sigma E\left[\nabla h(X)\right] = \Sigma E\left[2AX\right], $$ giving the result.

steveo'america
  • 503
  • 2
  • 11