How to calculate the Jacobian of the transformation ( for covariance matrix)

Question

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,R\, \text{diag}(S)$, where $S$ is $k\times 1$ vector of standard deviations, $\text{diag}(S)$ is the diagonal matrix with diagonal elements $S$, and $R$ is $k \times k$ correlation matrix. Elements of the covariance matrix $\Sigma$ can be written as follows:

• $\Sigma_{ii} = s^{2}_{ii}$
• $\Sigma_{ij} = s_i s_j r_{ij}$.

Now let's assume that $\Sigma$ has a stadard inverse-Wishart distribution $W^{-1}_k(I,\nu), \nu\geq k$,that is, when $\Sigma$ has density function $$f_k(\Sigma;I,\nu) \propto |\Sigma|^{-\frac{1}{2}(\nu + k+1)}\, \exp(-\frac{1}{2}\operatorname{tr}(\Sigma^{-1}))$$ Now under the transformation $\Sigma \rightarrow (S,R)$ the Jacobian has value of $2^k\left(\displaystyle \prod _{i}^k s_i\right)^{k}$. The density function under this transformation is transformed into the following form $$f_k(S,R;I,\nu) \propto |R|^{-\frac{1}{2}(\nu + k+1)} \displaystyle \prod_i^k s_i^{-(\nu+1)} \exp\left(-\frac{r^{ii}}{2 s_i^2}\right) $$ where $r^{ii}$ is the $i$th diagonal element of $R^{-1}$.

Can someone point me in the right direction to compute this Jacobian?

Answer 1

The algebra of differential forms makes short work of this.

I take it that $\Sigma = (\sigma_{ij})$ is coordinatized by means of one of its triangles, say the upper triangle with components $\sigma_{ij},$ $1\le i\le j \le k.$ The differential element of those components, $|\prod_{ij}\mathrm{d}\sigma_{ij}|,$ therefore is the (absolute) wedge product of these $\mathrm{d}\sigma_{ij}.$ Using

$$\mathrm{d} \sigma_{ii} = 2s_i \mathrm{d}s_i\tag{1}$$

and

$$\mathrm{d} \sigma_{ij} = s_i s_j \mathrm{d} r_{ij} + s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i;\ i \ne j\tag{2}$$

it's surprisingly easy to simplify the differential element in the new coordinates:

$$\eqalign{ \prod_{ij}\mathrm{d}\sigma_{ij} &= \pm\prod_i \mathrm{d}\sigma_{ii} \wedge \prod_{i\lt j} \mathrm{d}\sigma_{ij} \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_i r_{ij} \mathrm{d} s_j + s_j r_{ij} \mathrm{d} s_i + s_is_j\mathrm{d}r_{ij}\right) \\ &= \pm\prod_i 2s_i \mathrm{d}s_i \wedge \prod_{i\lt j}\left(s_is_j\mathrm{d}r_{ij}\right) \\ &= \left(\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j\right)\ \mathrm{d} s_1\wedge \cdots \wedge \mathrm{d}s_k\ \wedge\ \mathrm{d} r_{12} \wedge \cdots \wedge dr_{k-1,k}. }$$

The second line substituted the differentials $(1)$ and $(2)$ determined by the transformation. The third line recognized that any term involving $\mathrm{d}s_i$ in the right hand product would be multiplied by one of the $\mathrm{d}s_i$ in the left hand product, resulting in zero because $\mathrm{d}s_i\wedge \mathrm{d}s_i=0.$ The last line merely pulls the constants (the "zero-forms") over to the left.

By definition, the Jacobian is the absolute value of those constants. Evidently each $s_i$ appears $k$ times. The absolute value is determined from the fact that all the $s_i$ are non-negative, yielding

$$|\pm 2^k \prod_i s_i \prod_{i \lt j} s_is_j| = 2^k \left(\prod_i s_i\right)^k,$$

QED.

---

To justify the use of this Jacobian in the Wishart context, we also need to know the transformation between $\Sigma$ and $(S,R)$ is almost surely one-to-one. But that's clear, because $(S,R)$ obviously determines $\Sigma;$ while the diagonal of $\Sigma$ determines the $s_i$ which then can be used to compute the $r_{ij}$ from the $\sigma_{ij}$ when $i\ne j.$

Answer 2

At first I was a little bit puzzled how to transform covariance matrix the way I would transform random variables (e.g. $Y_1 = X_1X_2$). Then I realized that covariance matrix $\Sigma = (\sigma_{ij})$ of order $k$ consists of $k^2$ variables and only of $\frac{k(k+1)}{2}$ unique variables (i.e. $\sigma_{ij} = \sigma_{ji}$), which can be divided into two groups (diagonal elements and let's say upper triangle without the diagonal)

1. $\sigma_{ii} = s_{ii}^2$
2. $\sigma_{ij} = s_is_jr_{ij}$

So I have $\frac{k(k+1)}{2}$-dimensional random variable $$\mathbf{X} = (\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k},\sigma_{11},\ldots,\sigma_{kk})$$ and look for $\frac{k(k+1)}{2}$-dimensional random variable $$\mathbf{Y} = (r_{12},\ldots,r_{1k},r_{23},\ldots,r_{2k},\ldots,r_{(k-1)k},s_{11},\ldots,s_{kk})\text{,}$$ where the Jacobian of the transformation has the following form

\begin{eqnarray*} \mathbf{J} &=& \begin{vmatrix} \frac{\partial \sigma_{12}}{\partial r_{12}} & \frac{\partial \sigma_{12}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{12}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{12}}{\partial s_{1}} & \frac{\partial \sigma_{12}}{\partial s_{2}} & \frac{\partial \sigma_{12}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{12}}{\partial s_{k}} \\ \frac{\partial \sigma_{13}}{\partial r_{12}} & \frac{\partial \sigma_{13}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{13}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{13}}{\partial s_{1}} & \frac{\partial \sigma_{13}}{\partial s_{2}} & \frac{\partial \sigma_{13}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{13}}{\partial s_{k}} \\ \vdots & & & & & & & & \vdots \\ \frac{\partial \sigma_{kk}}{\partial r_{12}} & \frac{\partial \sigma_{kk}}{\partial r_{13}} & \cdots & \frac{\partial \sigma_{kk}}{\partial r_{(k-1)k}} & \frac{\partial \sigma_{kk}}{\partial s_{1}} & \frac{\partial \sigma_{kk}}{\partial s_{2}} & \frac{\partial \sigma_{kk}}{\partial s_{3}} & \cdots & \frac{\partial \sigma_{kk}}{\partial s_{k}} \\ \end{vmatrix} \\ &=&\begin{vmatrix} s_1s_2 & 0 & \cdots & 0 & s_2r_{12} & s_1r_{12} & 0 & \cdots & 0\\ 0 & s_1s_3 & \cdots & 0 & s_3r_{13} & 0 & s_1r_{13} & \cdots & 0\\ \vdots & & & & & & & & \vdots \\ 0 & 0 & \cdots & 0 & 0 & 0 & 0 & \cdots & 2s_{k} \end{vmatrix} \end{eqnarray*}

This Jacobian $\mathbf{J}$ can be easily computed by Leibniz formula for determinants

$$ \text{det}(A) = \displaystyle\sum_{\tau \in S_n} \text{sgn}(\tau) \displaystyle\prod_{i=1}^n a_{i,\tau(i)} $$ This formula says that the determinant is the sum taken over all possible permutations of $1,\ldots,n$, that is, the summands are all possible products containing exactly one entry from each row and from each column. The sum has $n!$ terms. The only non-zero summand in that sum (for the Jacobian $\mathbf{J}$) is the product of diagonal elements. For diagonal elements of $\Sigma$ (i.e. $\sigma_{11},\ldots,\sigma_{kk})$ we get $2^k\left(\displaystyle\prod_{i=1}^k s_i\right)$ and for non-diagonal elements of upper triangle of $\Sigma$ (i.e. $\sigma_{12},\ldots,\sigma_{1k},\sigma_{23},\ldots,\sigma_{2k},\ldots,\sigma_{(k-1)k}$) we get $\left(\displaystyle\prod_{i=1}^k s_i^{k-1}\right)$, so the Jacobian is $$\mathbf{J} = 2^k \left(\displaystyle\prod_{i=1}^k s_i \right)^k$$ Example: Let's say we have covariance matrix of order $k=2$ $$\Sigma = \begin{pmatrix}\sigma_{11} & \sigma_{12} \\ \sigma_{21} & \sigma_{22}\end{pmatrix}$$ then the Jacobian has the following form $$ \mathbf{J} = \begin{vmatrix} s_1s_2 & s_2r_{12} & s_1r_{12}\\ 0 & 2s_1 & 0\\ 0 & 0 & 2s_2 \end{vmatrix} = 2^2\left(\displaystyle\prod_{i=1}^2s_i\right)^2$$