3

1) Introduction :

I am interested in computing the variance of an observable $$ O=\frac{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}} $$ where $\left(a_{\ell m}, \ell \in\{1, \cdots, N\},|m| \leq \ell\right)$ and $\left(a_{\ell m}^{\prime}, \ell \in\{1, \cdots, N\},|m| \leq \ell\right)$ are independent random variables, with $a_{\ell m} \sim \mathcal{N}\left(0, C_{\ell}\right)$ for each $|m| \leq \ell$ and $a_{\ell m}^{\prime} \sim \mathcal{N}\left(0, C_{\ell}^{\prime}\right)$ for each $|m| \leq \ell .$ We recall the properties of a few basic distributions. We have :

  1. $\mathcal{N}(0, C)^{2} \sim C\chi^{2}(1)=\Gamma\left(\frac{1}{2}, 2 C\right)$,
  2. $\langle\Gamma(k, \theta)\rangle=k \theta$ and $\operatorname{Var}(\Gamma(k, \theta))=k \theta^{2}$, and
  3. $\sum_{i=1}^{N} \Gamma\left(k_{i}, \theta\right) = \Gamma\left(\sum_{i=1}^{N} k_{i}, \theta\right)$ for independent summands.

2) Important precision : for each $\ell$, I have the relation $C_\ell=\dfrac{b}{b'}C'_\ell$ with $b$ and $b'$ being constants, I wonder how it could help for the rest of post.

3) Partial solution not finished (only mean $\langle O\rangle$ ) :

We have by points 1 and 3 $$ \begin{aligned} \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2} & = \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} \Gamma\left(1 / 2,2 C_{\ell}\right) \\ & = \sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right) \end{aligned}\quad(1) $$ where the summands are independent. Similarly, using points 1 and 3 again, we obtain $$ \begin{aligned} \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2} & = \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} \Gamma\left(1 / 2,2 C_{\ell}^{\prime}\right) \\ & = \sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}^{\prime}\right) \end{aligned}\quad(2) $$ where the summands are independent. By independence of the sequences $\left(a_{\ell m}, \ell \in\{1, \cdots, N\},|m| \leq \ell\right)$ and $\left(a_{\ell m}^{\prime}, \ell \in\{1, \cdots, N\},|m| \leq \ell\right)$, equations (1) and (2), we obtain $$ \begin{aligned} \langle O\rangle &=\left\langle\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}\right)^{2}\right\rangle\left\langle\left(\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}\right)^{-1}\right\rangle \\ &=\left\langle\sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right)\right\rangle\left\langle\left(\sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}^{\prime}\right)\right)^{-1}\right\rangle \end{aligned} $$

The first factor simplifies :

$$\left\langle\sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right)\right\rangle=\sum_{\ell=1}^{N}(2 \ell+1) C_{\ell}$$

As you can see, I can't conclude on the second factor (expectation of the inverse of sum of Gamma distributions), especially since I can't manage to simplify it.

I have looked for a solution on the web but none solution for the instant.

UPDATE 1:

From the following link Expectation of inverse of sum of random variables, if we have $X_i$'s ($i=1,..,n$) be i.i.d. random variables with mean $\mu$ and variance $\sigma^2$, there is a method that can be used to compute $\mathbb{E}[1/(X_1+...+X_n)]$ :

Assuming the expectation does exist, and further assuming $X$ to be positive random variables: $$ \mathbb{E}\left(\frac{1}{X_{1}+\cdots+X_{n}}\right)=\mathbb{E}\left(\int_{0}^{\infty} \exp \left(-t\left(X_{1}+\cdots+X_{n}\right)\right) \mathrm{d} t\right) $$ Interchanging the integral over $t$ with expectation: $$ \mathbb{E}\left(\int_{0}^{\infty} \exp \left(-t\left(X_{1}+\cdots+X_{n}\right)\right) \mathrm{d} t\right)=\int_{0}^{\infty} \mathbb{E}\left(\exp \left(-t\left(X_{1}+\cdots+X_{n}\right)\right)\right) \mathrm{d} t $$ Using iid property: $$ \int_{0}^{\infty} \mathbb{E}\left(\exp \left(-t\left(X_{1}+\cdots+X_{n}\right)\right)\right) \mathrm{d} t=\int_{0}^{\infty} \mathbb{E}(\exp (-t X))^{n} \mathrm{~d} t $$ So should you know the Laplace generating function $\mathcal{L}_{X}(t)=\mathbb{E}\left(\mathrm{e}^{-t X}\right)$ we have: $$ \mathbb{E}\left(\frac{1}{X_{1}+\cdots+X_{n}}\right)=\int_{0}^{\infty} \mathcal{L}_{X}(t)^{n} \mathrm{~d} t $$

How could I apply it in my case with $\Gamma$ distribution, i.e for the expectation $\Bigg\langle\left(\sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}^{\prime}\right)\right)^{-1}\Bigg\rangle$ ?

From 2) Important precision, the only thing I can reformulate is about the scale parameter $\dfrac{b}{b'}$ :

$$\Bigg\langle\left(\sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}^{\prime}\right)\right)^{-1}\Bigg\rangle=\Bigg\langle\left(\sum_{\ell=1}^{N} \dfrac{b'}{b}\Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right)\right)^{-1}\Bigg\rangle$$

UPDATE 2:

I wonder if I should rather write only :

$$\begin{aligned} \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2} & \sim \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} \Gamma\left(1 / 2,2 C_{\ell}\right) \\ & \sim \sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right) \end{aligned}$$

? what do you think about this slight modification but with important consequences on the following ?

youpilat13
  • 21
  • 1
  • 12
  • 1
    Chi-squared distributions are Gamma distributions. Although your notation is unclear, one can infer that the "$C_l$" are related to scale rather than shape. Regardless, linear combinations of Gamma distributions of different shapes are not Gamma distributions. See https://stats.stackexchange.com/questions/72479 for the calculation. – whuber Aug 06 '21 at 12:49
  • @whuber . Thanks for your quick answer. So at first sight, you would say that the reasoning is false. How could I simplify this expression $\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}$ from which I am looking for calculating the mean ? – youpilat13 Aug 06 '21 at 13:54
  • The link I provided answers that question. But you don't need to know the distribution to compute the mean: expectations are linear functions. – whuber Aug 06 '21 at 13:56
  • 1
    Re the edit: your attempt looks invalid at the outset. (I have to make assumptions about your notation and what "$a^\prime$" might refer to.) Think about the simpler case of a ratio of random variables where the denominator can take on the values $\pm 1$ with equal probability, so that (a) the fraction is just the numerator with a random sign, making it clear that if the numerator has an expectation, then so does the fraction; but (b) the expectation of the denominator is zero. What formula would you try to write in this case that is analogous to the one in your attempt? – whuber Aug 06 '21 at 15:38
  • @whuber . Sorry for the missing assumption : I assume that the $a_{lm}$ follows a Normal distribution $\mathcal{N}(0,C_\ell$) since $C_\ell$ is the variance of $a_{lm}$ for a given $\ell$ . I recall I am in a cosmology context where the temperature field of Cosmic Microwave Background (CMB) is decomposed in harmonic spherical which involve $C_\ell$ and $a_{lm}$ quantities. I hope you will understand – youpilat13 Aug 06 '21 at 16:54
  • Yes, I recognized the spherical harmonics at the outset. But my previous comments still pertain. – whuber Aug 06 '21 at 16:55
  • So, I could write simply : $\langle O\rangle=\frac{\sum_{\ell=1}^{N}(2 \ell+1) C_{\ell}}{\sum_{\ell=1}^{N}(2 \ell+1) C_{\ell}^{\prime}}$ but my issue is this $\langle O\rangle$ will be yet a random variable since I don't do statistics calculations on variable $\ell$. You see where is the issue ? – youpilat13 Aug 06 '21 at 17:02
  • 1
    You seem to be asking about the distribution of the ratio of two independent quantities, each of which is the sum of squares of independent Normal distributions of different variances. Would that be correct? If the variances were all the same, you would have an $F$ ratio distribution; but when they are not the same, you will get the messy result I linked to earlier (concerning sums of Gamma distributions) for both the numerator and the denominator. The distribution of the ratio will not then be calculable in a closed form. – whuber Aug 06 '21 at 18:09
  • @whuber . Thanks for your support. The computation of `Moschopoulos` http://www.ism.ac.jp/editsec/aism/pdf/037_3_0541.pdf is relatively complex but not apparently impossible. Could you take a look please at my **UPDATE**, I would be glad to get help about the computation of mean or variance of a linear combination of Gamma functions with different shape and scale parameters (actually in my case, shape parameter = $(2\ell+1)/2$ and scale parameter is equal to $2\,C'_\ell$. I will surely launch a bounty if needed. Regards – youpilat13 Aug 07 '21 at 19:41
  • @EdV . Thanks for your remark, have you got by chance a code or an algorithm that I could adapt for my problem ? Regards – youpilat13 Aug 07 '21 at 23:42
  • @EdV .it is kind – youpilat13 Aug 07 '21 at 23:49
  • @EdV . Thanks for sharing. it is such a pity there is not code but it already fine from your part to give all these informations. I have to go through it. – youpilat13 Aug 08 '21 at 00:49
  • 4
    This question is unpleasantly long. It could be shorter — and more appealing to read and to answer — by removing the inner sums and normal variables entirely, and asking directly about a ratio of sums of $\Gamma(\ell+\frac12,2C_\ell)$ variables. – Matt F. Aug 10 '21 at 10:24
  • is $C_{\ell}$ the variance or the standard deviation? – Spätzle Aug 16 '21 at 11:06
  • @MattF. I think you are right, my issue is directly to know how to compute the ratio of sums on $\ell$ of $\Gamma(\ell+\frac12,2C_\ell)$. Isn't there a distribution that could allow to compute the expectation and variance of a such quantity ? – youpilat13 Aug 20 '21 at 13:05
  • 1
    I was really hoping to see if there was some way to sum gamma distributions having scale parameters that were not all equal and were not all integers. Apparently, the 1985 Moschopoulos paper and the Welch-Satterthwaite approximation are still the two best options. It would be nice to be proven wrong! – Ed V Aug 20 '21 at 13:13
  • @EdV So you agree that calculation in the answer awarded is wrong ? – youpilat13 Aug 20 '21 at 13:26
  • 1
    Sadly, and with no disrespect to the person who answered, yes. The forest got lost among the trees, as it were. From looking at questions and answers here, I think there might be some viable additional approximation methods, but I have no time to follow those leads and probably not enough savvy to do so anyway. So I have been cheering you on from the sidelines, hoping you win. – Ed V Aug 20 '21 at 13:33
  • 1
    I agree with @EdV. In fact, if you had not abandoned your [first version of this question](https://stats.stackexchange.com/questions/539275) and reposted it, everyone would have the advantage of reading the comments there--of which the first one warns people against the very mistake that has been made in the answer here. I have therefore merged the old version with this new one. – whuber Aug 20 '21 at 13:57
  • @EdV . Could you provide please your code about the Moschopoulos algorithm. I saw there is also a C++ version in R ? I just want to get a nice histogram and compute myself numerically the expectation and variance from this histogram. If you could, this would be fine from you part. Regards – youpilat13 Aug 20 '21 at 14:03
  • @EdV . oh great ! thanks a lot, I would glad to test it quickly. I will provide you my email. Regards – youpilat13 Aug 21 '21 at 15:48
  • @EdV . Thank you. Ideally, a python script would be easier to code, I am going to try to do it with your documentation and the python code from https://stackoverflow.com/questions/62633941/can-we-make-this-faster-moschopoulos-algorithm that I have to understand and double check. By the way, don't hesitate to mention remarks about this algorithm or check your code with this python code. Regards – youpilat13 Aug 21 '21 at 20:34
  • ok, I will keep you informed about my code if necessary. you can wish good luck for me ! Regards – youpilat13 Aug 21 '21 at 20:49

1 Answers1

2

$$O=\frac{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}}$$

$a_{\ell m}\sim N(0,C_{\ell})$ so

Preface: I assume that $C_{\ell}={Var(a_{\ell m})}$.

Let $a_{\ell m}\sim N(0,C_{\ell})$ so $\frac{a_{\ell m}}{\sqrt{C_{\ell}}}\sim N(0,1)$. Using this, we get $a_{\ell m}^2=C_{\ell}\cdot \left(\frac{a_{\ell m}}{\sqrt{C_{\ell}}}\right)^2\sim\chi^2_{C_{\ell}}$, Summing up, we get $\sum_{m=-\ell}^{\ell} {a_{\ell m}^{2}}\sim\chi^2_{(2\ell+1)C_{\ell}}$, and for the overall sum we get

$$\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}\sim\chi^2\left(\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}}\right)$$ which can also be written as $\Gamma(\frac{1}{2}\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}},2)$. For simplicity, denote $K=\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}}$, so the numerator as $\Gamma(0.5K,2)$ distribution. We can also note that $$2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}\sim\Gamma(K,1).$$

Let's observe the denominator:

$$\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}\sim\Gamma(\frac{1}{2}\sum_{\ell=1}^{N}{(2\ell+1)C'_{\ell}},2)$$ as $C'_{\ell}=\frac{b'}{b}C_{\ell}$, we get $$2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} \left(a'_{\ell m}\right)^{2}}\sim\Gamma(\frac{b'}{b}K,1).$$

As these are two Gamma variables, the ratio $O=\frac{2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}}{2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} \left(a'_{\ell m}\right)^{2}}}$ as a beta prime distribution:

$$\frac{2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}}{2\sum_{\ell=1}^{N} {\sum_{m=-\ell}^{\ell} \left(a'_{\ell m}\right)^{2}}}\sim\beta'\left(K, \frac{b'}{b}K\right)$$

So, according to the properties of the beta prime distribution, $$E\left[ \frac{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}} \right]=\frac{K}{\frac{b'}{b}K-1}=\frac{bK}{b'K-b}=\frac{b\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}}}{b'\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}} - b}$$

and if $\frac{b'}{b}K > 2$,

$$Var\left( \frac{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}} \right)=\frac{K(K+\frac{b'}{b}K-1)}{(\frac{b'}{b}K-1)^2(\frac{b'}{b}K-2)}=\frac{b^2K(bK+b'K-b)}{(b'K-b)^2(b'K-2b)}=\frac{b^3K^2+b^2K(b'k-b)}{(b'K-b)^3-b(b'k-b)^2}.$$

Dirty, but that's it.

Spätzle
  • 2,331
  • 1
  • 10
  • 25
  • Hi Spätzie ! Thanks for your detailled answer. I have a difficulty on this step : Can we have : $\begin{aligned} \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2} & \sim \sum_{\ell=1}^{N}(2 \ell+1) \chi^{2}\left(C_{\ell}\right) \\ & \sim \chi^{2}\left(\sum_{\ell=1}^{N}(2 \ell+1) C_{\ell}\right) \end{aligned}$ if I have a different value of $C_\ell$ for each different $\ell$. I wonder we can write this above when we have a linear combination of $\chi^2$. – youpilat13 Aug 17 '21 at 03:22
  • .When i say "linear combination", I talk about the sum : $\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2} \stackrel{d}{=} \sum_{\ell=1}^{N}(2 \ell+1) \chi^{2}\left(C_{\ell}\right)$ – youpilat13 Aug 17 '21 at 03:22
  • Recall that $\chi^2(N)$ is the sum $\sum_{i=1}^{N}{Z_i^2}$ where $Z_i\sim N(0,1)$. As long as you're summing over standard normal variables (which is the case, as I've shown above) you're ok to add up the $C_{\ell}$'s. – Spätzle Aug 17 '21 at 05:00
  • thanks for your support. Actually, the normal distribution is $\dfrac{a_{\ell m}}{\sqrt{C_{\ell}}} \sim N(0,1)$ and you write after : $a_{\ell m}^{2}=C_{\ell} \cdot\left(\dfrac{a_{\ell m}}{\sqrt{C_{t}}}\right)^{2} \sim \chi_{C_{\ell}}^{2}$ : how do you get rif of the first factor $C_\ell$ in the right hand side of this expression, i.e how to conclude that : $C_{\ell} \cdot\left(\dfrac{a_{\ell m}}{\sqrt{C_{t}}}\right)^{2} \sim \chi_{C_{\ell}}^{2}$ knowing we have a $C_\ell$ different for each multipole $\ell$ : – youpilat13 Aug 17 '21 at 21:54
  • is it a property of the $\chi^{2}$ to include a different $C_\ell$ in the $\chi^2$ to get the final expression $\sum\limits_{m=-\ell}^{\ell} a_{\ell m}^{2} \sim \chi_{(2 \ell+1) C_{\ell}}^{2}$ ? : this is just this detail which disturbs me. If you could double check this point, this would be fine from your part. Best regards – youpilat13 Aug 17 '21 at 21:55
  • Again - let $X_1=\sum_{i=1}^{a}{Z_i^2}\sim\chi^2(a)$ and $X_2=\sum_{i=1}^{b}{Y_i^2}\sim\chi^2(b)$, where $Z_i,Y_i\sim_{iid} N(0,1)$. If we look at $X_3=X_1+X_2$ it is the sum of $a+b$ independent squared normal RVs so $X_3\sim\chi^2(a+b)$. That's a basic property of the $\chi^2$ distribution. – Spätzle Aug 18 '21 at 06:08
  • Thanks ! There is a litte typo at the end of variance : we must put `K` instead of little `k`. Best regards – youpilat13 Aug 19 '21 at 03:54
  • Hi ! sorry to bore you again but I made a little mistake : actually, $\dfrac{C}{C'}=\dfrac{b^2}{b'^{2}}$. Wouldn't it disturb you to change your answer ? If you don't want, this is not bad, I understand. If it doesn't bore you, I could myself edit your answer to apply the modifications (if I have enough reputation) and you will validate the entire demonstration. Best regards – youpilat13 Aug 19 '21 at 20:53
  • For such case, the expected value would be $E\left[ \frac{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} a_{\ell m}^{2}}{\sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2}} \right]=\frac{b^2\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}}}{(b')^2\sum_{\ell=1}^{N}{(2\ell+1)C_{\ell}} - b^2}$ and the variance is $\frac{K(K+\frac{(b')^2}{b^2}K-1)}{(\frac{(b')^2}{b^2}K-1)^2(\frac{(b')^2}{b^2}K-2)}$, have fun simplifying the latter. – Spätzle Aug 20 '21 at 07:07
  • oh misery, I realize that condition $\frac{b'}{b}K > 2$ is not respected in my numerical computation : how to deal with this for expectation and variance ? Regards – youpilat13 Aug 20 '21 at 07:16
  • I didn't see it before for variance because in $\frac{b^2K(bK+b'K-b)}{(b'K-b)^2(b'K-2b)}$, the numerator is negative and the denominateur also. – youpilat13 Aug 20 '21 at 07:21
  • Is there an alternative to circumvent this issue of $\frac{b'}{b}K > 2$ which is not respected in my case ? – youpilat13 Aug 20 '21 at 10:04
  • Given the fact that I have $K$ value very small (~ 1e-6), maybe I should consider a logarithm function for $C_\ell$ ? – youpilat13 Aug 20 '21 at 11:15
  • I wonder if I should prefer to write only : $\begin{aligned} \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell}\left(a_{\ell m}^{\prime}\right)^{2} & \sim \sum_{\ell=1}^{N} \sum_{m=-\ell}^{\ell} \Gamma\left(1 / 2,2 C_{\ell}\right) \\ & \sim \sum_{\ell=1}^{N} \Gamma\left((2 \ell+1) / 2,2 C_{\ell}\right) \end{aligned}$ ? what do you think about this slight modification but with important consequences on the following ? – youpilat13 Aug 20 '21 at 12:29
  • 2
    This attempted solution errs at the outset: the sum of chi-squared variates with (potentially) different scale factors does *not* have a chi-squared distribution. – whuber Aug 20 '21 at 13:55
  • @whuber . Hi ! could you take a look please at the following post : https://stats.stackexchange.com/questions/543403/include-sum-into-a-chi2-when-we-have-a-sum-of-chi2 , and give your opinion. It is related to what we discuss about here. Best regards. – youpilat13 Sep 04 '21 at 10:15