7

Say the success probability $X$ is a random variable with mean $\mu$ and Variance $\sigma^2$ which takes values in $[0,1]$. How can I compute the variance of a random Variable $Y$ which is 1 with probability $X$ and 0 with probability $1-X$. So there are "two layers" of variance.

Silverfish
  • 20,678
  • 23
  • 92
  • 180
jonaden
  • 87
  • 6
  • Is this a homework or practice problem? It sounds like you're already given the variance. – Arya McCarthy May 16 '21 at 15:54
  • 1
    It is just something that I want to understand myself. I mean that there is now another random variable $Y$ which is 1 with probability $X$ (which is random itself) and 0 with probability $1- X$. So there are "two layers" of variance and I dont know how to deal with this. – jonaden May 16 '21 at 16:00
  • Hi, please edit important details like that into the question itself. – Arya McCarthy May 16 '21 at 16:02

1 Answers1

8

In general, you solve problems like this using the 'law of iterated variance'.

Let $Y|X \sim \text{Bern}(X)$ and use your stipulated prior mean and variance for the success probability $X$. Using the law of iterated variance, you get:

$$\begin{align} \mathbb{V}(Y) &= \mathbb{V}(\mathbb{E}(Y|X)) + \mathbb{E}(\mathbb{V}(Y|X)) \\[6pt] &= \mathbb{V}(X) + \mathbb{E}(X(1-X)) \\[6pt] &= \mathbb{V}(X) + \mathbb{E}(X) - \mathbb{E}(X^2) \\[6pt] &= \mathbb{E}(X^2) - \mathbb{E}(X)^2 + \mathbb{E}(X) - \mathbb{E}(X^2) \\[6pt] &= \mathbb{E}(X) - \mathbb{E}(X)^2 \\[6pt] &= \mu(1-\mu). \\[6pt] \end{align}$$

As you can see, the variance of $Y$ is determined by $\mu$, and is unaffected by $\sigma$. It turns out that this is true for all the moments of $Y$. In fact, we have $\mathbb{E}(f(Y)) = f(0) + [f(1)-f(0)] \mu$, so the expectation of any function of $Y$ is determined by $\mu$ and is unaffected by $\sigma$.

Ben
  • 91,027
  • 3
  • 150
  • 376
  • 1
    Thank you. This answers my question perfectly! However, I think there is a slight mistake in the second to last row, which also changes the result. – jonaden May 16 '21 at 23:19
  • @Ben Surely, higher-order moments of $Y$ are _not_ affected by $\sigma^2$ when $Y|X\sim \operatorname{Bern}(X)$ since the marginal point mass function of $Y$ can not be anything other than that of a Bernoulli random variable. – Jarle Tufto May 17 '21 at 07:44
  • The higher-order moments still depend on $X$, and $\sigma$ affects the distribution of $X$, so I would expect it to affect the higher-order moments. – Ben May 17 '21 at 09:11
  • @Ben No, the marginal distribution of $Y$ is clearly a Bernoulli distribution. Once its mean $EY=E(EY|X)=EX=\mu$ is given, the two point masses can only be $P(Y=0)=1-\mu$ and $P(Y=1)=\mu$ and the higher order moments are the same as that of any ordinary Bernoulli distribution and do not depend on $\sigma^2$. – Jarle Tufto May 17 '21 at 09:35
  • 1
    Okay, yes, I see what you mean. That is correct. I have edited the answer to revise this part. – Ben May 17 '21 at 10:42