0

I came across an example in Casella and Berger's Statistical Inference (Example 10.2.1) where we have a random variable, $X$, that $(1-\delta)\times100$% of the time is equal to a $N(\mu, \sigma^{2})$ random variable, and the remaining $\delta \times 100$% of the time, $X$ is equal to a random variable with unknown distribution but whose mean is $\theta$ and variance is $\tau^{2}$.

The book claims:

$Var(X) = (1-\delta)\sigma^{2} + \delta \tau^{2} + \delta(1-\delta)(\theta - \mu)^{2}$

I'm having trouble deriving this same formula. My logic going into this is $Var(X) = E[X^{2}] - (E[X])^{2}$, where $E[X] = (1-\delta)\mu + \delta \theta$.

$E[X^{2}]$, I think, is just a weighted sum of the second moments of the two component random variables, i.e. $(1-\delta)(\sigma^{2} + \mu^{2}) + \delta(\tau^{2} + \theta^{2})$.

But when I compute $E[X^{2}] - (E[X])^{2}$, I don't get the simple expression in the book. Is my approach incorrect?

Moreover, how is this different than a traditional linear combination of two random variables? I don't think the variance expression in the book is derived from the traditional expression for the variance of a linear combination of two random variables, i.e. $Var(X) = (1-\delta)^{2}\sigma^{2} + \delta^{2}\tau^{2} + 2Cov(\text{component}_{1}, \text{component}_{2})$.

Frank F
  • 13
  • 3
  • See https://stats.stackexchange.com/questions/16608/what-is-the-variance-of-the-weighted-mixture-of-two-gaussians/16609#16609. – whuber Jul 11 '18 at 19:03
  • I'd argue this is a similar, but different problem. The problem in the post cited frames the random variable as a linear combination of two random variables; this post explicitly frames the problem as $X|Y$, rather than as $X = pY_{1} + (1-p)Y_{2}$. If this is a duplicate, then the solution to my problem (applying law of total variance) is a much more straightforward solution for the original. Lastly - I was alluding to the solution for the linked post: $E[X^2]−(E[X])^2$, where $X = (1-\alpha)X_{1} + (\alpha)X_{2}$, but had asked why this wasn't resulting in the expected solution. – Frank F Jul 13 '18 at 21:44
  • A mixture can indeed be formulated in terms of conditional distribution, as well as a linear combination of random variables--but not in the way you seem to be thinking. One way to learn why a particular approach is not working is to review a successful approach to the same problem, which is why we believe the references to other threads will be useful. – whuber Jul 13 '18 at 22:14

1 Answers1

2

Try using the Law of Total Variance (Theorem 4.4.7 in Casella & Berger) with $Y\sim \text{Bernoulli}(\delta)$.

tddevlin
  • 3,205
  • 1
  • 12
  • 27
  • Oooo nice! I always forget to use that. You're right. Ok, so if we let $Y\sim \text{Bernoulli}(\delta)$, then $E[X|Y] = (1-y)\mu + y\theta$, because once we know $Y$ then the mean of $X$ is either $\mu$ or $\theta$. Similarly, $Var(X|Y) = (1-y)\sigma^{2} + y\tau^{2}$. $E[Var(X|Y)] = (1-\delta)\sigma^{2} + \delta\tau^{2}$. $Var(E[X|Y]) = Var(\mu -y(\mu - \theta)) = (\mu - \theta)^{2}(\delta)(1-\delta)$ (using the expression for the variance of a bernoulli RV). Thanks! – Frank F Jul 11 '18 at 18:18