Propagation of error using 2nd-order Taylor series

Question

I am reading a text, "Mathematical Statistics and Data Analysis" by John Rice. We are concerned with approximating the expected value and variance of the random variable $Y$. We are able to calculate the expected value and variance of the random variable $X$ and we know the relation $Y = g(X)$. So, it's possible to approximate the expected value and variance of $Y$ using the Taylor series expansion of $g$ about $\mu_X$.

On page 162, he lists 3 equations.

1. The expected value of $Y$ using the 1st-Order Taylor series expansion. It is: $\mu_Y \approx g(\mu_X)$. This is referred to later in my question as $E(Y_1)$.

2. The variance of $Y$ using the 1st-Order Taylor series expansion. It is: $\sigma_Y^2 \approx \sigma_X^2 (g'(\mu_X))^2$. This is referred to later in my question as $Var(Y_1)$.

3. The expected value of $Y$ using the 2nd-Order Taylor series expansion. It is $\mu_Y \approx g(\mu_X) + \frac12 \sigma_X^2 g''(\mu_X)$. This is referred to later in my question as $E(Y_2)$.

Note that there are two different expressions for $Y$ because we are using two different orders in the Taylor series expansion. Equations 1 and 2 refer to $Y_1 = g(X) \approx g(\mu_X) + (X-\mu_X)g'(\mu_X)$. Equation 3 refers to $Y_2 = g(X) \approx g(\mu_X) + (X-\mu_X)g'(\mu_X) + \frac12 (X-\mu_X)^2 g''(\mu_X)$.

Note that specifically the equation for $Var(Y_2)$ is not given. Later, the author seems to use the equation for the variance of $Y_1$ (Equation 2), when in fact he is referring to expected value of $Y_2$ (Equation 3). This seems to imply $Var(Y_2) = Var(Y_1)$.

I have tried to calculate by hand $Var(Y_2)$, and I am getting a somewhat complicated expression. Here is my work (I stopped because at the end I am getting $X^3$ terms in the expectation): $$ \begin{aligned} Var(Y_2) &= E[( g(\mu_X) + (X-\mu_X)a + \frac12 (X-\mu_X)^2 b - g(\mu_X) - \frac12 \sigma_X^2 b )^2] \\ &= E\left[((X-\mu_X)a + \left(\frac12 (X-\mu_X)^2 - \frac12 \sigma_X^2)b\right)^2\right] \\ &= E\left[(ca + \left(\frac12 c^2 - \frac12 \sigma_X^2)b\right)^2\right] \\ & = E[c^2 a^2 + ca(c^2 - \sigma_X^2)b + \frac14(c^2-\sigma_X^2)^2 b^2] \\ & = E[(X^2 - 2X \mu_X + \mu_X^2)a^2 + (X-\mu_X)a((X^2 - 2X\mu_X + \mu_X^2) - \sigma_X^2)b \\ & + \frac14((X^2 - 2X\mu_X + \mu_X^2)-\sigma_X^2)^2 b^2] \end{aligned} $$

Note that in the above equations, $a = g'(\mu_X)$, $b = g''(\mu_X)$, and $c = X-\mu_X$. What is $Var(Y_2)$?

Thanks.

Answer 1

Assuming $Y=g(X)$, we can derive the approximate variance of $Y$ using the second-order Taylor expansion of $g(X)$ about $\mu_X=\mathbf{E}[X]$ as follows:

$$\begin{eqnarray*} \mathbf{Var}[Y] &=& \mathbf{Var}[g(X)]\\ &\approx& \mathbf{Var}[g(\mu_X)+g'(\mu_X)(X-\mu_X)+\frac{1}{2}g''(\mu_X)(X-\mu_X)^2]\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}+\frac{1}{4}(g''(\mu_X))^2\mathbf{Var}[(X-\mu_X)^2]\\ & & +g'(\mu_X)g''(\mu_X)\mathbf{Cov}[X-\mu_X,(X-\mu_X)^2]\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}+\frac{1}{4}(g''(\mu_X))^2\mathbf{E}[(X-\mu_X)^4-\sigma_{X}^{4}]\\ & & +g'(\mu_X)g''(\mu_X)\left(\mathbf{E}(X^3)-3\mu_X(\sigma_{X}^{2}+\mu_{X}^{2})+2\mu_{X}^{3}\right)\\ &=& (g'(\mu_X))^2\sigma_{X}^{2}\\ & & +\frac{1}{4}(g''(\mu_X))^2\left(\mathbf{E}[X^4]-4\mu_X\mathbf{E}[X^3]+6\mu_{X}^{2}(\sigma_{X}^{2}+\mu_{X}^{2})-3\mu_{X}^{4}-\sigma_{X}^{4}\right)\\ & & +g'(\mu_X)g''(\mu_X)\left(\mathbf{E}(X^3)-3\mu_X(\sigma_{X}^{2}+\mu_{X}^{2})+2\mu_{X}^{3}\right)\\ \end{eqnarray*}$$

As @whuber pointed out in the comments, this can be cleaned up a bit by using the third and fourth central moments of $X$. A central moment is defined as $\mu_k=\mathbf{E}[(X-\mu_X)^k]$. Notice that $\sigma_{X}^{2}=\mu_2$. Using this new notation, we have that $$\mathbf{Var}[Y]\approx(g'(\mu_X))^2\sigma_{X}^{2}+g'(\mu_X)g''(\mu_X)\mu_3+\frac{1}{4}(g''(\mu_X))^2(\mu_4-\sigma_{X}^{4})$$