Variance of a function of a random variable as function of the original variable

Question

Suppose $X$ a random variable and $Y=f(X)$ a function of this variable

I know I can write $\mathbb V(Y)$ as $\mathbb E(f(X)^2)-\mathbb E(f(X))^2$

I would like to know if it is possible to write $\mathbb V(Y)$ as a function of $\mathbb V(X)$

e.g : $\mathbb V(Y)=g(\mathbb V(X))$ given the function $f$

In other words, I need to find an expression for the variance of the transformed variable $Y$ in which the variance of the initial variable $X$ appears explicitely, preferably as its parameter.

Details about the problem :

1. The distribution of $X$ is unknown, and I don't think we need it because we compute variances of the whole population, not a sample of it.
2. As it happens, the function $f$ is continuous, strictly decreasing, and positive (somewhat like $f(x)=1/x$ with $x>0$).

Answer 1

Let $X\sim \mathcal N(0,\sigma^2)$ denote a normal random variable and let $f$ be the function $$f(x) = \begin{cases}+1, & x > 0,\\-1, &x \leq 0.\end{cases}$$ Then, $f(X)$ is a random variable taking on values $\pm 1$ with equal probability and so $f(X)$ has variance $1$. On the other hand, if $X\sim \mathcal N(1,\sigma^2)$, then $f(X)$ takes on values $\pm 1$ with probabilities $\Phi\left(-\frac{1}{\sigma}\right)$ and $1-\Phi\left(-\frac{1}{\sigma}\right)$ and its variance is not $1$ even though the variance of $X$ is unchanged. Thus, it is not necessarily possible that $\mathbb V(f(X))$ can be expressed as a fixed function $g(\cdot)$ of $\mathbb V(X)$, that is, $\mathbb V(f(X))\neq g(\mathbb V(X))$.

Are there any functions $f(\cdot)$ for which $\mathbb V(f(X))$ equals $g(\mathbb V(X))$ for some fixed function $g(\cdot)$? Sure, there are. If $f(x) = ax+b$, then $$\mathbb V(f(X)) = \mathbb V(aX+b) = a^2\mathbb V(X) = g(\mathbb V(X))$$ where $g(x) = a^2x$.

What if $X$ is a positive random variable and $f(x) = x^{-1}$ for $x >0$? Well, one case where the desired relationship might hold is when if $X$ is a Gamma random variable with order parameter $3$ or more. See this answer of mine for some details of how this might be made to work.

Answer 2

The exact formula for the variance of $Y$ requires use of the function $f$ and the full distribution of $X$ (not just its variance). Nevertheless, while there is no exact formula of the kind you want, you can get approximate formulae using Taylor approximation (also called the "delta method").

---

To facilitate analysis using the delta method, suppose we let $\mu$, $\sigma^2$, $\gamma$ and $\kappa$ denote the mean, variance, skewness and kurtosis of $X$ respectively, and suppose that $f$ is differentiable up to the required orders for our formulae. The first-order Taylor approximation to the variance is:

$$\begin{aligned} \mathbb{V}[f(X)] &\approx f'(\mu)^2 \cdot \sigma^2. \\[6pt] \end{aligned}$$

The second-order Taylor approximation (shown in this related question and answer) is:

$$\begin{aligned} \mathbb{V}[f(X)] &\approx ( f''(\mu)^2 \mu^2 - f'(\mu)f''(\mu) \mu + f'(\mu)^2 ) \cdot \sigma^2 \\[6pt] &\quad - \frac{f''(\mu)(f'(\mu) + \mu f''(\mu))}{2} \cdot \gamma \sigma^3 + \frac{f''(\mu)^2}{4} \cdot (\kappa-1) \sigma^4. \\[6pt] \end{aligned}$$

So, you can get an approximating formula that uses only the mean and variance of $X$ by using the first-order Taylor approximation to the variance. This is not an exact result, and it is not a particularly good approximation. If you are willing to also use the skewness and kurtosis of $X$ then you can use the second-order Taylor approximation to the variance. This is also not an exact result, but it is a reasonable approximation in a wide class of cases.

Answer 3

Let's see how far we can towards characterizing functions $f$ where such a formula will work. We know it works for linear functions, but are there any others? How about when the random variables $X$ have restricted values?

---

The setting of the question is one in which $f$ is given but the distribution of the random variable $X$ is unknown and arbitrary--although possibly with restrictions on the values it can assume. Thus, the appropriate sense of such a formula "working" would be

For which functions $f$ is there an associated function $V_{f}:\mathbb{R}^+\to \mathbb{R}^+$ satisfying $$V_{f}(\operatorname{Var}(X))=\operatorname{Var}(f(X))\tag{*}$$ for all random variables $X:\Omega\to \mathcal{X}\subset \mathbb{R}$?

($\Omega$ is some abstract probability space whose details don't matter.)

Let's exploit the basic properties of variance to explore these possibilities. Begin by dismissing the trivial cases where $\mathcal X$ is empty or has just one element, or when $f$ is a constant function, thereby enabling us to assume $\mathcal X$ contains two numbers $x_1$ and $x_2$ for which $f(x_2)\ne f(x_1).$ The random variables $X$ whose values are confined to this subset $\{x_1,x_2\}$ are all (at most) binary. When $\Pr(X=x_2)=p,$ direct calculation establishes $$\operatorname{Var}(X) = p(1-p)(x_2-x_1)^2.$$ The same calculation yields $$\operatorname{Var}(f(X)) = p(1-p)(f(x_2)-f(x_1))^2.$$

Keeping $(x_1,x_2)$ fixed for a moment, write $a=(x_2-x_1)^2 \gt 0$ and $b=(f(x_2)-f(x_1))^2 \gt 0.$ As $p$ varies through the interval $[0,1],$ $p(1-p)a = \lambda$ varies through the interval $[0,a/4].$ In terms of $(*),$ the preceding results tell us

$$V_f(\lambda) = V_f(\operatorname{Var}(X)) = \operatorname{Var}(f(X)) = \frac{b}{a}(\lambda).$$

This exhibits $V_f$ as a non-constant linear function defined on the interval $[0,a/4].$ Consequently, because the $x_i$ are arbitrary, $V_f$ must be a non-constant linear function defined on all values $\lambda$ from $0$ through the supremum of $(x_2-x_1)^2/4$ (which might be infinite).

This solves the problem when $\mathcal X$ has at most two elements. Suppose it has a third element $x\in\mathcal X$ distinct from $x_1$ and $x_2.$ Let $\mu$ (obviously non-negative) be the slope of $V_f$ as previously established. Thus

$$\mu = \frac{(f(x_2)-f(x))^2}{(x_2-x)^2} = \frac{(f(x_1)-f(x))^2}{(x_1-x)^2}.$$

Clearing the denominators and taking square roots produces

$$f(x_i) - f(x) = \pm \mu(x_i-x),\ i=1,2.\tag{**}$$

But it's also the case that $f(x_2) - f(x_1)=\pm \mu (x_2-x_1).$ You can (readily) check that this contradicts $(**)$ unless all the signs of all the square roots are the same. Consequently,

$f$ must be an affine function of the form $f(x) = \nu + \mu x$ for some constant numbers $\nu$ and $\mu.$

This conclusion subsumes the earlier cases where the cardinality of $\mathcal X$ is $0,$ $1,$ or $2.$

Of course--to close this logical loop--when $f$ has this form, $V_f(\lambda) = \mu^2\lambda$ is the rule for computing the variance of $f(X)$ in terms of the variance of $X$ for any random variable $X.$