9

It is common knowledge that: $$\begin{equation}\label{3} Var(X) \geq 0 \end{equation}$$ for every random variable $X$. Despite this, I do not remember seeing a formal proof of this.

Is there a proof of the above inequality? What if we include the realm of complex numbers, does this open up the possibility to the above inequality being wrong?

Community
  • 1
OliverVD
  • 129
  • 1
  • 5
  • 10
    The variance is the expectation of a squared quantity. The meaning of "$\lt$" and related relations are *defined* in mathematics by declaring all squares to be non-negative. Do you therefore need any proof? For another approach in which it is obvious variances are non-negative, see https://stats.stackexchange.com/a/18200/919 (where variances are expressed as areas). For complex numbers nothing changes: the definition of complex variance is arranged to agree with the definition for real numbers and to always be non-negative. – whuber May 25 '21 at 19:40
  • 2
    This can also be seen from Cauchy-Schwarz inequality. – StubbornAtom May 26 '21 at 08:39
  • 1
    It's simple to reason that if rv can have more than one value than inequality holds true, otherwise it's 0 – quester May 26 '21 at 14:03

2 Answers2

30

Go to your definition of variance:

$$ \operatorname{Var}(X) = \int(x-\mu)^2f(x)\,dx $$

The $(x-\mu)^2$ component is non-negative, and the $f(x)$ component is non-negative, so the integrand, $(x-\mu)^2f(x)$ is non-negative.

When you integrate an integrand that is always at the x-axis or above, the area under that curve will be non-negative.

This might be a bit easier to see if the variance is written as a sum (for a discrete variable):

$$ \operatorname{Var}(X) = \sum_i p(x_i)(x_i -\mu)^2 $$

As before, $p(x_i)\ge 0$ for all $x_i$, and $(x_i - \mu)^2\ge 0$ for all $x_i$, so that is a sum of non-negative values.

Michael Hardy
  • 7,094
  • 1
  • 20
  • 38
Dave
  • 28,473
  • 4
  • 52
  • 104
  • 1
    If you really wanted to be formal and prove the integral inequality, it can be proved from the fundamental theorem of calculus. – qwr May 26 '21 at 06:26
  • 5
    @qwr That is overkill, the inequality follows directly by the definition of the integral (limit of sums which are non-negative). The FTC only applies when one has a continuous density function. – Jannik Pitt May 26 '21 at 09:47
  • 1
    @Dave, typo in final line, the *square* of difference from mean is positive. – Jivan Pal May 26 '21 at 19:14
  • 2
    I keep seeing people saying that an integral such as the one here is the DEFINITION of variance or of expected value. But this is a _theorem_, not a _definition_. One can say $$ \operatorname E(X) = \int_\Omega X(\omega) P(d\omega) = \int_{\mathbb R} xf_X(x)\,dx $$ and the first equality, not the second, is the definition. Look at this: $\cdots \qquad$ – Michael Hardy May 26 '21 at 22:23
  • $$ \begin{align} & \operatorname{var}(X) = \operatorname E\big( (X-\mu)^2\big) \\ & {} \qquad \quad \, \uparrow \quad \text{This is the definition.} \\ {} \\ {} \\ = {} & \int_{\mathbb R} f_{(X-\mu)^2} (u)\, du \\ {} \\ = {} & \int_{\mathbb R} (u-\mu)^2 f_X(u) \, du. \end{align} $$ Why would the second integral above have a better claim than the first to be called the definition? $\qquad$ – Michael Hardy May 26 '21 at 22:25
  • 1
    @MichaelHardy. Do you have any examples where taking the second integral rather than the first to be the definition leads one awry? – microhaus May 26 '21 at 23:59
  • 2
    @microhaus It isn’t that one is right and the other is wrong. It has to do with something called the law of the unconscious statistician. The equation in terms of its expected value is the true definition of the second central moment. However, since it can be shown to be equal to what I wrote, pretty much everyone treats my equation as a definition. – Dave May 27 '21 at 00:03
  • 1
    @microhaus : I wasn't proposing that either of them is the definition. – Michael Hardy May 27 '21 at 03:18
7

As for your question regarding complex numbers, the variance is defined as being the expectation of the absolute value, or modulus, squared of the deviation from the mean. If the absolute value is not taken, that is referred to as the "pseudo variance".

See https://en.wikipedia.org/wiki/Complex_random_variable#Variance_and_pseudo-variance

Henry
  • 30,848
  • 1
  • 63
  • 107
Acccumulation
  • 3,688
  • 5
  • 11