1

I'm working on proving Chebyshev's Inequality. I watched this YouTube video and it almost makes sense. There is one step in the proof I don't understand. Using Markov's Inequality you substitute values and the square both sides to get:

$P((x-\mu)^2\ge\alpha)\le\mathbb{E}[(x-\mu)^2]/\alpha$

That much makes sense. But then they say (I've looked up multiple proofs and they all seem to have this step) that the term $\mathbb{E}[(x-\mu)^2]$ is the same as $\sigma^2$. This is where I get lost. $\sigma^2$ is equal to $\sum_{1}^{N}(x-\mu)^2/N$. Where did the $N$ go? I don't see how those two values are equivalent.

Grant Curell
  • 225
  • 4
  • 4
    $\sigma^2 = \mathbb{E}[(X - \mu)^2]$ is correct, where $X$ is a random variable, rather than a realisation. Where things go awry is when you write $\sigma^2 = 1 / N \sum^N_{n=1} (x - \mu)^2 $, which possibly suggests a confusion between the variance and the uncorrected sample variance. The inclusion of $\mu$ in your problematic specification of $\sigma^2$ also needs clarification. It would be helpful if you could parse in words what you think your expression for $\sigma^2$ consists of, as that would assist the community in identifying the source of your confusion. – microhaus Jun 10 '21 at 18:09

1 Answers1

5

As noted by microhaus in the comment, the formula you wrote for the variance,

$$\frac{1}{N}\sum_{n=1}^N(x_i- \overline{x})^2$$

is actually the sample variance. The formula used in the probabilistic proof of the Chebyshev inequality,

$$\sigma^2 = \mathbb{E}[(X-\mu)^2]$$

Is the second central moment or the variance. The equations are not equal and are not meant to be. Rather, the sample variance is meant to be a finite sample analog of the variance which itself is a population parameter. Intuitively, you can see that this is a natural estimator because the expectation can be thought of as an average. This discussion might also be helpful in understanding the difference.

You might find this previous question on the difference between the sample and population instructive.

microhaus
  • 2,186
  • 1
  • 4
  • 17
Ariel
  • 2,273
  • 2
  • 23
  • I’ve come back to this a few times - as far as I can tell, if you expand the expectation, aside from one having the true mean and the other having the sample mean, they are the same? Is that the only difference between the two? – Grant Curell Jun 13 '21 at 16:00
  • Well, that and the one I wrote assumes all events have equal probability but you could rewrite that formula as being the expectation with the sample mean and then that would account for that – Grant Curell Jun 13 '21 at 16:01
  • 2
    $\sigma^2$ is a population parameter. The expectation denotes that we take an integral under the distribution of $X$. The sample mean is simply a within-sample approximation to this when do not know the full distribution of $X$. The difference between the sample and the population is fundamental. For example, the above inequality does not generally hold for the sample mean (e.g. https://en.wikipedia.org/wiki/Chebyshev%27s_inequality#Finite_samples). – Ariel Jun 13 '21 at 21:41