0

Suppose we have a normally distributed random variable $X$ representing some population.

Suppose we draw $n$ samples from the population: $$ (X_1,X_2,\dots,X_n). $$ This can be done using the rnorm function in R.

Suppose we plot the density of the sample (the density can be obtained using the density function in R). It is clear that as we take larger values of $n$, the more the density of the sample approaches the density of the population $X$.

How is this described mathematically? I'm not so familiar with convergence in probability/statistics setting yet, so roughly speaking, I am looking for something along the lines of $$ |\text{pdf}(X) - \text{pdf}(\text{sample}(n))| \to 0, \quad \quad \text{as} \ n \to \infty. $$

So how does the proper mathematical description go?

ManUtdBloke
  • 743
  • 4
  • 13
  • 1
    See https://en.wikipedia.org/wiki/Empirical_distribution_function#Asymptotic_properties – Sergio Sep 12 '20 at 13:52
  • 1
    The sample *never* has a density, because its empirical distribution is discrete. You need to work with the CDF, not the PDF. See my illustration at https://stats.stackexchange.com/a/3904/919. – whuber Sep 12 '20 at 14:54
  • @whuber If we use the R density() function on the sample it gives a result that approaches the population density on a plot as $n \to \infty$. Or plot a histogram with a very small bin size, and the result visually approaches the population density. I realise we are dealing with a discrete density for the samples and a continuous density for the population, my question is how to connect the relationship between these two objects mathematically as visually it is clear that they can be made close to each other in some sense. It is that notion of 'sense' that I am trying to make precise. – ManUtdBloke Sep 12 '20 at 15:42
  • @whuber Your other answer is very helpful for the intuition, but I want a mathematical statement for the convergence phenomenon I am seeing on the plot/histogram of the sample density. – ManUtdBloke Sep 12 '20 at 15:46
  • 2
    You can find several mathematical statements for the convergence from an empirical CDF (`ecdf`, much more 'empirical' than a kernel density estimate) to the CDF in https://en.wikipedia.org/wiki/Empirical_distribution_function#Asymptotic_properties – Sergio Sep 12 '20 at 16:01
  • 1
    `density` is a kernel density estimator which by default uses a Gaussian kernel whose width is determined from the data. All those details are needed to analyze its behavior, especially the last one. The correct object to analyze remains the CDF. Indeed, you can view the `R` `density` output as the convolution of the ECDF with the Gaussian kernel. Provided the kernel's width goes to zero, the problem breaks into two parts: (1) show the ECDF converges to the underlying distribution (a classical result) and (2) show the convolution converges to the identity function: this is easy. – whuber Sep 12 '20 at 17:46
  • 1
    Check "Glivenko–Cantelli theorem" in Wikipedia. – user295357 Sep 12 '20 at 20:36

0 Answers0