Should Dirac's delta function be regarded as a subclass of the Gaussian distribution?

Question

In Wikidata it is possible to link probability distributions (like everything else) in an ontology, e.g., that the t-distribution is a subclass of the noncentral t-distribution, see, e.g.,

https://angryloki.github.io/wikidata-graph-builder/?property=P279&item=Q209675&iterations=3&limit=3

There are various limiting cases, e.g., when the degrees of freedom in the t-distribution goes to infinity or when the variance approaches zero for the normal distribution (Gaussian distribution). In the latter case the distribution will go towards Dirac's delta function.

I note that on the English Wikipedia the variance parameter is presently stated as larger than zero, so with a strict interpretation one would not say that the Dirac's delta function is a subclass of the normal distribution. However, to me it seems quite ok, as I would say that the exponential distribution is a superclass of the Dirac's delta function.

Are there any problems with stating that Dirac's delta function is a subclass of the Gaussian distribution?

Answer 1

Dirac's delta is regarded as a Gaussian distribution when it is convenient to do so, and not so regarded when this viewpoint requires us to make exceptions.

For example, $(X_1, X_2, \ldots, X_n)$ are said to enjoy a multivariate Gaussian distribution if $\sum_i a_iX_i$ is a Gaussian random variable for all choices of real numbers $a_1, a_2, \ldots, a_n$. (Note: this is a standard definition in "advanced" statistics). Since one choice is $a_1=a_2=\cdots=a_n=0$, the standard definition treats the constant $0$ (a degenerate random variable) as a Gaussian random variable (with mean and variance $0$). On the other hand, we ignore our regard for the Dirac delta as a Gaussian distribution when we are considering something like

"The cumulative probability distribution function (CDF) of a zero-mean Gaussian random variable with standard deviation $\sigma$ is $$F_X(x) = P\{X \leq x\} = \Phi\left(\frac{x}{\sigma}\right)$$ where $\Phi(\cdot)$ is the CDF of a standard Gaussian random variable."

Note that this statement is almost right but not quite right if we regard the Dirac delta as the limiting case of a sequence of zero-mean Gaussian random variables whose standard deviation approaches $0$ (and hence as a Gaussian random variable). The CDF of the Dirac delta has value $1$ for $x \geq 0$ whereas $$\lim_{\sigma\to 0}\Phi\left(\frac{x}{\sigma}\right) = \begin{cases} 0, & x < 0,\\ \frac 12, & x = 0,\\ 1, & x > 0.\end{cases}$$ But, lots of people will tell you that regarding a Dirac delta as a Gaussian distribution is sheer nonsense since their book says that the variance of a Gaussian random variable must be a positive number (and some of them will down-vote this answer to show their displeasure). There was a very vigorous and illuminating discussion of this point a few years ago on stats.SE but unfortunately it was only in the comments on an answer (by @Macro, I believe) and not as individual answers, and I cannot find it again.

Answer 2

The delta functions fit into a mathematical theory of distributions (which is quite distinct from the theory of probability distributions, terminology here could not be more confusing).

Essentially, distributions are generalized functions. They cannot be evaluated like a function can, but then can be integrated. More precisely, a distribution $D$ is defined as follows

Let $T$ be the collection of test functions. A test function $\theta$ is a true, honest to god function, smooth, with compact support. A distribution is a linear mapping $D: T \rightarrow \mathbb{R}$

An honest function $f$ determines a distribution by the integration operator

$$ T(\theta) = \int_{-\infty}^{+\infty} f(x)\theta(x) dx $$

There are distributions that are not associated to true functions, the dirac operator is one of them

$$ \delta(\theta) = \theta(0) $$

In this sense, you can consider the dirac a limiting case of the normal distributions. If $N_t$ is the family of pdf's of normal distributions with mean zero and variance $t$, then for any test function $\theta$

$$ \theta(0) = \lim_{t \rightarrow 0} \int_{-\infty}^{+\infty} N_t(x) \theta(x) dx $$

This is probably more commonly expressed as

$$ \theta(0) = \int_{-\infty}^{+\infty} \delta(x) \theta(x) dx = \lim_{t \rightarrow 0} \int_{-\infty}^{+\infty} N_t(x) \theta(x) dx $$

which a mathematician would consider an abuse of notation, because the expression $\delta(x)$ does not actually make any sense. But then again, who am I to criticize Dirac, who is the best.

Of course, whether this makes the dirac a member of the family of normal distributions is a cultural question. Here I'm just giving a reason why it may make sense to consider it so.

Answer 3

No. It's not a subclass of normal distribution.

I think the confusion comes from one of the representations of Dirac function. Remember that it's defined as follows:

$$\int_{-\infty}^\infty\delta(x)dx=1$$ $$\delta(x)=0,\forall x\ne 0$$

It's defined as an integral, which is great but sometime you need to operationalize it by a function representation rather than an integral. So, people came up with all kinds of alternatives, one of them looks like Gaussian density: $$\delta(x)=\lim_{\sigma\to 0} \frac{e^{\frac{-x^2}{2\sigma^2}}}{\sqrt{2\pi}\sigma}$$

However, this is not the only representation, e.g. there's this one: $$\delta(x)=\frac{1}{2\pi}\sum_{k=-\infty}^\infty e^{ikx}, \forall x\in (-\pi,\pi)$$

Hence, it's best to think of Dirac function in terms of its integral definition, and take the function representations, such as Gaussian, as tools of convenience.

UPDATE To @whuber's point, a better even example is this representation of Dirac's delta:

$$\delta(x)=\lim_{\sigma\to 0} \frac{e^{-\frac{|x|}{\sigma}}}{2\sigma}$$

Does this look like Laplacian distribution to you? Shouldn't we consider then Dirac's delta as a subclass of Laplacian distribution?