16

What was the first derivation of the normal distribution, can you reproduce that derivation and also explain it within its historical context?

I mean, if humanity forgot about the normal distribution, what is the most likely way I would rediscover it and what would be the most likely derivation? I would guess the first derivations must have come as a byproduct from trying to find fast ways to compute basic discrete probability distributions, such as binomials. Is that correct?

  • 2
    It's not very difficult to come up with probability distributions: take any positive integrable function, normalize it, and you thus have a probability density. Now if you want to do likelihood based inference with a family of distributions, you need the logarithm of the density to be a simple convex function. More precisely, if you want the maximum likelihood to minimize a given convex loss function, then the exponential of this loss is an appropriate choice of density. The squared error gives rise to the Normal distribution, and might be the simplest example of a convex loss. – Olivier Jan 25 '18 at 12:08
  • 1
    @Olivier, just because you can invent a probability distribution easily it doesn't mean it's useful or that it shows up everywhere. The discovery of the gaussian distribution is related to solving real problems I would guess, not just normalizing a function. –  Feb 01 '18 at 08:40
  • 2
    There are a number of questions and answers already that relate to this history, that may answer or partly answer your question. – Glen_b Feb 09 '18 at 04:38
  • @Glen_b could you point to them, I didn't find any. –  Feb 09 '18 at 04:53
  • 2
    The section in the Wikipedia on the history https://en.wikipedia.org/wiki/Normal_distribution#History is worth reading. The conclusion I draw is that priority here is, as so often, a matter of international dispute. You can take your pick from De Moivre, Laplace, Gauss, ... – mdewey Feb 09 '18 at 09:32
  • 2
    Have a look a this question here and the answer by @Glen_b https://stats.stackexchange.com/questions/227034/how-did-scientists-figure-out-the-shape-of-the-normal-distribution-probability-d I guess one way of how _you_ could rediscover the normal distribution is by taking measurements and realizing that there is an uncertainty/error associated with your measurement, i.e. if you repeat your measurements over and over the outcome will not be 100% identical. Then you want to quantify the uncertainty/error. And then you need some calculus :) Also the Stahl reference is really worth a read! – Stefan Feb 09 '18 at 13:46
  • 1
    The Gaussian/normal/bell-curve density would almost necessarily come about as a need to quantify an observable phenomenon: the tendency of errors about the mean in experimental science. It would not be, nor was it ever, a purely theoretical mathematical finding. Along those lines, the derivation of the normal distribution as the **maximum entropy distribution** would be the most sensible approach. Maybe, the adept statistician would have the sense to not call it a "normal" RV. – AdamO Feb 09 '18 at 19:44
  • @AdamO this is interesting Adam, maybe you want to expand it and post as an answer? –  Feb 12 '18 at 04:06

6 Answers6

6

Stahl ("The Evolution of the Normal Distribution", Mathematics Magazine, 2006) argues that the first historical traces of the normal came from gambling, approximations to the binomial distributions (for demographics) and error analysis in astronomy.

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
  • 4
    Yes, but in most (all?) of those cases the Normal distribution was not explicit. This sounds a little like concluding Ben Franklin knew (or invented) Maxwell's Equations because he experimented in electricity. – whuber Jan 24 '18 at 14:43
  • Could you provide the derivations these authors did? –  Jan 25 '18 at 02:05
  • For instance, what math did they need to derive it? –  Jan 25 '18 at 02:12
6

I would guess the first derivations must have come as a byproduct from trying to find fast ways to compute basic discrete probability distributions, such as binomials. Is that correct?

Yes.

The normal curve was developed mathematically in 1733 by DeMoivre as an approximation to the binomial distribution. His paper was not discovered until 1924 by Karl Pearson. Laplace used the normal curve in 1783 to describe the distribution of errors. Subsequently, Gauss used the normal curve to analyze astronomical data in 1809.

Source : NORMAL DISTRIBUTION

Other sources with historical context:

Nowadays the fact that the Normal distribution is an approximation for Binomials for large $n$ is considered as a special case of the Central Limit Theorem. It can be found in most text books and is considered as elementary. You can find a proof on Wikipedia. The exponential just shows up as $e^x=\lim(1+\frac{x}{n})^n$ after some Taylor expansion of the characteristic function that yield $-\frac{t^2}{2}$. Sometimes you still find special proofs for Binomials in textbooks and this is known as DeMoivre-Laplace theorem.

Benoit Sanchez
  • 7,377
  • 21
  • 43
  • Benoit, the derivation of DeMoivre does not seem elementary, could you include it in your answer?. This DeMoivre derivation is something I'm looking for ( as a side note, do you know if all the calculus and approximations results - stirlings approximation for instance - were available for DeMoivre already, or is this a modern version of his proof?) –  Feb 12 '18 at 04:08
  • 1
    This a modern version. I don't know DeMoire's historical derivation. The only historical info I've got is the article pointed by both Stephan and me. – Benoit Sanchez Feb 12 '18 at 10:45
3

The question's historical part was answered already, possibly, multiple times on this forum, e.g. see the accepted answer to a similar question. No, it was not discovered as an approximation to discrete distributions. I doubt there was even a notion of probability distribution at the time. It was discovered by guys who's be called physicists or mathematicians these days, I guess nature philosophers at the time.

How would another civilization discover the normal distribution is an interesting question. Anyone who studies errors and disturbances of any kind would have found it. It happened so that our civilization found it while studying celestial bodies. I doubt that it is likely that other humans would develop statistics before physics or mathematics.

Aksakal
  • 55,939
  • 5
  • 90
  • 176
1

What' special about the normal distribution is the Central Limit Theory. For details and derivation/proof see: https://en.wikipedia.org/wiki/Central_limit_theorem

elliotp
  • 670
  • 3
  • 13
1

I also asked myself that question and this youtube video is the best answer I have found

https://www.youtube.com/watch?v=cTyPuZ9-JZ0

I don`t think that it is the original derivation but the description of the video says "This argument is adapted from the work of the astronomer John Herschel in 1850 and the physicist James Clerk Maxwell in 1860. "

Sebastian
  • 2,733
  • 8
  • 24
1

It's hard to parse this question. Who is the "I" in this question? And when is the time in question? An almost trivial answer is finding a location/scale family that is $\propto \exp(-x^2)$. The OP then goes on to ask "If humanity forgot about the normal distribution, in what manner would it be rediscovered"? This is an altogether different question. I think a relevant answer here is one that 1) borrows the perspective of modern science 2) provides an answer that is different from the most frequently encountered historical answer, aka the Central Limit Theorym.

In quantum mechanics, information theory, and thermodynamics, the entropy quantifies the state of a system. In these fields, the quantum state is in fact, wholly random or stochastic. Contrast this with classical mechanics. In classical mechanics, states are fixed but our observation is imperfect due to the contribution of hundreds or millions of unobserved influencing factors: this kind of result gives rise to the CLT.

In quantum mechanics, we use Bayesian probability to quantify our belief about the state of the system. Along those lines, proofs have been presented, and tweaked, that the Gaussian or normal random variable has maximum entropy among all random variables with finite mean or standard deviation.

https://www.dsprelated.com/freebooks/sasp/Maximum_Entropy_Property_Gaussian.html

https://en.wikipedia.org/wiki/Differential_entropy

http://bayes.wustl.edu/etj/articles/brandeis.pdf

AdamO
  • 52,330
  • 5
  • 104
  • 209