Let me start, that i know that it's not very difficult to generate a probability distribution. If one takes any positive integrable function and normalizes it, this results in a probability density. Furthermore, i am familiar with the usage of certain distributions, so an answer should not direct to "use this distribution for that problem".
This question to the history of the normal distribution states, that
the first derivations must have come as a byproduct from trying to find fast ways to compute basic discrete probability distributions, such as binomials.
But how did things go on and lead us to the "discovery" of distributions like Cauchy, Gumbel, Weibull, ...
This wikipedia article provides an overview on the relationship among probability distributions. I wonder, how all of them were discovered and in fact got their legitimacy for applying in statistical practice!
Were there some "real world" problems leading to the discovery of well-known statistical distributions? Were they developed as special cases of other distributions and later become useful on certain problems? I know, sounds like a chicken-egg-problem, but how was the typical process in discovering new distributions in average?
I appreciate any hint on problems or applications, leading to the discovery of a certain distribution. Also references on the history of their development would be very useful.
The question:
How were statistical distributions discovered?
Edit:
In particular, i am interested in the typical process leading to the discovery of statistical distributions. Rather than going through the history of a certain distribution, i would like to understand the general mechanisms who resulted in "new" distributions.