16

I'm studying machine learning and every book I open I bump into chi-squared distribution, gamma-function, t-distribution, Gaussian, etc.

Every book I have opened so far only defines what the distributions are: they don't explain or give the intuition on where the specific formulas for the functions come from.

For example, why is chi-squared distribution the way it is? What is the t-distribution? What is the intuition behind the distribution? Proofs? etc.

I would like to have a clear and fundamental understanding of the most commonly used distributions so that every time later on when I see them, I truly understand what is a t-distribution, what is a Gaussian distribution and most importantly why are they the way they are.

It would be nice if the books / tutorials can explain the concepts to a layman so that in order to understand them you don't already need to understand them x) Many books are like this, they don't fit for beginners :(

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
jjepsuomi
  • 5,207
  • 11
  • 34
  • 47
  • 1
    Most undergraduate texts on theoretical statistics or probability theory have a chapter on distribution theory that covers these questions. But how much mathematical background would you want to assume? – Scortchi - Reinstate Monica Jul 02 '13 at 10:05
  • undergraduate mathematical background :) The fundamental building blocks. Is that sufficient? What kind of level of mathematics should I acquire before learning about the distributions? I have read a basic book about statistics, which only shortly presented the distributions I described in the question. – jjepsuomi Jul 02 '13 at 10:07
  • Some probability theory & calculus ought to do it - it depends how deep you want to go. – Scortchi - Reinstate Monica Jul 02 '13 at 11:35
  • Okay, thank you :) Mostly I'd just want to understand what I'm doing – jjepsuomi Jul 02 '13 at 11:48
  • 1
    You also might find useful the references posted in this thread: http://stats.stackexchange.com/questions/56385/good-resources-online-or-book-on-the-mathematical-foundations-of-statistics/56388#56388. – Andre Silva Jul 02 '13 at 13:02

3 Answers3

10

If you've no mathematical impediments there's a good overview in Ch. 3 of Casella & Berger, Statistical Inference, & much is covered in Grinstead & Snell, Introduction to Probability (it's free); for more detail I'd recommend Severini, Elements of Distribution Theory. But there are lots - it would be more difficult, I think, to find a less mathematical treatment that still gives the reader some feel for where different distributions come from.

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
  • why it doesn't have "probability mass function" ? – Woeitg Oct 03 '16 at 21:36
  • 1
    Perhaps the OP is looking for a book like this https://www.amazon.com/Handbook-Statistical-Distributions-Applications-Statistics/dp/1584886358... the coverage seems to be great but I cannot speak to its quality ... or this one https://www.amazon.com/Statistical-Distributions-Applications-Parameter-Estimates/dp/3319651110/ref=pd_rhf_dp_p_img_5?_encoding=UTF8&psc=1&refRID=V5X7VKEADFT2VNW82R15 – ColorStatistics Mar 22 '20 at 23:15
  • It is puzzling why this answer was marked as answer of choice when it doesn't answer the question. Casella & Berger has only 50 pages on it and doesn't cover many of the common distributions. Grinstead & Snell spend only 40 pages on it, the usual treatment and emphasis in most statistics books out there. Finally, Severini's book doesn't catalogue and explain the distributions any more than any other mathematical statistics book.... Larsen and Marx's "An Introduction to Mathematical Statistics and Its Appliations", also dedicates 50 pages and its less mathematical than Cassella & Berger – ColorStatistics Mar 22 '20 at 23:31
  • @ColorStatistics: I suppose I was taking the question to ask as much for an introduction to distribution theory as for a compendium of brand-name distribution families. Your suggestions certainly seem worth putting into an answer. For book recommendations in particular, there often doesn't seem much good reason for OPs to accept one answer rather than another; but note that they can change their mind to accept a more useful one that appears later. – Scortchi - Reinstate Monica Mar 24 '20 at 10:14
  • @Scortchi: I've actually come around and think your answer is on point. There are other books that cover the topic more comprehensively but, I've come to conclude, not more intuitively than the sources you've mentioned. I just up-voted your answer myself; and your point that the books you mention cover distribution theory as well as give a compendium of distribution families, is another good one. – ColorStatistics Mar 24 '20 at 15:46
  • @ColorStatistics: Thank you. I'd still suggest you write an answer on the Krishnamoorthy & Thomopoulus books. – Scortchi - Reinstate Monica Mar 24 '20 at 23:13
5

You should read "Continuous univariate distributions" Vol. 1 & 2. by Johnson and Kotz. Also "The Weibull distribution A Handbook" by Horst Rinne. Second one is a useful book to understand a distribution although this book focus on Weibull distribution. May be some material is not easy to under stand but early chapters give you some useful knowledge.

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
SAAN
  • 531
  • 5
  • 16
2

For a short and easy overview of a lot of probability distributions, I recommend Probability and statistics EBook. Most distributions are described in chapter XV, but the more common ones are spread out in earlier parts of the book.

Jonatan Kallus
  • 121
  • 1