I would like to find a reference, preferably free on the internet, where I can read about the theoretical or practical justification for the use of parametric / analytic probability distributions.
By parametric distributions I mean the named ones like Normal, Weibull, etc.
Nice question. I like Ben Bolker's descriptions from his book, Ecological Models and Data in R (preprint of the relevant chapter; the bestiary of distributions starts on page 19).
For each distribution, he has a few sentences to a page on where it comes from and what it's used for, plus some math and graphs.
In some sense there is no such thing as a statistics without "parameters" and "models". It is an arbitrary labelling to some extent, depending on what you recognise as a "model" or "parameter". Parameters and models are basically ways of translating assumptions and knowledge about the real world into a mathematical system. But this is true of any mathematical algorithm. You need to somehow convert your problem from the real world into whatever mathematical framework you intend to use to solve it.
Using a probability distribution which has been assigned according to some principle is one way to do this conversion in a systematic and transparent way. The best principles I know of are the principle of maximum entropy (MaxEnt) and the principle of transformation groups (which I think could be also called the principle of "invariance" or "problem-indifference").
Once assigned you can use Bayesian probability theory to coherently manipulate these "input" probabilities which contain your information and assumptions into "output" probabilities which tell you how much uncertainty is present in the analysis you're interested in.
A few introductions from the Bayes/MaxEnt perspective described above can be found here, here, and here. These are based on the interpretation of probability as an extension of deductive logic. They are more on the theoretical side of things.
As a minor end-note, I recommend these methods mainly because they seem most appealing to me - I can't think of a good theoretical reason for giving up the normative behaviours which lie behind the Bayes/MaxEnt rationale. Of course, you may not be as compelled as I am, and I can think of a few practical compromises around feasibility and software limitations though. "real world" statistics can often be about which ideology you are approximating (approx Bayes vs approx Maximum Likelihood vs approx Design based) or which ideology you understand and are able to explain to your clients.
A Bayesian way to introduce and motivate parametric models is through Exchangeability and De Finetti's Representation Theorem. There is some discussion in this question:
What is so cool about de Finetti's representation theorem?
A great introduction is given in the first chapter of Schervish's Theory of Statistics. All the measure theoretic language needed for the discussion is given in his tour de force Appendix (with complete proofs!). I've learned much from this book, and I strongly recommend you to buy it.
This paper studies the generality of the Bayesian construction:
Sandra Fortini, Lucia Ladelli and Eugenio Regazzini
Sankhyā: The Indian Journal of Statistics, Series A (1961-2002)
Vol. 62, No. 1 (Feb., 2000), pp. 86-109
It's available for download here: http://sankhya.isical.ac.in/search/62a1/62a17092.pdf
