2

Frequentist statistics textbooks typically consider point and interval estimation but not density estimation of a parameter. Since the density (of the sampling distribution) of the estimator is involved in deriving the interval estimator, why not consider the density itself? Is there something wrong with or uninteresting about it so that it is not considered?

Edit: a/the relevant term seems to be confidence distribution (thanks to @KjetilHalvorsen) as in

I wonder why confidence distributions are not covered right after interval estimation in typical textbooks.

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
  • It is considered even at the most elementary level. You would be hard pressed to find any intro textbook that does not explain, for instance, how to estimate the density of a distribution that is assumed to be Normal. – whuber Jan 10 '20 at 19:32
  • @whuber, I am talking about the density of a parameter estimator, not of raw data. I noticed that most textbooks start with point and end with interval estimation of parameters, and the density does not get analyzed. – Richard Hardy Jan 10 '20 at 19:41
  • Then I am even more baffled, because consideration of an estimator's distribution is fundamental to all probability-based statistical techniques. I can't imagine any book on the subject that doesn't cover this. Going back to the previous example: even elementary textbooks explain how the usual estimator of the mean has a Normal distribution (and how to estimate its parameters) and the usual estimator of the variance has a distribution proportional to a chi-squared distribution (and how to estimate its parameters). – whuber Jan 10 '20 at 19:44
  • @whuber, this is true. However, this is still not what I am getting at. I am looking at the progression: point estimation, interval estimation, density estimation, all targeting the same parameter. I do see posterior densities of parameters in Bayesian statistics but not the "corresponding" densities in frequentist statistics (while point estimattion and interval estimation have such "correspondences" between frequentist and Bayesian approaches). – Richard Hardy Jan 10 '20 at 19:51
  • What use would you make of it other than deriving point estimates (e.g., MLE) and finding confidence intervals / performing hypothesis tests? – jbowman Jan 10 '20 at 19:57
  • 1
    @jbowman, not many other uses than the ones you say. Nevertheless, I think it would be convenient to have a density reported so that any user could take his/her favourite loss function and calculate the appropriate point estimate or choose a favourite confidence level to obtain an appropriate confidence interval. The density is so much more informative than a single point or a single interval. So it is kind of surprising to see it missing from the textbooks. Perhaps this is a matter of time; in the forecasting literature, density prediction is much more common now than it was some years ago. – Richard Hardy Jan 10 '20 at 20:03
  • @CagdasOzgenc, could be. I have not considered that. In the context of models that get analyzed in basic textbooks – and these are often linear models – where point and interval estimation is described, this should not be a problem. – Richard Hardy Jan 10 '20 at 20:29
  • Point estimation of parameters is tantamount to estimating the entire density in any parametric family. Knowledge of the parameters translates to knowledge of sampling distributions of estimators. Your reference to posterior distributions is to a purely Bayesian construct and so is irrelevant. – whuber Jan 10 '20 at 21:57
  • Seems you should look into the concept of *confidence distribution*, see for instance https://stats.stackexchange.com/questions/131130/how-to-compute-confidence-interval-from-a-confidence-distribution (maybe a dup?) and the book https://www.amazon.com/Confidence-Likelihood-Probability-Distributions-Probabilistic/dp/0521861608/ref=sr_1_1?keywords=schweder+hjort&qid=1578695888&sr=8-1, also https://stats.stackexchange.com/questions/265031/changing-the-size-of-a-confidence-interval-in-order-to-emphasize-results/266656#266656 – kjetil b halvorsen Jan 10 '20 at 22:39
  • Also note that sampling distribution of parameters may be different than test statistic. For example linear regression parameters have normal sampling distribution. But test statistic is t because we don’t have error variance at hand. Which one are you after? – Cagdas Ozgenc Jan 11 '20 at 08:10
  • @whuber, it seems I am not getting my message across, sorry. The good news is, confidence distribution might be *the* thing I am talking about! – Richard Hardy Jan 11 '20 at 08:23
  • @kjetilbhalvorsen, thank you very much! It might just be the thing I am looking for. In addition, perhaps the term *fiducial* could also be relevant somewhere in the discussion. – Richard Hardy Jan 11 '20 at 08:23
  • @CagdasOzgenc, good point, I should think more about it. For now I guess my target is the *confidence distribution*. – Richard Hardy Jan 11 '20 at 08:35
  • I think that your point would still be valid but much more accessible and understandable for people if you instead of parameter would have said prediction. – hejseb Jan 11 '20 at 12:49
  • @hejseb, maybe. My impression is that density predictions have become reasonably popular already (though still rare comparing to point and interval predictions) and it is a matter of time until they will be covered in, say, forecasting textbooks (perhaps they already are in some). Meanwhile, I do not see the same development with parameter estimation, so I was wondering why. Perhaps it is because fiducial statistics was problematic from the start and the majority of statisticians lost interest in it. – Richard Hardy Jan 11 '20 at 13:35

0 Answers0