10

I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing aspect of the proof is that you do not need distributional assumptions to prove the claim.

I've been studying up on GLMs (and using them) now for several years and you can definitely draw various analogies between GLMs and the linear model.

I've been trying to understand how the GM theorem-or a generalization of it-fits into the GLM theory. It seems like the only place the distribution assumption is used-and even then only sometimes used-is for the link choice. Notably hypothesis testing does not use the distributional assumption instead usually using a Gaussian, or large sample approximation.

My question is if for estimation purposes there is an analogous result to the GM theorem stating BLUE? This may require a generalized definition of linearity but should still be stateable and provable. If anyone is aware of this result a reference would be appreciated.

Lucas Roberts
  • 3,819
  • 16
  • 45
  • 2
    It's worth noting that MLE estimates of GLM parameters are generally biased: https://stats.stackexchange.com/questions/60723/bias-of-maximum-likelihood-estimators-for-logistic-regression – Matthew Drury Sep 19 '17 at 04:59
  • 2
    @Matthew Drury -> For finite samples the GLM estimates are biased but they are asymptotically unbiased. I guess the equivalent statement to BLUE in GLM world would be an asymptotic statement, good point. – Lucas Roberts Sep 19 '17 at 13:14
  • I wonder if this (and the comments) does not answer your question? https://stats.stackexchange.com/questions/346304/is-the-ols-estimator-the-umvue-assuming-normality – Patrick May 16 '18 at 08:55
  • @Patrick, not quite, that post has a Gaussian error assumption. I'm looking for an analogous proof of for a general GLM. The proof may have more conditions (e.g. an asymptotic statement only) but there should be an analogous statement for GLMs too. – Lucas Roberts May 16 '18 at 13:38
  • This is a great question, too bad it never received an answer. – Digio Feb 13 '19 at 08:16
  • @Digio, maybe you can write an answer? :) – Lucas Roberts Feb 13 '19 at 13:33
  • 1
    I wish I knew the answer, I actually got here because I had the same question. I could probably come up with an answer but I'd rather just have _the_ answer. – Digio Feb 14 '19 at 19:13
  • 1
    @Digio, FWIW I think the answer is it doesn't really matter other than some conventions but no mathematical theory. For example, a gamma distribution conventionally uses a log-link but that is not the canonical link. As I indicated in the question, the hypothesis tests do not really rely on distribution b/c they are asymptotic and normal cutoff values. Kind of difficult to prove that this doesn't matter though. – Lucas Roberts Feb 14 '19 at 21:28
  • +1 Interesting question! I would think the answer is in the negative. In GLMs other than linear regression, the idea of a "linear" estimator does not make sense to me. If we drop the linearity condition from the Gauss-Markov theorem, depending on what error distribution there is, the usual OLS solution might not be the BUE (best, possibly nonlinear, unbiased estimator). – Dave Jan 14 '22 at 20:18

0 Answers0