0

I recently read: Is there any theoretical problem with averaging regression coefficients to build a model? and was intrigued as it brings a basic machine learning concept to good old fashioned OLS. The consensus from the accepted answer was that as OLS estimates are BLUE (best linear unbiased), the advantages one would stand to gain by averaging are few and would likely even underperform the normal OLS estimates. Also of note were the influence of outliers would be slightly less severe with the averaged approach. For this question, assume no outliers.

After thinking about this, I remained curious about the case for non-linear models. I originally intended to tag this as a comment, but I think it's too lengthy and the community would be better served if this was its own question. I wasn't sure if I should go with the average or median as there might be some evidence to suggest taking the median of OLS coefficients is warranted. So I will leave average and median in scope.

Question: Assuming no misspecification/functional form error, would averaging or taking the median of non-linear regression coefficients be theoretically undesirable? One thing I'm interested in is: unlike the OLS case, we are now optimizing with methods like maximum liklihood. Some particular regression models I had in mind are:

  • ARMA/ARCH/GARCH
  • LOGIT/PROBIT
  • FAVAR

Answers don't need to explain for all the models above per se; they are just for reference.

Arash Howaida
  • 741
  • 7
  • 19
  • 2
    Please give thought to what "poorly" might mean. Suppose, for instance, you have *any* statistical estimation problem and you have selected a *good* procedure: that is, according to some criterion it is "best" within a given class of procedures. If you now replace it by some other procedure in the same class that is carried out by averaging estimates found among subsets of the data, then *a fortiori* it isn't as good as the one you originally selected. Is that "poor" or not? If it is poor, then your question seems trivial; but if not, what does "poor" mean? – whuber Jul 03 '18 at 21:00
  • @whuber sorry for the confusion, will add. I meant all else equal and assuming no specification / functional form error. – Arash Howaida Jul 04 '18 at 02:28
  • That's not the issue, though: indeed, everything I wrote implicitly supposed that whatever model you might have adopted to select and justify the original procedure is the correct one for the data. – whuber Jul 04 '18 at 10:22
  • 1
    @whuber After thinking about it for a few days, perhaps you are right. Maybe "poor" is not the best choice of words. I have rephrased the wording from "poor" to "theoretically undesirable" – Arash Howaida Jul 08 '18 at 17:09

0 Answers0