1

I understand why ridge regression (equivalent to using a Gaussian prior on the coefficients in a Bayesian setting) works well in the presence of multi-collinearity, but couldn't one argue that other shrinking priors would also help? (e.g. say a Laplace prior, as in LASSO).

Moreover, if OLS is effectively equivalent to using a uniform prior (with a Gaussian likelihood), what type of priors can arguably with multi-collinearity? In other words, what conditions does a prior have to have to help with multi-collinearity?

Josh
  • 3,408
  • 4
  • 22
  • 46
  • 2
    In terms of shrinkage and multicollinearity, elastic net regression gives you the benefits of LASSO and ridge regression at the cost of an additional hyperparameter. – Sycorax Apr 24 '17 at 22:21
  • Thanks @Sycorax I understand one can obviously combine both with Elastic Net. I am particularly interested in the relationship these priors have with multi-collinearity (I have updated the OP to clarify this point). – Josh Apr 25 '17 at 12:12
  • 1
    If I understand correctly, it seems like any unimodal prior may work. All else equal, the selected fit would be the one which is most probable under the prior. – user795305 Apr 30 '17 at 17:23

0 Answers0