1

Is there any way to estimate the bias of the estimate of the betas in a linear regression model when the actual beta values are unknown?

The well known Mean Square Error (MSE) criterion is used to quantify the performance of different biased estimators but to calculate the bias you need to know what the correct value of the Betas should be.

$MSE= var^2 +bias^2$

What can you do when you don't know what the correct value is?

I wonder if you can use the OLS value as a proxy (as it is unbiased)? However the very situations in which biased estimators are likely to be used are those in which OLS is likely to struggle, just wondered if there were any other methods to estimate the bias?

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
Baz
  • 1,583
  • 3
  • 13
  • 26
  • 1
    One place where ridge regression is used is when there is colinearity. In this case, the OLS parameter estimates are not biased, but their variances are off. – Peter Flom Jan 28 '14 at 10:40
  • 1
    I think you've answered your own question @Baz. If we could do what you suggest, then why would we run the regression at all? – pkofod Jan 28 '14 at 10:50
  • @Peter are the OLS estimates unbiased for all levels of multicollinearity up to and including perfect multicollinearity – Baz Jan 28 '14 at 12:11
  • If that is the case then it seems we can use the OLS estimate to estimate the bias? – Baz Jan 28 '14 at 12:12
  • @Baz I am not sure about perfect multicolinearity, but that is (in my experience) always an error that is easy to fix (e.g. you include height in inches and height in cm as IVs) and I don't believe ridge regression can cope with it either. – Peter Flom Jan 28 '14 at 12:14
  • Thanks again Peter, OK except in case of perfect MC I can use OLS to estimate bias? – Baz Jan 28 '14 at 12:37
  • So what you have is a setting where you know the OLS-estimator is unbiased, but you then employ some other estimator which you don't know if it is unbiased and you want to assess that, have I understood you correctly? – ekvall Jan 28 '14 at 14:38
  • I know that the estimator I am using is biased but I want to be to estimate it's MSE. I can estimate the variance quite easily by generating different realisations of the errors and recalculating the beta parameters but to calculate the MSE I must find someway to estimate the bias without knowing what the true value is. – Baz Jan 28 '14 at 14:45
  • 1
    This answer may help http://stats.stackexchange.com/questions/81806/omitted-variable-bias-verification-in-gretl/81813#81813 – Alecos Papadopoulos Jan 28 '14 at 19:33

0 Answers0