0

I understand that we like working with square error instead of absolute error because it makes the calculus easy. But I was wondering about the parameters of Linear Regression minimized for absolute error.

Just to make it a little bit more clear, in the derivation of parameters for linear regression, we minimize

$$ \sum_{i=1}^{n} (Y_i - \hat{Y_i})^2 $$

to arrive at the parameters $$ \beta_0 = \bar{Y} - \beta_1\bar{x} $$ $$ \beta_1 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(Y_i - \bar{Y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2} $$

Now how would the parameters change if I were to minimize $$ \sum_{i=1}^{n} |Y_i - \hat{Y_i}| $$

I cannot find a derivation online.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
  • https://en.wikipedia.org/wiki/Least_absolute_deviations – user2974951 Sep 24 '21 at 07:06
  • @user2974951 Thanks for this. Quick question. I'm not able to specifically find a formula for the parameters derived using this. Is it not possible to mathematically arrive at a specific formula for Beta_0 and Beta_1 using LAD? – Vishal Balaji Sep 24 '21 at 10:43
  • `Unlike least squares regression, least absolute deviations regression does not have an analytical solving method. Therefore, an iterative approach is required`. – user2974951 Sep 24 '21 at 10:47
  • @user2974951 Ah didn't catch that. Thanks! – Vishal Balaji Sep 24 '21 at 10:50
  • A number of answers on site here discuss aspects of this approach (it's called several different names -- e.g. L1 regression, least absolute deviations regression/LAD regression, it's also a special case of quantile regression about which there are a number of relevant answers) so it might take a few searches; there are hundreds of posts related to it in all. Many packages can fit a regression like this. Note, however, that (like a median in univariate analysis), the LAD regression estimate might not be unique. – Glen_b Sep 24 '21 at 21:38
  • See https://stats.stackexchange.com/questions/260504/how-to-solve-least-absolute-deviation-by-simplex-method, https://stats.stackexchange.com/questions/228468/when-does-least-square-regression-lsq-line-equal-to-least-absolute-deviation, https://stats.stackexchange.com/questions/454878/reference-for-doing-linear-regression-with-mean-absolute-deviation – kjetil b halvorsen Sep 25 '21 at 08:44

0 Answers0