Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [least-absolute-deviations]

A regression that minimizes the sum of absolute errors (instead of the sum of squared errors).

37 questions
33
votes
7 answers

Why is using squared error the standard when absolute error is more relevant to most problems?

I recognize that parts of this topic have been discussed on this forum. Some examples: Is minimizing squared error equivalent to minimizing absolute error? Why squared error is more popular than the latter? Why square the difference instead of…
Ryan Volpi
  • 1,638
  • 8
  • 17
12
votes
2 answers

How to solve least absolute deviation by simplex method?

Here is the least absolute deviation problem under concerned: $ \underset{\textbf{w}}{\arg\min} L(w)=\sum_{i=1}^{n}|y_{i}-\textbf{w}^T\textbf{x}|$. I know it can be rearranged as LP problem in following way: $\min \sum_{i=1}^{n}u_{i}$ $u_i \geq…
southdoor
  • 121
  • 1
  • 5
11
votes
2 answers

Residual using absolute loss linear regression

For ordinary least square linear regression, we have sum of residuals as zero, what about the sum of residuals for linear regression calculated using absolute loss?
william007
  • 897
  • 1
  • 6
  • 9
8
votes
1 answer

When does Least Square Regression (LSQ) line equal to Least Absolute Deviation (LAD) line?

I have the following question at hand. Suppose $(x_1,y_1),(x_2,y_2),\cdots,(x_{10},y_{10})$ represents a set of bi-variate observations on $(X,Y)$ such that $x_2=x_3=\cdots =x_{10}\ne x_1.$ Under what conditions will the Least Square Regression…
Qwerty
  • 1,047
  • 10
  • 22
7
votes
0 answers

Is there any geometric intuition on least absolute deviation regression?

There are a lot of geometric intuitions for regression with least square, e.g., projection, orthogonal, etc. (This and this answers are good examples.) Is there similar geometric intuition for least absolute deviation regression?
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
7
votes
1 answer

What is the maximum likelihood/GLM version of least absolute deviations for robust linear regression?

Robust linear regression from minimising the absolute deviationresults in a regression line of medians conditional on covariates, instead of means using the standard least squares methodology: Is minimizing squared error equivalent to minimizing…
Alex
  • 3,728
  • 3
  • 25
  • 46
4
votes
0 answers

Most efficient LAD solver

What is the most efficient way to solve linear Least absolute deviation regression problem? I know it can be solved using linear programming, is there a better/faster method? Edit: I'm interested in the more theoretical aspect, hence by "efficient"…
Meni
  • 183
  • 7
4
votes
0 answers

Interpretation of the least absolute deviations linear regression coefficient

In linear regression of $y$ onto $x$, one finds a $\beta_0$ and $\beta_1$ minimizing $\sum \|y - (\beta_1 x + \beta_0)\|^2$. One can show that $$\beta_1 = \rho(x,y) \frac{\sigma(y)}{\sigma(x)},$$ where $\rho(x,y)$ is the correlation between $x$…
Apprentice
  • 181
  • 4
3
votes
0 answers

Why is there no improvement when training Xgboost with pseudo-Huber loss?

In this StackOverflow post I asked if there was something wrong with my syntax when training an XGboost model (in R) with the native pseudo-Huber loss reg:pseudohubererror, since nor training or test error improve (remain constant). There doesn't…
PaulG
  • 793
  • 2
  • 10
3
votes
1 answer

Reference for doing linear regression with mean absolute deviation?

I am looking for a resource that goes over how to derive the coefficients for a linear regression model while minimizing the mean absolute deviation. I am hoping for both a mathematical and computational solution - if possible. If not, just a…
confused
  • 2,453
  • 6
  • 26
3
votes
2 answers

Is there any library for least absolute deviation (LAD) regression with regularization terms?

We know that LASSO and ridge and ElasticNet all apply regularization terms on the coefficients of least squares regression. However, I have not yet found any R / python libraries that compute regularization of of Least Absolute Deviation (LAD):…
felixc
  • 31
  • 2
3
votes
2 answers

Model that optimizes mean absolute error always gives same prediction

My gradient boosting regression model (GBM) is trained to minimize mean absolute error (MAE) but gives the same prediction for every record on my highly skewed dataset. I believe there is a quick fix to the immediate problem (use RMSE) but my…
Ryan Zotti
  • 5,927
  • 6
  • 29
  • 33
3
votes
1 answer

$R^2$ for least absolute deviation regression

I know that $R^2$ is for the least square regression. Is there an analogous measure of fit to $R^2$ in LAD (Least Absolute Deviations) regression? Here I am concerned with the "fitting quality".
CCKKLLGG
  • 87
  • 8
2
votes
0 answers

Efficiency of OLS versus Quantile regression estimator

If I have a linear model $ y_i = x_i'\boldsymbol\beta + \epsilon_i $ and I assume that OLS estimator of $\boldsymbol\beta$ is unbiased and consistent and Least absolute deviation (LAD) estimator of $\boldsymbol\beta$ is also consistent, what are the…
doremi
  • 127
  • 5
2
votes
0 answers

Comparing the robustness least absolute deviation with OLS

My professor told us that the OLS estimator can be influenced by outliers because $$\hat{\beta}_{OLS}=\text{argmin}\left\lVert y - X^T \beta\right\rVert_2^2 $$ implies that the first order condition $$2X^T(y-X\beta)=\sum_{i=1}^{n}2x_i (y_i…
Stan Shunpike
  • 3,623
  • 2
  • 27
  • 36
1
2 3