4

In linear regression of $y$ onto $x$, one finds a $\beta_0$ and $\beta_1$ minimizing $\sum \|y - (\beta_1 x + \beta_0)\|^2$. One can show that $$\beta_1 = \rho(x,y) \frac{\sigma(y)}{\sigma(x)},$$ where $\rho(x,y)$ is the correlation between $x$ and $y$ and $\sigma(\cdot)$ is the standard deviation function.

A common "criticism" of the least-squares approach above is that it is not resistent to outliers. A simple modification of the above framework is to instead consider the $\beta_0$' and $\beta_1$' which minimize the absolute deviations, $\sum \|y - (\beta_1' x + \beta_0')\|$. This is sometimes called LAD linear regression.

There are L1-analogoues to the covariance and standard deviation. For example, this paper discusses robust estimators of the covariation and variance, the "co-median" and "MAD" estimator. I believe there are others.

Question: One might guess that $\beta_1' = \frac{\text{COMEDIAN}(x,y)}{\text{MAD}(x)}$. Does $\beta_1'$ indeed have a "natural" interpretation of this form, in terms of "natural" L1 quantities, analogous to L2/linear regression coefficient $\beta_1$?

[Edited to stress the search for a "natural" expression].

Apprentice
  • 181
  • 4
  • 1
    The sense of your question about "interpretation" is unclear, because you could always *define* the "co-median" (the word absolutely needs a hyphen!) to be $\hat\beta_1'$ times the MAD. What kind of answer are you looking for? – whuber Nov 16 '17 at 22:16
  • I guess my question is whether there is a "natural" interpretation given "natural" quantities people have already studied. The co-median (I thought Falk's lack of hyphen was rather comedic!) and MAD estimator, for instance, are natural in the sense that they are rather simple robust modifications of the covariance and variance. I would of course be thrilled to see a closed form solution or approximation in terms of MAD and a brand new estimator of the "robust covariance"! – Apprentice Nov 17 '17 at 15:00
  • @whuber Please tell me if you think my question is not sensible or needs to be further revised, in light of my comments. – Apprentice Nov 17 '17 at 15:01
  • Your comments are helpful, and would be even more helpful if included in the text of the question itself. I'm afraid the answer is in the negative, though: the $L^1$ solution cannot be this simple. But it is of some interest to inquire why not and/or to explore the properties of your formula for $\hat\beta_1'$. – whuber Nov 17 '17 at 15:05
  • Thanks for the comment, @whuber. Is your intuition that it isn't possible related to the fact that the L1 optimization is not over a smooth, convex set? – Apprentice Nov 17 '17 at 17:35

0 Answers0