1

Intuitively lasso leads more sparsity, but is that theoretically possible they have exact same solution vector?

display name
  • 11
  • 1

1 Answers1

4

Yes, they reduce to unregularized linear regression if the regularization coefficients are both 0. Or if all the points $(x_i, y_i)$ in the data have the same $y$ value, then of course both will find $\hat \beta = \vec 0$ as the optimal solution.

shimao
  • 22,706
  • 2
  • 42
  • 81
  • Thank you very much sir, but is there a probabilistic approach like lasso leads sparsity more than ridge with x probabilty. What i mean is by isosurfaces it is obvious that lasso hits corners more likely, but by using isosurface volumes for lp norms can we make a probabilistic analysis? – display name May 10 '18 at 21:37
  • @displayname - Ridge regression never results in sparsity, if by sparsity you mean "some coefficient estimates = 0". It shrinks all the parameter estimates towards 0, but none of them all the way. – jbowman May 11 '18 at 04:02
  • @jbowman Is it never or results with very low probability? Is there a mathematical way to show that ridge never results in sparsity? – display name May 12 '18 at 07:30
  • See https://stats.stackexchange.com/questions/74542/why-does-the-lasso-provide-variable-selection/74569#74569. – jbowman May 12 '18 at 13:33