1

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression:

$$\hat\beta = \underset{\beta_0, \beta}{\min}({\frac{1}{2N}||y - \beta_01_N-X\beta||^2_2 + \lambda||\beta||^2_2})$$

I have been working on this problem for over 4 hours, and have absolutely no idea how to prove it. The worst part is even when I look at the solution answer, I have no idea what does it mean at all. Really need help on this problem.

solution answer page 1

solution answer page 2

FantasticAI
  • 417
  • 1
  • 4
  • 12
  • From my understanding that in the end of the proof, the two ridge regression has to be equal since $X\hat\beta = X\tilde{\beta}$, so that is why we have contradiction? – FantasticAI Mar 06 '20 at 22:50
  • There is a contradiction, because b^ is assumed to be minimiser, but b~ is smaller. – seanv507 Mar 06 '20 at 23:46
  • 1
    Consider a different line of attack, such as the characterizations of the solution at https://stats.stackexchange.com/a/164546/919 or https://stats.stackexchange.com/questions/220243. – whuber Mar 07 '20 at 14:01

0 Answers0