1

Let $X$ be an $D \times N$ matrix. Let $I$ be a $D \times D$ identity matrix. Also let $y$ be a $N \times 1$ column vector. Suppose we are trying to solve $(X X ^T + k I) w = Xy$ for a $D$ dimensional column vector $w$. Assuming that $X X ^T + kI$ is invertible, the solution is given by $w_1 = (XX ^ T + k I w) ^ {-1} Xy$.

Now suppose we want to find a solution for the equation $(X X ^T + (k + 1) I) w = Xy$. Assuming that $X X ^T + (k + 1)I$ is invertible, the solution is given by $w_2 = (XX ^ T + (k + 1) I w) ^ {-1} Xy$. Is it possible to express $w_2$ recursively using $w_1$? I tried using Woodbury Matrix Identity, but I couldn't continue much.

J Doe
  • 11
  • 1
  • If you solve the original system with an SVD, this becomes trivially easy. See the last section of https://stats.stackexchange.com/questions/220243/the-proof-of-shrinking-coefficients-using-ridge-regression-through-spectral-dec/220324#220324. – whuber Sep 18 '18 at 16:19

0 Answers0