I was experimenting with weighted ridge regression for a linear system, where the closed-form solution is given by: $$ b =(X^T WX + \lambda I)^{-1}X^T W y $$
and also weighted least squares whose closed-form solution is given by $$ b =(X^T WX)^{-1}X^T W y $$
The results in both cases are different with way better results from weighted least squares. But when I solve weighted ridge regression iteratively as shown below I get the same result as weighted least squares. $$ b^{(m+1)} = b^{(m)} + (X^T WX+ \lambda I)^{-1}X^T W z_{(m)} $$ $m$ is the iteration number, $z_{(m)}$ is the residual at every iteration. I am not sure why the result is the same when it is solved iteratively? Can someone please guide me in the right direction or explain this behavior?
