4

I'm playing around with different ways to solve least squares, and am using numpy to derive values for $\beta$ in a regression problem.

I know that if you do a $QR$ factorization of $X$ such that $ X = QR $ where Q is an $m x n$ orthonormal matrix and $R$ is an $n x n$ upper triangular matrix, then you can derive $\beta$ by:

$\beta$ = $R^{-1}Q^{T}y$.

In numpy this looks like this:

beta = np.linalg.inv(R).dot(Q.T.dot(y))

However, my understanding is that, from an optimization standpoint, it's always a bad idea to take the inverse of a matrix.

So, if one wanted to do a QR factorization to derive the correct values of $\beta$, then how would one do this without taking the inverse of $R$?

Jonathan Bechtel
  • 287
  • 1
  • 8

1 Answers1

5

First, observe that $R \beta = Q^\top y$ involves a triangular matrix $R$, which is easy to solve for $\beta$ without forming an explicit inverse.

In python, we can solve this using the specialized triangular system solver:

beta = scipy.linalg.solve_triangular(R, Q.T.dot(y))
Sycorax
  • 76,417
  • 20
  • 189
  • 313