0

I am currently working a lot with R and Python. I am not able to access the C code the the R function lm_fit. I am wondering how is the linear regression optimize in R and python ?

I am pretty sure in R it is optimize with the Normal equation (since R is good with matrix multiplication) and I would think it is optimize with Gradient descent in Python ?

Am I right ?

Nico Coallier
  • 301
  • 3
  • 11

2 Answers2

2

By default lm uses QR decomposition, but there are other options available -- see the documentation. Using the normal equations is a pretty bad way to solve this problem because of problems related to numerical precision; this point is covered extensively on Stats.SE. See this thread for an example.

There are several OLS and GLM implementations in Python so there's no single answer to that question.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
1

R uses QR decomposition. My answer here has some useful links on how to solve a least square problem.

Also, check this awesome answer for the details of lm. Least Squares Regression Step-By-Step Linear Algebra Computation.

Finally, I do not think you can find the C code, since it uses basic linear algebra package such as BLAS / LAPACK, many of them are compiled and the source is in FORTRAN. Check slide 4 in this link.

Haitao Du
  • 32,885
  • 17
  • 118
  • 213