1

I have just started learning Machine Learning and one of the very first topics that I have encountered in this venture is Simple Linear Regression. From Andrew Ng's course, I have learned to perform simple linear regression using gradient descent using the loss function. But recently I was watching this YouTube video where the professor gives an equation to find the slope and the interception in 1 step.

Say if my equation is this $y=a_1 x + a_0$

Then, we can calculate $a_1$ and $a_0$ by these equations

$a_1$=$\frac{\sum_{i=1}^{n}x_i y_i  – \sum_{i=1}^{n}x_i\sum_{i=1}^{n}y_i}{n\sum_{i=1}^{n}{x_i}^2 -(\sum_{i=1}^{n}x_i)^2 }$

$a_0$=$\bar{y}-a_1 \bar{x}$

wherer $\bar{y}=$ $mean$ $of$ $y$ and $\bar{x}=$ $mean$ $of$ $x$

Question 1: Since this method involves no iteration that is needed to compute slope and intercept in the case of Gradient Descent, isn't it better than gradient descent?

Question 2: And what is the name of this method?

Turing101
  • 421
  • 2
  • 8
  • 4
    Answer to 2: *Ordinary Least Squares,* aka OLS. Nobody uses gradient descent for OLS: the point is to illustrate and teach gradient descent with in this extremely well-understood setting. – whuber Aug 12 '20 at 13:07
  • In multiple linear regression Andrew Ng has taught a method known as Normal Equation for solving multiple linear regression. Cant we OLS for Multiple linear regression as well and have a direct formula to derive the values ? – Turing101 Aug 12 '20 at 13:10
  • 3
    As you might imagine, this is a well-discussed topic: see the top-voted posts on regression and multiple regression here. – whuber Aug 12 '20 at 13:13
  • https://stats.stackexchange.com/questions/343069/why-not-use-the-normal-equations-to-find-simple-least-squares-coefficients I have seen this link and it is saying that OLS itself is Normal Equation, I am really confused here – Turing101 Aug 12 '20 at 13:24
  • 2
    See https://stats.stackexchange.com/search?q=regression+normal+equations. For an account to bridge your algebraic notation to the matrix notation needed for multiple regression, see https://stats.stackexchange.com/questions/352130. If that's not perfectly clear, see the explicit example for two regressors at https://stats.stackexchange.com/questions/196807. – whuber Aug 12 '20 at 13:53

0 Answers0