My input data
x =
[[ 1.]
[ 2.]
[ 3.]
[ 4.]
[ 5.]
[ 6.]
[ 7.]
[ 8.]
[ 9.]
[10.]]
y =
[[ 2.45151081]
[ 3.85549532]
[ 6.98435572]
[ 7.11091693]
[ 5.91860089]
[ 7.33033817]
[ 8.77851672]
[ 9.77186197]
[ 9.87780907]
[11.52364423]]
... is actually approximating the function y = -1/10 * X**2 + 2*x
Using:
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
I am trying to understand how SKLearn takes the following to approximate the function above...
polynomial_features = PolynomialFeatures(degree=odr_polynomial)
x_poly = polynomial_features.fit_transform(x)
model = LinearRegression()
model.fit(x_poly, y)
if I print(x_poly) I get
[[ 1. 1. 1. 1.]
[ 1. 2. 4. 8.]
[ 1. 3. 9. 27.]
[ 1. 4. 16. 64.]
[ 1. 5. 25. 125.]
[ 1. 6. 36. 216.]
[ 1. 7. 49. 343.]
[ 1. 8. 64. 512.]
[ 1. 9. 81. 729.]
[ 1. 10. 100. 1000.]]
it looks like this function has iterated x and generated
i**0 , i**1 , i**2 , i**3 (based on me asking for 3rd order polynomial)
I don't know the proper name for this but I'd describe it as "the base bits of the polynomial without the coefficients". I'm an amature coder. To the best of my ability to read the code of sklearn.linear_model.LinearRegression it uses scipy.linalg.lstsq under the hood and that function calls this the "design matrix"
Reading scipy.linalg.lstsq code is where I get lost. I understand how LEast Squares works but I can't understand how least squares is being applied to the 'design matrix" to derive the coefficients.
If I run the code with odr_polynomial = 3 I get:
coefficients: [[ 0. 3.05733751 -0.44144351 0.02535316]]
y-intercept: [-0.12880603]
If I run the code with odr_polynomial = 2 (because the underlying function we're trying to approximate is 2nd order) I get:
coefficients: [[ 0. 1.12796227 -0.02311642]]
y-intercept: [2.04649483]
Although I have asked this question in the context of using SKLearn and Scipy, I think this is also something that could be done with paper & pen. I really want to understand how the coefficients are being derived from the "design matrix".
I'd be grateful for any advice or insight you can share.
