0

I have 6 independent x variables and have used OLS to get a polynomial model to describe their relationship with my dependent y variable.

Here is what I get from statsmodels:

==============================================================================
Dep. Variable:                      p   R-squared:                       0.209
Model:                            OLS   Adj. R-squared:                  0.138
Method:                 Least Squares   F-statistic:                     2.949
Date:                Sat, 09 Oct 2021   Prob (F-statistic):             0.0129
Time:                        15:06:33   Log-Likelihood:                -634.99
No. Observations:                  74   AIC:                             1284.
Df Residuals:                      67   BIC:                             1300.
Df Model:                           6                                         
Covariance Type:            nonrobust                                         
==========================================================================================
                             coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------------------
const                   1.177e+04   1107.772     10.629      0.000    9563.438     1.4e+04
Plasma Temperature (K)     0.0025      0.003      0.843      0.402      -0.003       0.008
Kp*10                     16.1861     27.491      0.589      0.558     -38.687      71.059
Dst                      -14.2056     14.523     -0.978      0.332     -43.195      14.783
Proton Density            83.2930     69.770      1.194      0.237     -55.969     222.555
f10.7                    -15.9405      8.069     -1.975      0.052     -32.047       0.166
Ey                      -197.5733    292.431     -0.676      0.502    -781.268     386.121
==============================================================================
Omnibus:                        6.537   Durbin-Watson:                   2.089
Prob(Omnibus):                  0.038   Jarque-Bera (JB):                6.412
Skew:                           0.485   Prob(JB):                       0.0405
Kurtosis:                       4.067   Cond. No.                     7.71e+05
==============================================================================

As someone with no experience with statistics, I looked up what all of these different parameters meant but am currently trying to determine how to get the most accurate polynomial using only some of the variables. How would I approach this? Is it simply running a case of running all of the combinations until I get the highest Adj. R-squared or lowest P>t?

Thanks!

scikitnoob
  • 1
  • 1
    Hi scikitnoob and welcome! Your question is a very well known one, and goes by the name of "variable selection". It has been discussed on this site multiple times already, Check out this one https://stats.stackexchange.com/questions/122931/variable-selection-for-logistic-regression. It has a recommendation to use lasso as an answer and some more links in the comment to the question. – psarka Oct 09 '21 at 21:03
  • @psarka Thank you so much for sharing the name of the problem!! I'll check it out, much appreciated! – scikitnoob Oct 09 '21 at 21:21
  • Hi @psarka, correct me if I am wrong, but from my understanding Lasso simply removes any coefficients under a certain value (so, near-zero). Am I misunderstanding Lasso? I only ask because when using lasso, I receive the exact same polynomial I already had, but I know for sure that removing at least one of the x variables from the model will improve my model... – scikitnoob Oct 09 '21 at 21:35
  • 1
    Lasso is more subtle than just normal regression + removing coefficients under a certain value. I find it surprising, that you get the same result. My advice would be to double check the code to make sure you don't have any bugs, and if not, post a new question. – psarka Oct 09 '21 at 21:44
  • You state you have fitted a polynomial but I see no sign of such terms. What do you mean by most accurate polynomial? – mdewey Oct 10 '21 at 13:50
  • @mdewey in the "Coeff" column, its the coefficients for those variables + the intercept that model my data – scikitnoob Oct 10 '21 at 16:37
  • @AdrianKeister Logistic regression isn't what I'm trying to do but that link was useful regardless – scikitnoob Oct 10 '21 at 16:38
  • [Search our site](https://stats.stackexchange.com/search?tab=votes&q=feature%20selection) for many threads on this subject. – whuber Oct 10 '21 at 16:49

0 Answers0