1

I want to know why everytime I run my algorithm (XGBoost regressor) with a different random state (applied to train/test split part) I get different values for R2 and RMSE. For example :

  • Random state = 0 -> RMSE = 4.67, R2 = 0.78
  • Random state = 42 -> RMSE = 8.58, R2 = 0.54
Ashraf.R
  • 55
  • 5

1 Answers1

1

Your code is doing what it should. There is a random component to your machine learning modeling, and that random component is different for the different random states, the same as how the following two code blocks will give different random numbers.

np.random.seed(2021)
np.random.normal(0, 1, 10)

vs

np.random.seed(2022)
np.random.normal(0, 1, 10)
Dave
  • 28,473
  • 4
  • 52
  • 104
  • Watch out for interpreting $R^2$ in your nonlinear model, however. As I discuss [here](https://stats.stackexchange.com/q/551915/247274), there is a sense where $R^2$ is totally legitimate, but $R^2$ in nonlinear models does not have the same interpretation as in vanilla OLS linear regression. – Dave Nov 15 '21 at 16:22
  • I see. Thank you. So how can I fix the results so that they don't change. Is there an alternative to random_state? Is there a better metric than R2 for non linear regression? – Ashraf.R Nov 15 '21 at 16:30
  • If you want the results to be the same every time, use the same random state (or whatever the correct terminology/code is for your particular implementation). // Nonlinear regression metrics would be for a separate question. – Dave Nov 15 '21 at 16:40
  • Okay. Thank you for your help – Ashraf.R Nov 15 '21 at 16:42