6

I have crossvalidated my models and measured RMSE between the modelled values and reality:

RMSE <- function(err) sqrt(mean(err^2))
RMSE(predicted - reality)

I am going to choose model with lowest RMSE. But I would like to know what difference in RMSE is still significant. I guess I could do some F-test comparisons, but as I have tens of models I don't like the idea of pairwise comparisons. This leads me to question

How can my RMSE function be extended to compute SE of RMSE?

How to do this in R? I guess this will be somehow based on Chi-square distribution parametrized by length(err) and maybe somehow scaled by sd(err) (?), but I don't know how to make this statistically correctly in R.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Tomas
  • 5,735
  • 11
  • 52
  • 93
  • 2
    Someone posted [the same question](http://stats.stackexchange.com/q/78079/5509) later. – Tomas Dec 03 '13 at 10:50

0 Answers0