2

Is taking the average of different rmse valid? for example average rmse = (rmse1+rmse2+rmse3)/3 Thank you for your help!

angelo
  • 21
  • 1
  • 2
  • 4
    Valid for what exactly? Sure, you can average them [you can multiply them, too](http://stats.stackexchange.com/a/432/7290). – gung - Reinstate Monica May 19 '14 at 14:13
  • 1
    if for example I want to know which is the best among different models and I get the rmse for three different input to each model. Therefore I have 3 rmse for each. Is it correct to just average the three rmse's in able to select the best model? – angelo May 19 '14 at 14:44
  • Its answer lies in reading sampling theory. –  May 19 '14 at 16:00

1 Answers1

4

I actually wasn't sure about this either so I tested it out with a short example:

## Create simple function to calcualte the error
rmse <- function(error){sqrt(mean(error^2))}

## Define two example error vectors
error1 <- c(0.4, 0.2, 0.01)
error2 <- c(0.1, 0.3, 0.79)

## Find the RMSE of each error vector
rmse1 <- rmse(error1)
rmse2 <- rmse(error2)

## Compare the RMSE variants
print(rmse_all <- rmse(c(error1, error2)))
[1] 0.3924708
print(rmse_avg <- mean(rmse1, rmse2))
[1] 0.2582634

So we can se that they are not equal.

## As described by @whuber in the comments:

a <- rmse1^2*length(rmse1)                 # - square each rmse & multiply
b <- rmse2^2*length(rmse2)                 #   it by its associated count
c <- sum(a, b)                             # - sum that stuff up,                  
d <- c/sum(length(rmse1), length(rmse2))   # - divide by the total count,          
print(total_rmse <- sqrt(d))               # - take the square root.
[1] 0.3924708
n1k31t4
  • 541
  • 2
  • 5
  • 17
  • 3
    The answer lies in front of you. Since you have defined rmse as the root of the mean of certain values, then just square each rmse, multiply it by its associated count, sum that stuff up, divide by the total count, and take the square root. The "slight bastardization" can be dreadfully wrong. – whuber Jan 14 '16 at 21:51
  • @whuber - I have edited to show what you describe. Please feel free edit my post with comments to add clarity. – n1k31t4 Jan 14 '16 at 22:41