Explain Root Mean Square Error to non-technical audience

Question

My company is in the process of switching equipment from one vendor to another. We measured several metrics from the existing and new equipment and compared the time series. The ideal is to have no difference between the old and new equipment. For that, we use Root Mean Square Error (RMSE) to measure the difference.

We are to present the findings to upper management and employees at the plant, many of whom have no background in statistics. How should I explain the concept of RMSE to those audiences? If series X and Y have an RMSE of 5, how should I explain that 5 in layman's terms?

Answer 1

Starting with the most general, and then with increasing details:

• RMSE is a measure of error (or in your case, difference between two equipments.)
• The unit of RMSE is the same as the unit of the metric used. More details in The unit of Root Mean Square Error (RMSE)
• In estimation problems, RMSE corresponds to the standard error of an estimator.
• You didn't give details of your comparison procedure, but the explanation of RMSE should be tied to the details of the comparison. So in the following points some possibilities.
• Say you have measurements with one equipment, when you compare to a gold standard, say specimenes with very precise measurements taken as the truth (and the lab didn't know the thruth.) Then the RMSE is $\sqrt{ \frac1{n}\sum_{i=1}^n (x_i-\text{gold}_i)^2 }$ so is the standard deviation of the measurements errors. So, if the measurement errors $x_i-\text{gold}_i$ is normalishly distributed, around 95% of the measurement errors will be in $\pm 2\cdot\text{RMSE}$. You should also show some plots with your actual data. Say the equipment measures some length in mm, then your example value of 5 means a typical error of around 5 mm. (With a gold standard, your measurements can also be used to see if the measurements are typically to low or too high, or the bias. The RMSE does not measure bias alone, but bias is a component of it. See Bias-Variance decomposition derivation.)
• Without a gold standard, say you have two series of measurements, using the two equipments on the same specimenes. Then the RMSE is $ \sqrt{ \frac1{n}\sum_{i=1}^n (x_i-y_i)^2 } $ is a measure of the typical difference between the measurement results. It should be given together with a plot of the differences. Again, the squaring forgets about the sign of the difference, which could be important too.

Searching the web for information, in this case can be difficult, as RMSE also is an important concept in electricity, and most of the hits will reflect that! Some other posts here that can be useful is Forecast accuracy calculation and What is the difference between Mean Squared Deviation and Variance?, see also Wikipedia.