1

I want to assess the repeatability of a measurement device. Let's say I have made three measurement with the same device under the same conditions. What statistic is useful for assessing the repeatability for such a case. An example of the measurement is shown in the plot below.

Example of measurments

H321
  • 11
  • 1
  • Can you clarify: what's x? – Weiwen Ng Apr 12 '18 at 15:24
  • I didn't want to give the units of the measurement variables, because of the difficulty in interpreting the measurement, but is it really necessary for you to know the units? – H321 Apr 12 '18 at 15:30
  • Just want to make sure I am answering correctly. Normally, I would assume that y1, y2, and y3 correspond to the 3 measures on your dependent variable. And no, those units don't matter. But, is x the identity of the observation/person/whatever, is it some other variable you measured, is it time, etc? – Weiwen Ng Apr 12 '18 at 15:36
  • It seems like a good metric could be the integrated squared distance. But it's hard to see how each output covaries as a function of the input. "Repeatability" doesn't make sense here. Are you saying the curve must predict *every* point with *100%* accuracy? Surely they're not all independent. I don't think there's enough information to answer the question. Some more info on context, tolerance bounds, and applications would help. – AdamO Apr 12 '18 at 15:47
  • I am sorry, more context is indeed useful. The measurement device is a medical instrument and it measures two quantities, so y is one quantity and x is another. Time is also measured so I could also create time series (x vs time and y vs time). I hope this makes it a little clearer. – H321 Apr 12 '18 at 16:21
  • Not only should we know what x is, we should know what you mean by "repeatability". – Peter Flom Apr 13 '18 at 12:36
  • Peter Flom is correct. I **think** the OP is after something like test-retest reliability, as described in my answer. However, my answer was also written before I knew that x and y were two different quantities he was interested in measuring. So, OP: what are y1, y2, and y3? Did I answer the question you were intending to ask, or are you interested in something else? If the latter, can you clarify? Are you interested in just how consistently the machine measures x and y separately, are you interested in something about the relationship of x and y, or what? – Weiwen Ng Apr 13 '18 at 14:30

1 Answers1

0

It's not 100% clear what you mean by repeatability. But, in medicine, health services research, and psychology, we often want to know the correlation of repeated measures on the same person taken within a short time frame. For example, say I administered a depression symptom score to a group of people, and then I administered the same questionnaire two weeks later. I am expecting the correlation between the two scores to be high, like over 0.6 (which I think is what the real-life test reliability of the PHQ-9 or a similar score has been reported as). That is actually going to be low in many contexts, but depressive symptoms could fluctuate over two weeks. In our case, we can't really just repeat the test right afterward - people could remember and just regurgitate their answers. If you are taking someone's blood pressure, you will be able to measure at shorter intervals.

In your case, if you calculated the intra-class correlation among your 3 measures, that would assess the test-retest reliability of whatever it is you measured. I am assuming you measured them at the same time on the same people.

Weiwen Ng
  • 1,233
  • 6
  • 16