0

I need to state that there is no difference between the outcomes of the two tests (method 1 = fast and simple, method 2 = slow and hard to reproduce). I thought that a non-significant t-test would be enough, but I was wrong (can't affirm that a non-significant t-test means equivalence). So I've found the TOST (two one-sided t-tests) or equivalence test. Anyway, I'm struggling to get it correctly.

The TOST test examines the means, standard deviation and effect size (Cohen's d) to state that the values are equivalent. But, since I have almost the same results between method 1 and method 2, the Cohen's d will be obviously lower, so it does not allow me to compute the equivalence.

Am I doing something wrong?

mean_1=43.08649, mean_2=42.59865, sd_1=8.060118, sd_2=8.441185, n_1=37, n_2=37

Pitouille
  • 1,506
  • 3
  • 5
  • 16
Fede R
  • 1
  • 2
  • 2
    What is the minimum difference between method 1 and method 2 which you find relevant? In other words: what is the boundary of "difference too small to care about" and "meaningful difference"? – Alexis Dec 02 '21 at 18:21
  • Is it something that I have to decide? I'm having troubles to understand this. Should I decide the interval I state that this assumption is true or not? Something like "Ok, for me a difference of + - 1 is not relevant"? – Fede R Dec 02 '21 at 19:31
  • Yes, exactly! The equivalence threshold is a **researcher *choice***. See [my answer to this question](https://stats.stackexchange.com/questions/551637/null-hypothesis-stated-as-there-is-an-effect/551638#551638). – Alexis Dec 02 '21 at 19:42
  • Cool, thank you. Could you give me another help? Check me out. From the previous data, I'm placing my boundaries at +/- 0.5 so I get (with R): The equivalence test was significant, t(71.85) = -1.896, p = 0.031, given equivalence bounds of -4.126 and 4.126 (on a raw scale) and an alpha of 0.05. So, based on these results, I can state that my data are "statistically equivalent to 0" because the TOST p-value is < 0.05 (My alpha set).. right? – Fede R Dec 02 '21 at 20:22
  • The equivalence threshold is part of your findings. So "I found evidence the difference between method 1 & method two was equivalent **within an equivalence range of** $\pm0.5$ at the $\alpha=0.05$ level. – Alexis Dec 02 '21 at 23:04
  • Thank you Alexis, you helped me a lot. – Fede R Dec 03 '21 at 08:56

0 Answers0