ANOVA for equivalence testing

Question

How to perform a ANOVA for statistical equivalence?

I read about the two one-sided test (TOST) for equivalence, but (I think) for this study design it is not possible to perform a classical t-test, so a repeated measures ANOVA is needed.

The study design is a common pre-post treatment-control design: there are two different groups (control / treatment) and dependent two measure points at the baseline and a followup (MP1 / MP2). The research question is, if the treatments are equal.

My thought is, that the ANOVA analyses the differences between the groups. Is there a post-hoc for equivalence, special in R?

---

EDIT

Thanks to D_Williams, I read the using-lsmeans vignette and the lsmeans reference manual. There is a function called test. With this function you can do equivalence / noninferiority or nonsuperiority tests. Now I got stuck with a new problem. Because of the study design I need a linear regression with repeated measures. So here is a example dataset:

 dataset <- data.frame (ID    = rep(1:16),
                  GROUP = factor(rep(c("A","B"),8)),
                   MP1   = c(15,12,20,17,28,24,17,10,14,10,25,23,9,18,19,20),
                   MP2   = c(12,9,19,10,20,15,12,5,12,10,22,15,8,17,10,19),
                   )

The linear regression should be:

data.lm <- lm (MP2 - MP1 ~ GROUP, data=dataset)

As far as I understood this answer right, Case 2b is the correct one for me. Because there are two quantitative variables and a one dichotomous variable.

Afterwards I run these commands:

library("lme4", "lsmeans", "estimablity")
data.lsm <- lsmeans(data.lm, "GROUP")
test(data.lsm, null = log(100), delta = 0.20, side="equivalence")

Delta is the equivalence margin or the range of similarity. But I am not sure what the log(100) is.

My questions are:

1. Is this approach right?

   and

2. To be honest: I don't really understand the last steps. Does anybody have some code example for studying them?

Answer 1

Since you have only two groups, there is no need to perform an ANOVA. You can perform a TOST procedure by simply building a confidence interval (CI) for a two sample problem, say x and y, where x = MP2-MP1 in the control group and y = MP2-MP1 in the treated group. You need to pay attention to the usual assumptions for the statistical tests: normality, heteroskedasticity, etc. But, once you have your CI, you can see if it is contained within the +-delta region or not. In R you can do something like this:

set.seed(12)
x = rnorm(100)
y = rnorm(70)

# for two sample t-test with equal variances
tt <- t.test(x, y, var.equal = TRUE, conf.level = 0.90)

# for Welch two sample t-test
tt.uneq <- t.test(x, y, var.equal = FALSE, conf.level = 0.90)

# for two sample Wilcoxon test
wcox <- wilcox.test(x,y, conf.int = TRUE, conf.level = 0.90)

delta <- 0.32

library(plotrix)
plotCI(x = 1, y=diff(rev(tt$estimate)), ui = tt$conf.int[2], li = tt$conf.int[1],
       xlim=c(0,4), ylab = "Confidence intervals", ylim = c(-0.6, 0.6),
       xlab ="Methods")
plotCI(x = 2, y=diff(rev(tt.uneq$estimate)), ui = tt.uneq$conf.int[2],
       li = tt.uneq$conf.int[1], add=TRUE, col=2)
plotCI(x = 3, y=wcox$estimate, ui = wcox$conf.int[2],
       li = wcox$conf.int[1], add=TRUE, col=3)
abline(h = c(-delta, delta), lwd = 2, lty = 2)

Here, to illustrate the idea I'm considering three type of tests but there are many others in the literature. Hope this helps.

Answer 2

Generally equivalence testing is very limited to simple models. I have, however, recently learned that the lsmeans package in r does equivalence testing for models more complex than a t-test. I have not used it, but I have used lsmeans and it is a very flexible tool. This might work, but you will have to check out the package vignette for the details to be sure see here

As of 2022, the emmeans package (successor to lmmeans can be used for equivalence tests of linear models; vignette info is here.