In t-tests (paired or independent) we test if the difference in means if 0 (or another value). I'm wondering if it's possible to instead test if the difference is less than x?
Asked
Active
Viewed 1,187 times
1 Answers
11
Sure, you can do that. You don't have to test against a null hypothesis of $0$ (sometimes called a "nil null"); you can test against any value. You also don't have to do a two-tailed test; you can perform a one-tailed test (when specified a-priori). The paired $t$-test is:
$$
t = \frac{\bar x_D - \mu_{\rm null}}{\frac{s_D}{\sqrt N}}
$$
Thus, to combine the two less typical possibilities noted above, you substitute your specific value for $\mu_{\rm null}$, and run a one-tailed test.
Here is a simple example (coded in R
):
set.seed(2786) # this makes the example exactly reproducible
x1 = rnorm(20, mean=3, sd=5) # I'm generating data from a normal distribution
x2 = x1 - rnorm(20, mean=0, sd=1) # the true difference is 0
## this is a paired t-test of whether the difference is <1:
t.test(x1, x2, mu=1, alternative="less", paired=TRUE)
#
# Paired t-test
#
# data: x1 and x2
# t = -7.5783, df = 19, p-value = 1.855e-07
# alternative hypothesis: true difference in means is less than 1
# 95 percent confidence interval:
# -Inf -0.02484498
# sample estimates:
# mean of the differences
# -0.3278085

gung - Reinstate Monica
- 132,789
- 81
- 357
- 650
-
Thank you! I'm really wishing I took stats more seriously in university. I'm finding it so useful and interesting now that I'm actually applying it. If I want to show that $x$ and $y$ are very similar within **1** units of measurement, would I use the same values you just did? (alternative is that the true mean is less than 1 instead of the alternative being the true mean is greater than 1) – DeanAttali Apr 08 '16 at 23:15
-
3Yes, that's called [two one-sided tests](http://stats.stackexchange.com/tags/tost/info). Questions on CV on this topic can be found by searching on [tag:tost]. You may also be interested in reading my answer here: [Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?](http://stats.stackexchange.com/a/85914/7290) – gung - Reinstate Monica Apr 08 '16 at 23:19
-
Thank you for that. That's a great resource. My next question was going to be if that tost can be paired, and after finding an R package for it, it seems like it does have that parameter! – DeanAttali Apr 08 '16 at 23:52
-
You can use TOST w/ paired tests, if that's what you mean. – gung - Reinstate Monica Apr 08 '16 at 23:54
-
sorry, I'm a bit confused actually now: so if I want to show that x and y are no more different than some value, do I carry out a paired TOST, or the t-test that you showed in your answer? – DeanAttali Apr 08 '16 at 23:59
-
I don't follow you. The t-test that I demonstrated was a paired t-test. If you performed 2 paired t-tests (dif < upper limit; dif > lower limit), that is TOST. – gung - Reinstate Monica Apr 09 '16 at 00:01
-
So is the *p-value* from a TOST simply the minimum of the two separate *p-values* from the two tests? – DeanAttali Apr 09 '16 at 00:09
-
1@daattali please don't ask question after question in comments when you're not clarifying something relating to the meaning of the answer to the actual question you asked about, but asking a new question that arises because of understanding you gain from the answer (the first is fine, the second is not). [Equivalence tests](https://www.google.com/search?q=equivalence+tests|testing) (which are generally done via TOST) are a whole subject, and many posts on site relate to them. Read about them and then if you have a question not already answered on site, post a new question. – Glen_b Apr 09 '16 at 01:36
-
2@Glen_b I thought my question is directly related to the original question, because I wanted to know how TOST (which was suggested in the comments) related to the original question+answer. I now understand that a TOST is really just performing the t-test in the answer twice, and if both null hypotheses are rejected, then the null hypothesis of the TOST is rejected --> conclude equivalence. That's the missing link that I was trying to understand. – DeanAttali Apr 09 '16 at 05:04