8

In t-tests (paired or independent) we test if the difference in means if 0 (or another value). I'm wondering if it's possible to instead test if the difference is less than x?

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
DeanAttali
  • 247
  • 2
  • 8

1 Answers1

11

Sure, you can do that. You don't have to test against a null hypothesis of $0$ (sometimes called a "nil null"); you can test against any value. You also don't have to do a two-tailed test; you can perform a one-tailed test (when specified a-priori). The paired $t$-test is:
$$ t = \frac{\bar x_D - \mu_{\rm null}}{\frac{s_D}{\sqrt N}} $$ Thus, to combine the two less typical possibilities noted above, you substitute your specific value for $\mu_{\rm null}$, and run a one-tailed test.

Here is a simple example (coded in R):

set.seed(2786)                     # this makes the example exactly reproducible
x1 = rnorm(20, mean=3, sd=5)       # I'm generating data from a normal distribution
x2 = x1 - rnorm(20, mean=0, sd=1)  # the true difference is 0
## this is a paired t-test of whether the difference is <1:
t.test(x1, x2, mu=1, alternative="less", paired=TRUE)
# 
#  Paired t-test
# 
# data:  x1 and x2
# t = -7.5783, df = 19, p-value = 1.855e-07
# alternative hypothesis: true difference in means is less than 1
# 95 percent confidence interval:
#         -Inf -0.02484498
# sample estimates:
# mean of the differences 
#              -0.3278085 
gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
  • Thank you! I'm really wishing I took stats more seriously in university. I'm finding it so useful and interesting now that I'm actually applying it. If I want to show that $x$ and $y$ are very similar within **1** units of measurement, would I use the same values you just did? (alternative is that the true mean is less than 1 instead of the alternative being the true mean is greater than 1) – DeanAttali Apr 08 '16 at 23:15
  • 3
    Yes, that's called [two one-sided tests](http://stats.stackexchange.com/tags/tost/info). Questions on CV on this topic can be found by searching on [tag:tost]. You may also be interested in reading my answer here: [Why do statisticians say a non-significant result means “you can't reject the null” as opposed to accepting the null hypothesis?](http://stats.stackexchange.com/a/85914/7290) – gung - Reinstate Monica Apr 08 '16 at 23:19
  • Thank you for that. That's a great resource. My next question was going to be if that tost can be paired, and after finding an R package for it, it seems like it does have that parameter! – DeanAttali Apr 08 '16 at 23:52
  • You can use TOST w/ paired tests, if that's what you mean. – gung - Reinstate Monica Apr 08 '16 at 23:54
  • sorry, I'm a bit confused actually now: so if I want to show that x and y are no more different than some value, do I carry out a paired TOST, or the t-test that you showed in your answer? – DeanAttali Apr 08 '16 at 23:59
  • I don't follow you. The t-test that I demonstrated was a paired t-test. If you performed 2 paired t-tests (dif < upper limit; dif > lower limit), that is TOST. – gung - Reinstate Monica Apr 09 '16 at 00:01
  • So is the *p-value* from a TOST simply the minimum of the two separate *p-values* from the two tests? – DeanAttali Apr 09 '16 at 00:09
  • 1
    @daattali please don't ask question after question in comments when you're not clarifying something relating to the meaning of the answer to the actual question you asked about, but asking a new question that arises because of understanding you gain from the answer (the first is fine, the second is not). [Equivalence tests](https://www.google.com/search?q=equivalence+tests|testing) (which are generally done via TOST) are a whole subject, and many posts on site relate to them. Read about them and then if you have a question not already answered on site, post a new question. – Glen_b Apr 09 '16 at 01:36
  • 2
    @Glen_b I thought my question is directly related to the original question, because I wanted to know how TOST (which was suggested in the comments) related to the original question+answer. I now understand that a TOST is really just performing the t-test in the answer twice, and if both null hypotheses are rejected, then the null hypothesis of the TOST is rejected --> conclude equivalence. That's the missing link that I was trying to understand. – DeanAttali Apr 09 '16 at 05:04