4

Suppose two samples X and Z are dependent and of equal length. I see two approaches to investigate the possible difference in means:

1) Use a paired t-test if the normality assumption is met for each sample.

2) Subtract the samples from each other and apply a one-sample t-test to test whether the difference (X-Z) differs from 0.

Is there a difference in the way the test statistics are calculated? This thread (Paired difference t-test vs independent two sample t-test to assess means difference) with an excellent answer suggests there is not.

I confess, I often give in to using the test that suits me best for reporting. When both samples approximate normality I use a paired t-test, when they do not but the difference (X-Y) does I use a one-sample t-test. Is the latter approach wrong? In the proposed should situation should I always favour option (1) over option (2)?

Community
  • 1
Jef Van Alsenoy
  • 81
  • 9
  • What are the test statistics in each case? – Glen_b Aug 12 '16 at 11:47
  • They are exactly the same t(2677) = -23.741. However, if I'd have to criticize someone who is checking whether a sample mean differs from 0 and he (or she) would do a one-sample t-test after checking normality I'd have nothing to comment. But, if that same person would be performing a paired t-test on two samples that _are_ _not_ normally distributed, I'd criticize that person for performing a t-test when normality is violated. So where's the catch? The fact that the one sample t-test was not performed on a _sample_ by strict definition? Just trying to criticize my own approach here! – Jef Van Alsenoy Aug 12 '16 at 12:19
  • I didn't mean what is the value of the test statistic ... I meant the formula you substitute into to get those values. (It's a question with a point. I was hoping once you look at the formulas that you might understand something important about the question) – Glen_b Aug 13 '16 at 01:15
  • Specifically, they are *exactly* the same test with exactly the same assumptions, which relate to normality of the population of differences. The actual assumption is likely never satisfied in practice . The question you need to consider is not "can I detect non-normality?" but "is the non-normality I have going to affect the properties of my test (significance level and power) more than I am prepared to tolerate?" ... which is more of an effect-size kind of question than a hypothesis testing one – Glen_b Aug 13 '16 at 02:06
  • Yup! I did actually look at the formulas, that' why I posted the thread. I was just confused with the subject of the normality assumption regarding the paired t-test. And I completely agree on the latter statements about the normality assumption. – Jef Van Alsenoy Aug 13 '16 at 08:22

1 Answers1

1

To answer my own question (community wiki, feel free to adjust), after giving it some more thought and researching a bit:

Q: Is there a difference in the way the test statistics are calculated?

A: No. I suppose @Glen_b was hinting in that direction. T-statistic remains the same, as do the DF.

Q: Is the latter approach wrong?

A: No, the former is wrong. As can be read here (http://www.biostathandbook.com/pairedttest.html): The paired t–test assumes that the differences between pairs are normally distributed. Performing a t-test without checking the normality of the difference is wrong, regardless of the normality of X or Z (the constituents of the difference).

Q: In the proposed should situation should I always favour option (1) over option (2)?

A: Either options work but normality of the difference needs to be verified in either case. You might as well do the one-sample t-test because you need to calculate the difference anyhow.

Jef Van Alsenoy
  • 81
  • 9
  • +1 though I'm not sure the normality of the difference actually does "need to be verified"; for example (i) I may have a good reason to make an a priori assumption of normality, or (ii) I may actually be quite sure the distribution *isn't* actually normal, but nevertheless be quite sure that the impact on my inference is small. – Glen_b Aug 13 '16 at 08:52
  • I agree, that sounds like a correct adjustment of the answer. – Jef Van Alsenoy Aug 13 '16 at 09:35