2

Analyzing the difference between the means of two independent unbiased samples obtained from normally distributed variables with homogenous variances:

Two possible approaches:

  1. Perform a 2-sample t-test (H0: Mean1-Mean2=0)

  2. Calculate the 95% C.I of the difference between the means, and see whether it overlaps 0.

Question: Is it mathematically possible to have a 95% C.I (of the difference between means) overlapping 0, and a t-test p<0.05?
Vice-versa: is it possible to have a 95% CI non overlapping 0 and a t-test p<0.05?

Padmanabha Chowdhury
  • 137
  • 8
Alex
  • 21
  • 1
  • 2
    Casella and Berger’s *Statistical Inference* goes into this in the chapter on interval estimation. Briefly, hypothesis testing and confidence intervals are inverses of one another; the relationship is as your suspect it is. – Dave Jul 31 '21 at 04:18
  • Those are equivalent if used with care, but what is not equivalent would be constructing confidence intervals for each mean and seeing whether the confidence intervals overlapped – Henry Jul 31 '21 at 13:41
  • The two-sample situation is analyzed at https://stats.stackexchange.com/questions/18215. – whuber Jul 31 '21 at 15:07

1 Answers1

1

Provided the t test and the style of confidence interval match, the two methods should give the same result. By 'match' I mean that they should both be pooled variance procedures or both Welch procedures, and both should be two-sided (or both one-sided) procedures.

This is roughly illustrated by The following fictitious normal data and the Welch two-sided procedure t.test in R (without parameters var.eq=T, alt="less", or alt="g").

set.seed(2021)
x1 = rnorm(55, 100, 15)
x2 = rnorm(55, 110, 15)
t.test(x1,x2)

       Welch Two Sample t-test

data:  x1 and x2
t = -2.0026, df = 102.02, p-value = 0.04787
alternative hypothesis: 
  true difference in means is not equal to 0
95 percent confidence interval:
 -11.57271742  -0.05562016
sample estimates:
mean of x mean of y 
 99.11428 104.92845

The P-value $0.04787$ is just a bit below 5% and the 95% confidence interval just barely avoids $0.$

BruceET
  • 47,896
  • 2
  • 28
  • 76