5

I'm trying to run a t-test using t.test() but am confused about the p-value and confidence interval. Looking at the output I see a p-value of ~0.03. That leads me to believe I'd reject the null hypothesis at a 95% level. However, when I compare the difference in my means, 0.25, is solidly within the 95% confidence interval shown. With a small p-value I thought it would be outside, but I'm new to this whole process. How do I reconcile a small p-value with a difference within the confidence interval? Is the difference in mean not what the confidence interval shows?

> t.test(childs ~ vet, data = cols)
data:  childs by vet
t = 2.163, df = 130.97, p-value = 0.03236
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.02163258 0.48494263
sample estimates:
mean in group FALSE  mean in group TRUE 
           2.707134            2.453846 

> 2.707 - 2.4538
[1] 0.2532
Paul Rubel
  • 153
  • 1
  • 1
  • 5
  • Just to maybe help you answer your own questions: what does the 95% interval represent here, and what is the null hypothesis that you test? – horseoftheyear Oct 22 '15 at 14:13
  • The null hypothesis is that both means are equal. My thoughts on 95% is that it's the range the difference in mean could take due to selection while still actually being equal. – Paul Rubel Oct 22 '15 at 14:32
  • "*I'd reject the null hypothesis at a 95% level.*" --- that's not how it works. You'd reject at the 5% level (you'd never conduct a test with a type I error rate of 95%) – Glen_b Oct 22 '15 at 15:10
  • 2
    Sure, thanks for the clarification. My confusion was I thought that the confidence interval was centered around 0 (null being true) but it's the observed result. – Paul Rubel Oct 22 '15 at 15:35

2 Answers2

10

.2532 is always going to be in the confidence interval since this is how the interval was constructed. The formula is

$(\bar{x_1} - \bar{x_2}) \pm t^*_{df} \times SE(\bar{x_1} - \bar{x_2})$

Further more $\bar{x_1} - \bar{x_2}$ is called your observed result (or sample difference in means) and as you can see this is where the interval will be centered and thus always be contained in the confidence interval.

The hypotheses you are testing are

$H_0: \mu_1 - \mu_2 = 0$ vs. $H_1: \mu_1 - \mu_2 \neq 0$

Therefore, you when checking whether or not the null hypothesis is rejected based on the confidence interval you look to see if the null hypothesized value is in the confidence interval. In this case it is 0, since 0 is not in the interval we reject the null hypothesis at the .05 significance level. This is consistent with the decision made from the p-value.

Hopefully this helps but my response is very much an overview and there is a lot more going on conceptually here that I did not go into.

Lauren Goodwin
  • 561
  • 3
  • 10
-5

Why do smart people have such trouble explaining simple things.

  1. In laymen terms, p-value = 0.03236 means there is a 97% chance sample A > sample B.
  2. You're 95% confident the difference between sample A and B is between 0.02 and 0.48. This is a weird result given the difference between A and B is only 0.25. This indicates your data is skewed which probably means the 97% chance mentioned above isn't true due to T-Test assumptions being violated.

A normal 95% CI for a difference of means = 0.25 would be something smaller like 0.20 to 0.30 where you could say if my difference of means is 0.25 then the true mean is probably somewhere between 0.20 and 0.30. Your test shows true mean between 0.02 to 0.48 which is a huge range. Run a t.test on some fake data with a normal dist and the result will make sense.

Jay
  • 1
  • 1
  • 6
    Welcome to Cross Validated. I'm not very smart but I believe your first bullet is in error. See [here](https://en.wikipedia.org/wiki/P-value) and [here](https://stats.stackexchange.com/questions/166323/misunderstanding-a-p-value) for starters. – SecretAgentMan Nov 02 '18 at 05:28
  • 2
    And the 2nd: The point estimate is smack in the middle of the confidence interval, by construction. – Scortchi - Reinstate Monica Nov 02 '18 at 08:48