I have used statsmodel in python to compare the means between a control and test group. The difference of the means is 29 and I get a low p-value making my results statistically significant. The 95% confidence interval is [18, 25] which does not include 0 but also doesnt include my actual difference which is 29. Is there something wrong with these results or can I assume that the 29 value just happened to be in the 5% of the times that is outside the CI?
Asked
Active
Viewed 42 times
0
-
2How did you calculate the confidence interval and the difference in means? A coding error sounds like the likely culprit. – Dave Jul 13 '20 at 18:38
-
I just used this code for the confidence intervals https://www.statsmodels.org/stable/generated/statsmodels.stats.weightstats.CompareMeans.html for the difference in means I just did the mean of each group and subtracted – nicnaz Jul 13 '20 at 18:44
-
Please provide the specific code you used, and if possible show density plots or histograms of all the values for the 2 groups. Are there any large outliers in one or the other of the groups? – EdM Jul 13 '20 at 18:50
-
Are these results impossible to happen? – nicnaz Jul 13 '20 at 19:02
-
can you include the code? – StupidWolf Jul 13 '20 at 22:27
-
In bootstrapping, for example, bias in the estimates of the quantity of interest can lead to apparent [confidence intervals that don't include the mean](https://stats.stackexchange.com/q/156235/28500). That seems highly unlikely to be the case in what you describe, but without further details on the code and the data I would hesitate to say "impossible." – EdM Jul 14 '20 at 12:50