0

I am reading a research article and trying to interpret and report the results. The article is comparing gabapentin and estrogen for the treatment of hot flashes. There was no difference between gabapentin group and estrogen group change scores, t=0.53 and p=0.63. The alpha value set by the researcher was p<0.017. I know that with this p value I cannot rejected the null hypothesis. I am assuming that the null hypothesis is that there is no difference between the two groups? However, this does not necessarily mean that the null hypothesis is true. How do I report this? What is this p value telling me about the results? One of my references since that the p value is the probability that the obtained result is due to chance alone, how does this relate to these results?

Jackie
  • 1
  • 4
    You interpret the high p-value as saying that you have insufficient evidence to conclude that there is a difference between the groups. As for the definition, it is the probability of obtaining a more extreme result than the observed result, assuming that the null hypothesis is true. – mkt Apr 03 '18 at 06:03
  • 1
    To emphasize the last point made by @mkt: the p value does *not* represent "the probability that the obtained result is due to chance alone" – user20160 Apr 03 '18 at 06:31
  • 2
    @mkt it feels like you could just copy your comment into an answer, even if it is quite short. – Björn Apr 03 '18 at 06:38
  • @Björn Fair point - I'm always a bit uncertain with short answers. I'll think about fleshing it out a bit more and turning it into an answer. – mkt Apr 03 '18 at 07:38
  • So if I understand this correctly, when comparing the gabapentin and estrogen groups I cannot say with confidence that there is a difference between the group scores. However, this does not necessarily mean that the null hypothesis is correct and there is no difference between the groups. This could be due to a chance sample error from a small sample size? Is this correct? Does the fact that the p value is so large mean anything? Does is mean more than if the p value was p = 0.16 which is still over the alpha? – Jackie Apr 03 '18 at 12:32

1 Answers1

0

P value is NOT "the probability that the obtained result is due to chance alone." A P value of 0.63 means that there is a 63% chance that if you accept the alternate hypothesis (Ha; that there is a difference) you will be mistakenly rejecting the null hypothesis (Ho; no difference). In other words you cannot say with confidence that there is a difference. Hypothesis testing will not allow you to "accept" the Ho. It is like in criminal court the defendant is found either guilty (reject Null), or not guilty (cannot reject Null); there is no finding of innocent (accept Null). If the study was powered adequately and you believe that finding a difference in effect would require such a high sample size as to not clinically significant you might say (in the discussion) that there is likely no difference, however no test can determine no difference (i.e. accept the Null).

mrnmibrc
  • 1
  • 1
  • Also alpha of 0.017 is not standard. Most of the time alpha is 0.05. – mrnmibrc Apr 03 '18 at 07:12
  • 4
    I assume the unusual alpha is because of a multiple comparisons correction, such as Bonferroni. 0.017 is approximately 0.05/3 – mkt Apr 03 '18 at 07:37
  • A test for equivalence (difference of at most +- delta) is possible, but rarely used. – Björn Apr 03 '18 at 09:14
  • That is correct they did use the unusual alpha because of a multiple comparison correction. – Jackie Apr 03 '18 at 12:15