8

I am evaluating a journal article regarding its statistical interactions.

The article is trying to establish a relationship between less-tight control of blood pressure and progression to severe hypertension. It suspects that pre-existing hypertension is a prognostic factor (given $p=0.048<0.05$).

Its defense is that

The $p$-value is marginally significant. This may be by chance. If pre-existing hypertension were truly an adverse prognostic factor, the rates of low blood glucose should have been higher in the less-tight control group and the tight control group, compared with those without gestational hypertension.

Is this defense really valid? This argument sounds like refuting "pre-existing hypertension" as a risk factor for progression to severe hypertension.

Below are the data.

\begin{array} {|c|c|} \hline \text{Pre-existing hypertension} & OR=2.11 \\ \hline \text{Gestational hypertension} & OR=1.13 \\ \hline & p=0.048 \\ \hline \end{array}

\begin{array} {|c|c|c|} \hline \text{Pre-existing hypertension} & \text{Progression} & \text{No progression} \\ \hline \text{Less-tight control} & 159 & 210 \\ \text{Tight control} & 96 & 267 \\ \hline \end{array}

\begin{array} {|c|c|c|} \hline \text{Gestational hypertension} & \text{Progression} & \text{No progression} \\ \hline \text{Less tight control} & 41 & 83 \\ \text{Tight control} & 38 & 87 \\ \hline \end{array}

user2513881
  • 157
  • 2
  • 8
  • See [here](http://www.nejm.org/doi/suppl/10.1056/NEJMoa1404595/suppl_file/nejmoa1404595_appendix.pdf) for full information. – user2513881 Feb 09 '16 at 08:08
  • 2
    The key is, "The p-value is marginally significant." Essentially, the author goes on to say they expected a stronger effect and thus a smaller p-value (stronger evidence against the null hypothesis). I personally like that fact that this author considers that the p-value is marginal and not just going with the easy answer of "Oh p-value < 0.05 must be statistically significant." – Lauren Goodwin Feb 09 '16 at 16:29
  • 8
    @Lauren For counterpoint, consider that this is a *post hoc* statement. A truly rigorous experimenter would establish criteria both for p-values and effect sizes in advance and respect those criteria in their assessment of the results. If, as implied here, the p-value criterion is 0.05, then this result of 0.048 is *significant,* period. To state otherwise would be akin to challenging you to a mile-long race, losing, and then at the finish line proposing that the race really was two miles and therefore hasn't been lost. – whuber Feb 09 '16 at 16:43

1 Answers1

9

I don't have the required reputation to vote, so I'll add it as an answer instead.

I fully agree with what @whuber said. The typical approach in this kind of study is to a priori declare a level of significance. Quoted from the article, the authors indeed do this,

To accommodate the many comparisons made, two-tailed P values of less than 0.01 for the secondary outcomes and less than 0.001 for other outcomes were considered to indicate statistical significance

and

... for these exploratory analyses, the Breslow–Day test of homogeneity was used and P values of less than 0.05 were considered to indicate statistical significance

To mention a result as "marginally significant" is plainly wrong when you have already declared your levels as significant. Either something is significant, or it is not. Just to add, the authors also calculated the study had 80% power, assuming a detection level of alpha < 0.05.

On the other hand, if the authors provide an effect size (such as the OR) that has a p-value < 0.05 but is extremely close to 1, then I think it is fully justified to say "this was indeed significant, but has no clinical relevance due to the low effect size".

demodw
  • 449
  • 2
  • 8