1

We have a survival dataset in which the control group has zero events, while the other group does have some few events. Since hazard ratio calculation would yield division-by-zero errors, it appears to be common to correct for this issue by adding a fixed number (usually 0.5) to all cells. However, my research on this correction has turned up almost exclusively meta-analysis studies [1-3].

It is not immediately clear to me if this correction can also be legitimately applied to our single study. It seems counterintuitive to report a hazard ratio at all, given that the control group ultimately has zero hazard at all times of our dataset, meaning that the other group would have an "infinitely higher" risk (which, of course, does not reflect reality either).

Similar problems have been discussed before [4-7] but I could not find published/citeable evidence that using a fixed correction is legitimate outside of meta-analysis, or more specifically, in our single cohort.

References

[1] https://handbook-5-1.cochrane.org/chapter_16/16_9_2_studies_with_zero_cell_counts.htm

[2] https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.2528

[3] https://ebmh.bmj.com/content/21/2/72.long

[4] Cox regression when reference group had zero events

[5] Dealing with no events in one treatment group - survival analysis

[6] https://www.researchgate.net/post/How_to_calculate_OR_odd_ratio_if_one_of_groups_is_0_in_a_case-control_study

[7] https://www.researchgate.net/post/How_can_I_calculate_the_odds_ratio_CI_and_P_values_when_I_have_a_null_value

[EDIT] Edited to correct false display of references. [/EDIT]

Chris
  • 13
  • 3
  • From your question it seems that you have a simple 2-group comparison for which a standard [log-rank test](https://en.wikipedia.org/wiki/Logrank_test) can evaluate statistical significance. Your sense is correct that a hazard ratio and the associated concept of proportional hazards are meaningless when there are no events in one group. What do you hope to gain by the proposed type of correction? – EdM Mar 28 '19 at 17:09
  • Building on @EdM 's comment, possibly relevant aside: hazard ratios can have no unbiased causal interpretation, because they bake backdoor confounding right into their conditional definition. See Fine point 17.2 in Hernán, M. A., & Robins, J. M. (2019). [*Causal Inference*](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/) Boca Raton, FL: Chapman & Hall/CRC. – Alexis Mar 28 '19 at 17:28
  • @EdM Indeed, log-rank test confirms significant difference, but lacks an estimate of the effect size. I agree that the hazard ratio (HR) is probably not meaningful in this scenario, but it's the most reported parameter for describing differences in survival curves and apparently some people *do* use said correction to generate HR out of thin air, so I need citeable evidence to convince the lead author not to use it here. Graphpad Prism generates a seamingly meaningful HR, probably by using a correction like that, so I'm struggling to convince people why reporting it may be inappropriate. – Chris Mar 28 '19 at 17:45
  • @Alexis Thank you for the interesting read! The chapter seems to be a sound criticism of hazard ratios in general. While very insightful, I could not find a statement on the (il-)legitimacy of using continuity correction in the calculation of hazard ratios. You are right to question the use of HR in the first place, but currently I'm looking for citeable evidence that specifically having zero events in one group is enough to skip reporting an HR generated by adding 0.5 to all cells (which seems to be common practice in meta analysis, see my links above). – Chris Mar 28 '19 at 17:55

1 Answers1

1

GraphPad Prism uses what its software guide calls the "Mantel-Haenszel" approach as one of its ways to estimate a hazard ratio (HR). That is based on the difference between the number of events observed in one of the groups and the number of events that would have been expected in that group if there were no difference in survival between the 2 groups. It essentially works with an HR estimate between the survival curve of the group with events and a weighted average survival curve including both groups. That provides estimates of the HR and of associated confidence intervals (CI), even if there are no events in one of the 2 groups.

The other method used by GraphPad Prism for HR estimates is what they call the "logrank" approach.* This is based on the ratios of observed events in the 2 groups and thus will give either 0 or infinity for the HR in your case, consistent with your expectation. As its software guide says: "the [HR] results can differ when several subjects die at the same time or when the hazard ratio is far from 1.0."

The advantage of using the "Mantel-Haenszel" HR estimate in this type of situation is that it at least gives some idea of possible error in the hazard ratio and thus of an effect size. One disadvantage is that you have no way to test the underlying proportional hazard assumption that makes the HR interpretable. Another possible disadvantage is that I'm not sure how valid the assumptions underlying the calculation of HR and CI would be in this situation, even if there were a true proportional hazard that would be seen with more data or later time points.

Provided that authors appropriately cite the method used (something like: "Mantel-Haenszel estimates from GraphPad Prism, version 7"), then there is nothing necessarily wrong with presenting those HR and CI values. You are certainly correct, however, that interpretation of those values can be open to question.

What you found about adding 0.5 to cells with 0 counts as a "continuity correction" isn't what GraphPad Prism is doing here. That is done in meta-analyses to allow pooling of information from contingency tables across multiple studies. Adding to the confusion in nomenclature, the associated test for such meta-analysis is often called the "Cochran-Mantel-Haenszel" test.

*They also note that what they call the "Mantel-Haenszel" approach is called instead the "logrank" approach in the reference they cite but don't seem to link to. Terminology here can be quite confusing, differing among sources.

EdM
  • 57,766
  • 7
  • 66
  • 187
  • Thank you very much for clearing up my confusion in such detail. Have a good weekend! – Chris Mar 29 '19 at 12:54
  • One more follow-up question, if you find the time. Based on your answer that the continuity correction is mostly used in meta-analyses, would you say that the answers in REF [6] and [7] (of the references in my original post) are to be considered problematic? If I'm not mistaken, people suggested using a continuity correction outside of meta-analyses in those posts. – Chris Mar 29 '19 at 13:43
  • @Chris [continuity correction](https://en.wikipedia.org/wiki/Yates%27s_correction_for_continuity) has a long history for situations with small counts in a single contingency table. That's an attempt better to provide a continuous distribution as assumed by chi-square tests. Your concerns about interpretation of odds ratios calculated after that correction, however, are valid. Again, specifying the method clearly allows the reader to judge the quality of the result. [Fisher's exact test](https://en.wikipedia.org/wiki/Fisher%27s_exact_test) is now often used instead with low count numbers. – EdM Mar 29 '19 at 14:07
  • Thanks so much for yet another answer! So what I'm hearing is: Both the continuity correction and Graphpad's Mantel-Haenszel HR-Estimation are *in principle* legitimate ways of obtaining an HR in our table where one group has zero events (as long as properly disclosed), while both suffer from the mentioned problems of interpretation and meaningfulness. Could you confirm if I understood you correctly? – Chris Mar 29 '19 at 14:59
  • @Chris I think that's a reasonable way to put it. – EdM Mar 29 '19 at 15:37
  • Thank you for the time spent on my questions! – Chris Mar 29 '19 at 15:40