What are the pros and cons of using the logrank vs. the Mantel-Haenszel method for computing the Hazard Ratio in survival analysis?

Question

One way to summarize the comparison of two survival curves is to compute the hazard ratio (HR). There are (at least) two methods to compute this value.

• Logrank method. As part of the Kaplan-Meier calculations, compute the number of observed events (deaths, usually) in each group ($Oa$, and $Ob$), and the number of expected events assuming a null hypothesis of no difference in survival ($Ea$ and $Eb$). The hazard ratio then is: $$ HR= \frac{(Oa/Ea)}{(Ob/Eb)} $$
• Mantel-Haenszel method. First compute V, which is the sum of the hypergeometric variances at each time point. Then compute the hazard ratio as: $$ HR= \exp\left(\frac{(Oa-Ea)}{V}\right) $$ I got both these equations from chapter 3 of Machin, Cheung and Parmar, Survival Analysis. That book states that the two methods usually give very similar methods, and indeed that is the case with the example in the book.

Someone sent me an example where the two methods differ by a factor of three. In this particular example, it is obvious that the logrank estimate is sensible, and the Mantel-Haenszel estimate is far off. My question is if anyone has any general advice for when it is best to choose the logrank estimate of the hazard ratio, and when it is best to choose the Mantel-Haenszel estimate? Does it have to do with sample size? Number of ties? Ratio of sample sizes?

Answer 1

I think I figured out the answer (to my own question). If the assumption of proportional hazards is true, the two methods give similar estimates of the hazard ratio. The discrepancy I found in one particular example, I now think, is due to the fact that that assumption is dubious.

If the assumption of proportional hazards is true, then a graph of log(time) vs. log(-log(St)) (where St is the proportional survival at time t) should show two parallel lines. Below is the graph created from the problem data set. It seems far from linear. If the assumption of proportional hazards is not valid, then the concept of a hazard ratio is meaningless, and so it doesn't matter which method is used to compute the hazard ratio.

I wonder if the discrepancy between the logrank and Mantel-Haenszel estimates of the hazard ratio can be used as a method to test the assumption of proportional hazards?

Answer 2

If I'm not mistaken, the log-rank estimator you reference is also known as the Pike estimator. I believe it's generally recommended for HR < 3 because it exhibits less bias in that range. The following paper may be of interest (note that the paper refers to it as O/E):

• Estimation of the Proportional Hazard in Two-Treatment-Group Clinical Trials (Bernstein, Anderson, Pike)

[...] The O/E method is biased but, within the range of values of the ratio of the hazard rates of interest in clinical trials, it is more efficient in terms of mean square error than either CML or the Mantel-Haenszel method for all but the largest trials. The Mantel-Haenszel method is minimally biased, gives answers very close to those obtained using CML, and may be used to provide satisfactory approximate confidence intervals.

Answer 3

There are actually several more methods and the choice often depends on whether you are most interested in looking for early differences, later differences or - as for the log-rank test & the Mantel-Haenszel test - give equal weight to all time points.

To the question at hand. The log-rank test is in fact a form of the Mantel-Haenszel test applied to survival data. The Mantel-Haenszel test is usually used to test for independence in stratified contingency tables.

If we try to apply the MH test to survival data, we can start by assuming that events at each failure time are independent. We then stratify by failure time. We use the MH methods for by making each failure time a strata. Not surprisingly they often give the same result.

The exception occcurs when more than one event occurs simultaneously - multiple deaths at exactly the same time point. I can't remember how the treatment then differs. I think the log-rank test averages over the possible orderings of the tied failure times.

So the log-rank test is the MH test for survival data and can deal with ties. I've never used the MH test for survival data.

Answer 4

I thought I'd stumbled across a web site and reference that deals exactly with this question:

http://www.graphpad.com/faq/viewfaq.cfm?faq=1226 Start from "The two methods compared".

The site references the Berstein paper ars linked (above):

http://www.jstor.org/stable/2530564?seq=1

The site summarises Berstein et al's results nicely, so I'll quote it:

The two usually give identical (or nearly identical) results. But the results can differ when several subjects die at the same time or when the hazard ratio is far from 1.0.

Bernsetin and colleagues analyzed simulated data with both methods (1). In all their simulations, the assumption of proportional hazards was true. The two methods gave very similar values. The logrank method (which they refer to as the O/E method) reports values that are closer to 1.0 than the true Hazard Ratio, especially when the hazard ratio is large or the sample size is large.

When there are ties, both methods are less accurate. The logrank methods tend to report hazard ratios that are even closer to 1.0 (so the reported hazard ratio is too small when the hazard ratio is greater than 1.0, and too large when the hazard ratio is less than 1.0). The Mantel-Haenszel method, in contrast, reports hazard ratios that are further from 1.0 (so the reported hazard ratio is too large when the hazard ratio is greater than 1.0, and too small when the hazard ratio is less than 1.0).

They did not test the two methods with data simulated where the assumption of proportional hazards is not true. I have seen one data set where the two estimate of HR were very different (by a factor of three), and the assumption of proportional hazards was dubious for those data. It seems that the Mantel-Haenszel method gives more weight to differences in the hazard at late time points, while the logrank method gives equal weight everywhere (but I have not explored this in detail). If you see very different HR values with the two methods, think about whether the assumption of proportional hazards is reasonable. If that assumption is not reasonable, then of course the entire concept of a single hazard ratio describing the entire curve is not meaningful

The site also refer to the dataset in which "the two estimate of HR were very different (by a factor of three)", and suggest that the PH assumption is a key consideration.

Then I thought, "Who authored the site?" After a little searching I found it was Harvey Motulsky. So Harvey I've managed to reference you in answering your own question. You've become the authority!

Is the "problem dataset" a publicly available dataset?