2

How do Fisher's combined probability test and the Mantel-Haenszel and Breslow-Day approaches compare, when both are feasible? Is one of these approaches preferred when estimating the combined probability under the null hypothesis?

Joel W.
  • 3,096
  • 3
  • 31
  • 45

1 Answers1

2

Fisher's method and Mantel-Haenszel are designed for different purposes. Fisher's method assumes you do NOT have any data from any of the tests, which share the same hypothesis and are independent. All it needs are the individual p-values, from which it calculates the combined test statistics $T_F=-2\sum_{i=1}^k\log p_i$. The null and alternative hypothesis are

$H_0:$ all of the separate null hypotheses are true

$H_A:$ at least one of the separate alternative hypotheses is true

Under $H_0$, $T_F$ follows a chisq distribution with $2k$ degree of freedom.

Mantel-Haenszel assumes your data is in the form of a series of 2 × 2 (n x m in general) contingency tables and the odds ratios are the SAME across all strata. The null and alternative hypothesis are

\begin{align*} H_0 &: OR_1=OR_2=\ldots OR_k = 1 & \\ H_A &: OR_1=OR_2=\ldots OR_k \ne 1 & \end{align*}

When the assumption of common odds ratios is invalid, Mantel-Haenszel can have very low power.

Fisher's method is often used in meta-analysis where you want to assess a common hypothesis by combining results of independent experiments. On the other hand, Mantel-Haenszel is used to remove confounding effect in a single experiment. You will often see the word "matching" be associated with Mantel-Haenszel as it is the standard method for matched analysis. Suppose you want to estimate a drug effect, which is assumed to interfere with sex. You would then need to group patients by their gender, estimate separately their odds ratios and calculate an overall p-value with Mantel-Haenszel.

Say, later you find another study estimating the effect of the same drug, you would need Fisher's method to calculate a combined p-value.

Hope this clarifies things a bit.

Peter

Peter
  • 725
  • 3
  • 8
  • So, does the added assumption translate into greater power? If so, do you have any idea how much greater power? – Joel W. Oct 06 '14 at 19:43
  • I am not sure about the power gain. As I said, the two methods are designed to answer different questions. There are a lot of scenarios that MH method does not apply. That is when you need Fisher's method or other approaches. For example, MH cannot be used if you response is continuous or your data is not available. To me, MH is a very powerful test for a very SPECIFIC hypothesis and you should use it when your data can be organized as described above. – Peter Oct 07 '14 at 12:38
  • @Peter-- Where might I go to find a comparison of the power of the two approaches? Also, are there other general approaches that are alternatives to the Fisher approach? – Joel W. Oct 08 '14 at 17:49
  • No I don't. For other approaches and a review of meta-analysis, you can read this paper http://www.ncbi.nlm.nih.gov/pubmed/19254930 – Peter Oct 10 '14 at 11:51