2

Say I have a variable treatment A/B (0,1) and effect yes/no (0,1).

An easy way to calculate a P value from such a dataset would be to use Fisher's Exact test or a Chi2 test.

However, we also have the option of simply running a logistic regression with 1 dependent variable and 1 independent variable.

Would the former tests be "better" to answer the question, or would a regression be just as good?

Paze
  • 1,751
  • 7
  • 21
  • It is possible to guess, but... for what type of hypothesis do you wish to calculate a p-value? Also, what test do you consider to perform when 'running a logistic regression'? How is it different from what you consider to be 'running a logistic regression' in the case of a single binary DV and IV? – Sextus Empiricus Dec 19 '18 at 17:42

2 Answers2

4

To add to TMKB's answer:

Logistic regression is used for either prediction or analysis of relationships between variables. If the goal is the latter, the main difference between Pearson's chi-squared test and logistic regression usually lies in two aspects:

  1. Choice of statistical test (which determines the p-value of the regression coefficient) for the logistic regression - the options are: Wald, likelihood ratio and score test (the latter is rarely used in practice, but in fact it seems to be equivalent to the classic chi-squared test)
  2. Corrections that are applied (if any) - often, for chi-squared tests a continuity correction (Yates) is applied

If you choose the score test for logistic regression and do not apply a continuity correction for the chi-squared test, the results of chi-squared and logistic regression will be identical. For more information, take a look at this very(!) helpful answer by gung: Why do my p-values differ between logistic regression output, chi-squared test, and the confidence interval for the OR?

When applying Fisher's exact test, however, you condition on the margins (as pointed out by @Scortchi in the comments). Doing so implies the somewhat strange assumption that you know the true margins although this is usually not the case. Since in most practical situations these margins are not fixed by design, there are good arguments to not use Fisher's exact test. For reference, see Lydersen et al. 2009, Stat. Med., 28 (7), "Recommended tests for association in 2×2 tables" (link).

In conclusion: chi-squared test and logistic regression should lead to similar results or even identical, if you specify them as required. In some situations, logistic regression might provide you with an advanced tool kit since you can choose different tests which might lead to a gain in power in some situations. However, if you do not have a clear reasoning to choose either Wald or likelihood ratio and you actually simply want to analyze the relation between two binary variables, performing a logistic regression might be considered as overcomplication when you could instead simply perform a chi-squared test.

LuckyPal
  • 1,512
  • 7
  • 15
  • 3
    (+1) Note if you're carrying out FET you're not (necessarily) *assuming* all margins are fixed; rather conditioning on the margins you observe. For the reason see [On Fisher's exact test: What test would have been appropriate if the lady hadn't known the number of milk-first cups?](https://stats.stackexchange.com/q/136584/17230) & [When and how to avoid inappropriate use of Fisher's exact test](https://stats.stackexchange.com/a/364443/17230). – Scortchi - Reinstate Monica Dec 19 '18 at 15:35
1

If you don't have any other (control) variables, In general, ceteris paribus, go for the simpeler method (e.g. could be more robust in terms of interpretation).

Logistic regression would be more appropriate if you actually need to make predictions (it is an actual model).

TMKB
  • 71
  • 2