Why use the Wald test in logistic regression?

Question

Some statistical software use the Wald statistic when reporting on the regression coefficients. As examples, R and Stata report Wald by default.

The logistic regression article on Wikipedia says, unfortunately without reference:

“Rather than the Wald method, the recommended method to calculate the p-value for logistic regression is the likelihood-ratio test (LRT)”

How are Wald and LRT calculated for a logistic regression coefficient (independent variable)? This is for a reminder on how they are calculated and to highlight their differences.

From the Wald test page on Wikipedia:

• [T]he Wald test is not invariant to a reparametrization, while the Likelihood ratio tests will give exactly the same answer whether we work with R, log R or any other monotonic transformation of R.

So in the context of logistic regression, if you logged a regressor its p-value would be different compared to if it was unlogged (is this correct). Would the p-value change if the LRT was used?

From the same Wald test page:

• The other reason is that the Wald test uses two approximations (that we know the standard error, and that the distribution is χ2), whereas the likelihood ratio test uses one approximation (that the distribution is χ2).

Although Wald and likelihood ratio are asymptotically equivalent, in the logistic regression we are usually in the pre-asymptote setting, so this is not a reason to view then as equivalent.

Thus it seems that the Wald test disadvantages outweigh the advantages in the logistic setting, and the likelihood ratio is better.

It is my guess that the Wald test is used by logistic regression software routines for its easier computational efficiency, which was more important in the past when software such as R and Stata were first created. Then, through backwards compatibility and not wanting to change the semantics of their logistic functions, the Wald statistic has remained the default. Is there any evidence for this being the case?

Should I be changing the default Wald to likelihood ratio? A lesser question, is there an easy was to do this in R?

Answer 1

In logistic regression (& other generalized linear models with canonical link functions), the coefficient estimates $\hat\theta$ are arrived at by Fisher Scoring: iterating $$\vec\theta_{k+1} = \vec\theta_k + \mathcal{I}^{-1}(\vec\theta_k)U(\vec\theta_k)$$ where $\mathcal{I}$ is the Fisher information & $U$ the score, until convergence. When you're done, you're left with the covariance matrix $\mathcal{I}^{-1}$ for the coefficient estimates; the square roots of its diagonal elements are the variances you need for Wald tests of each coefficient. So you get Wald tests for free, almost, just by fitting a model; but likelihood-ratio tests require fitting a new model for each coefficient you want to test—with a large sample size & many predictors they'd take a good while longer to conduct. (This is also true more generally: if you're using observed information (the negative Hessian of the log likelihood) rather than expected information; or even if you're finding maximum likelihood estimates with an algorithm that doesn't involve calculating the Hessian, it's quicker to evaluate the Hessian numerically than to fit lots of models.)

If the point of logistic regression were to always test whether each & every coefficient is equal to zero, then there'd be an argument for statistical software's defaulting to the likelihood-ratio test when displaying a summary of the fitted model. But as that isn't always, or even often, the point—& especially as with some models many of the hypotheses tested may well be of no interest at all in general (see What value does one set, in general, for the null hypothesis of β0 in a linear regression model?)— it makes sense to provide the Wald tests & leave the analyst to choose which, if any, further tests to conduct, & what method to use.^† (It would also make sense to provide no tests, & force the analyst to think about which, if any, tests to conduct, &c.)

---

† I don't know of any R function to conduct LRTs for all coefficients of a model individually—it wouldn't be hard to write one—but both stats:::drop1 & car:::Anova conduct them for a default set of null hypotheses more likely to be of interest.

---

NB invariance to reparametrization means only that the LRT for, say, $H_0: \beta_7 =0$ is the same as the LRT for $H_0: \frac{1}{1+\mathrm{e}^{-\beta_7}}=1$ (which isn't the case for the Wald test). Replacing $\beta_7$ with $\log \beta_7$, on the other hand, would be fitting a substantively different model.