8

The theory behind profile likelihood (PL) confidence intervals (CIs) is clear to me. (See here, for example).

SAS is surprisingly quick in calculating the PL CIs for all the covariates in a given model. Therefore I was wondering if there is another way of calculating PL CIs that does not require the time-consuming loop of maximizing the likelihood (over the nuisance parameters) given different fixed values of the parameter of interest.

Example: I run a logistic regression in SAS on ~43000 subjects with 10 covariates (+ the intercept). It took 0.6 seconds to fit this model without asking for PL CIs, and only 3 seconds when I required also the PL CIs for all the 10 covariates (option clodds=pl in proc logisitc). In Stata, the same model was fit in 0.35 seconds (without PL CIs) and in 10 seconds when asking the PL CIs for just one of the 10 covariates in the model (I used the pllf command, which iteratively fixes values of the parameter of interest and maximizes the likelihood over the remaining ones).

boscovich
  • 1,606
  • 1
  • 11
  • 21

1 Answers1

8

Very good question.

In this link it is explained that SAS uses a numerical approximation (which basically consists of a modification of the Newton-Raphson algorithm)

Setting this option to both produces two sets of CL, based on the Wald test and on the profile-likelihood approach. (Venzon, D. J. and Moolgavkar, S. H. (1988), “A Method for Computing Profile-Likelihood Based Confidence Intervals,” Applied Statistics, 37, 87–94.)

The link to the paper in JSTOR is here, and the abstract is shown below

The method of constructing confidence regions based on the generalised likelihood ratio statistic is well known for parameter vectors. A similar construction of a confidence interval for a single entry of a vector can be implemented by repeatedly maximising over the other parameters. We present an algorithm for finding these confidence interval endpoints that requires less computation. It employs a modified Newton-Raphson iteration to solve a system of equations that defines the endpoints.

According to this abstract, it seems like this is the secret of the speedy calculation.

Macallan
  • 571
  • 1
  • 3
  • 3
  • 1
    (+1) for the link and reference. Since the link is to a blog that discusses the `confint` function in R, I want to mention that profiling in R (based on `confint.profile.glm` from the MASS package) is a combination of evaluation of the profile likelihood and a spline interpolation. – NRH Apr 12 '13 at 09:04
  • @NRH Thank you for this. A link to the SAS manual might be more credible but I could not find it. Interesting combination the one used by R. – Macallan Apr 12 '13 at 09:12
  • Thank you for your answer, @Macallan. I've found some additional information in the SAS/IML User's Guide: http://support.sas.com/documentation/cdl/en/imlug/65547/HTML/default/viewer.htm#imlug_nonlinearoptexpls_sect034.htm and in the SAS/STAT User's Guide: http://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_phreg_a0000000688.htm - In addition, the paper "Efficient Profile-Likelihood Confidence Intervals for Capture–Recapture Models" by O.Gimenez et al. contains interesting information. – boscovich Apr 13 '13 at 08:46
  • 1
    I've read Venzon's paper and I'll see if I can implement his method in Stata, because this is exactly what I need (any suggestions are more than welcome). Interestingly, the author of the `pllf` Stata command (P. Royston) writes in the paper presenting his command that _"Venzon and Moolgavkar (1988) inspired this article [...]"_ (The Stata Journal, 7, Number 3, pp. 376–387). – boscovich Apr 13 '13 at 09:05