2

I am estimating optimism bias in for example risk predictions. A method for doing that is described by Frank Harrell and implemented in the R package rms. I am trying to reproduce the bias-corrected calibration curve in SAS. However I am getting stuck and here is what I need help to understand:

After I have taken a bootstrap sample and fitted a binary logistic regression model to that sample I need to estimate the "actual" risk at a series of levels of predicted risk on this bootstrap sample. Now this is done with loess regression. From this we can obtain the distance from the "actual" risk at specific point of predicted risk to the 45 degree line.

the next step is to use the bootstrap-fitted-model on the original sample. In this case, I am wondering if I need to perform yet another loess regression given the new predictions?

The reason why I am asking is that if for example we obtain a predicted risk of 9.0% for a subject on the bootstrap sample but 9.19923% on the original sample we may not have an "loess-estimate" at that particular point because the loess was performed on the bootstrap sample. Maybe a function is fitted and used the first time loess regression was performed?

(Please also correct me if I have completely misunderstood something)

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
Danny
  • 805
  • 5
  • 17

1 Answers1

2

Think of a calibration curve as an estimate of the true risk when the predicted risk is 0.01, 0.02, ..., 0.99. Obtain this with a flexible nonlinear smoother. A regression spline works best, followed by loess. When using loess use linear interpolation to obtain estimates at 0.01, ..., 0.99 so that you always use the same grid of values for every bootstrap. Or just use R which provides much more output than just this.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322