1

I'm doing survival analysis with Cox PH. I have my final model based on averaged models and I have four categorical variables with multiple levels each. I computed the fitted values using predict function from R, varying the values of one variable and maintaining the values of the other variables in the most common level (if I had continuous variables it would be the mean level).

My question is, with this predicted values, and having multiple levels for each variable, can I trust the confidence intervals?

dimitriy
  • 31,081
  • 5
  • 63
  • 138
JMarcelino
  • 291
  • 1
  • 12
  • Is there some reason why you think that the confidence intervals shouldn't be trustworthy? And when you say "trustworthy," for what purpose? – EdM Nov 14 '14 at 19:11
  • My model average results do not show p-value results comparing each level with each other. Only the baseline level with the others, turning most of all not significant. I think thesame thing could be happening with the fitted values, since I am comparing all levels of one variable with the most frequent level of the other variables, and not the mean – JMarcelino Nov 17 '14 at 09:19
  • Is the final model significant overall? Look at the likelihood ratio test, Wald test, score test in the output from `summary(yourFinalModel)`. Also, it's not clear what you mean by "based on averaged models" as the basis for your final model. – EdM Nov 17 '14 at 21:28
  • I did model averaging, ie, my final model is a group of 7 models. I do not have a likelihood ratio test, Wald test or score test. – JMarcelino Nov 18 '14 at 09:47
  • So how do you know that your averaged model is better than no model at all? What process/function did you use to average the models? – EdM Nov 18 '14 at 15:47
  • I know that AIC weights sum up to 0.99 and indicate the relative likelihood of the model is the most appropriate. While the first model have a weight of 0.39. I used model.avg from the MuMIn package of R. – JMarcelino Nov 18 '14 at 19:06

1 Answers1

1

If you used the predict function on an object returned by model.avg, I don't see a reason for the confidence intervals to be technically untrustworthy. (There can be substantial issues arising from how the models were chosen for averaging in the first place, but I assume that you are aware of those.)

Provided that the proportional hazards assumption holds true, and that there are no substantial interactions among your predictors with respect to survival, it shouldn't matter that you have multiple levels for your categorical variables. If the assumptions are met then the residual error terms used to calculate confidence intervals should be the same regardless of your choices of reference values, even as relative hazard changes with the choice of reference. Whether a particular prediction case is "significantly" different from the reference case, based on the confidence intervals calculated from the residual errors, of course depends on the reference used.

You might want to consult the very useful page Prediction in Cox regression, listed as Related on this page, for other approaches to the model selection/prediction problem, issues of relative and absolute hazards, etc.

Community
  • 1
EdM
  • 57,766
  • 7
  • 66
  • 187