0

I have inherited some logistic regression analysis, and am trying to understand the approach that was taken.

In the generalized linear model, the response is binary (positive or negative) and so are all six potential predictors.

The previous analyst was interested in seeing the probabilities of a positive response for each predictor and combination of predictors. The approach they took was to build all possible models with the six predictors (so 63 in total), then for each model convert any coefficients (including for the intercept) to probabilities using the formula EXP(coefficient))/(1+EXP(coefficent)). They then sum these probabilities for each model to give a total probability of getting a positive response for that combination of predictor(s). They then present this as a list, showing which combinations have the highest and lowest total probabilities.

I've not come across this approach before, and am confused as to whether it is appropriate. I can see that two of the models (one with six predictors and one with five) have much lower AIC scores than the others, so I thought it would be best to use these models rather than all 63. If I was starting from scratch and aiming to get similar outputs I would produce predictions for each combination of predictors, probably using model averaging across the two best models. I would also specify se.fit=TRUE in my predictions, so I could add confidence intervals around the probabilities.

Which approach sounds most appropriate? The previous analysis has been in use for some time, so if I'm to change things I would need to be able to explain properly what is better about my new approach.

rw2
  • 1,009
  • 5
  • 18
  • 1
    You'd be adding up the probability outputs of 63 different logit models? (I think you mean to convert the log-odds output to a probability, not convert a coefficient.) – Dave Sep 21 '20 at 19:25
  • @Dave - adding up the outputs of 63 different models is what is done in the current approach. My thinking was that it would be best to just use the best model(s)... ? (I thought the coefficients (logits) were converted to odds using the exp function, and then to probabilities using odds / (1 + odds)) – rw2 Sep 22 '20 at 08:30

1 Answers1

3

Adding up all of those probabilities sounds bizarre. It does not even estimate what you want. If you want to estimate the probability for a given combination of predictors, simply use the model with all of those predictors, plug in the combination of interest, and get the probability. Just one model. If you choose the lowest AIC approach, that might exclude a predictor that you want, so if you have sufficient sample size and there are no horrific collinearities or other problems, just use them all.

BigBendRegion
  • 4,593
  • 12
  • 22
  • 1
    This is particularly true for a logistic regression model, which has an inherent [bias if predictors associated with outcome are omitted](https://stats.stackexchange.com/q/113766/28500). You don't want to overfit, but you do want to keep as many predictors as reasonable in such a model. (+1) – EdM Sep 22 '20 at 15:43
  • Thanks - I thought it seemed bizarre too, but wanted to check. The lowest AIC model has all six predictors, so I'll just use that one. I have sufficient sample size, but not sure how to check for collinearities when all the predictors are binary? – rw2 Sep 22 '20 at 15:51
  • The same way you do ordinarily - just look at the correlation matrix of the X variables. – BigBendRegion Sep 22 '20 at 21:51
  • It occurs to me that maybe the original idea was supposed to be to average and not to sum, thus getting some kind of "model average" estimates? In any event, as EdM mentioned, there some major biases here in using all the models. Maybe not so bad with a select few though. – BigBendRegion Sep 22 '20 at 21:55
  • So it turns out that they want probabilities for e.g. "predictor1" regardless of the other predictors. If using the full model to calculate the probability of a positive response when a subject is positive for predictor1, you also have to specify what the other predictors are, but they don't want to. Is there a better way to get what they want? – rw2 Sep 23 '20 at 10:09
  • Hmmm ... "regardless of other predictors" is somewhat ambiguous. You could just use the regression for predictor 1 (ignoring all other predictors) and be done, but that would implicitly average over the values of all the other predictors corresponding to whatever the data collection method is used. Or you could be more explicit in your averaging process by fitting the full model, getting the probabilities as a function of all the predictors, then average over the others in whatever way you want. – BigBendRegion Sep 23 '20 at 19:12