Using *EDIT* non-informative priors to account for perfect separation in logistic regression?

Question

In generating logistic regressions for treatment-survival data, perfect separation is a problem in a few of my data sets. I've decided to use a Bayesian approach to account for the perfect separation Sources: (Gelman 2008) and this question. This approach also allows me to easily make predictions from the regression using a developed function in R.

My understanding of this Bayesian approach is limited. I can set the prior df to an arbitrary value, increasing it until I don't have a problem fitting the model due to perfect separation, but I don't understand what this regularization is effectively doing.

From my understanding, using informative priors assumes I know something about the data and I'm including it as a starting point for model fitting. A weakly informative prior doesn't supply any controversial information but can pull data away from incorrect estimations. I think the latter is more aligned with my proposed method (Source). I've read this but I do not have any prior information on which to base prior distributions or df.

I know this is an oversimplified understanding which leads to my question:

What information am I providing to the model when I arbitrarily set the prior df (in R prior.df)at a level that allows for the model to converge?

Answer 1

I do not know what you mean by "df" and how this parameter (or whatever it is) influences the prior you are contemplating to use.

One common approach, if you truly just have binomial/binary data is to use Firth's penalized likelihood regression, which corresponds to maximum-a-posteriori estimation with a relatively vague prior (Jeffreys' prior). It is pretty vague unless you have super-sparse data, in that case you should especially not pay too much attention to the point estimates (and rather observe that usually CIs are extremely wide). This is implemented in a lot of software packages. It is usually sufficient to ensure convergence for a MAP estimate (not quite sure, if there are not any events at all). You can do more or less the same thing with a other Bayesian approaches - essentially as long as you specify a proper distribution as the prior, the posterior is a proper distribution (but may occasionally be hard to sample from).

Depending on what setting you are working on, there may be previous experiments that help you say what you would expect in your reference category see and you can always make the prior more vague or long-tailed, if there are questions as to the applicability of the prior. Similarly, you may know or believe that treatment effects in your research field rarely exceed a certain odds ratio (or other effect measure, just using this example, because you mentioned logistic regression), say, you may believe that true odds ratios outside of [0.1, 10] (or log-odds ratios outside of [log(0.1), log(10)]) are rare, which might make you consider e.g. a long-tailed Cauchy(0, 0.37) prior for your regression coefficients other than the intercept. See also the Stan prior Choice Recommendation Wiki.

The other thing I wondered about is whether your data is truly binomial and should be looked at using logistic regression, "treatment-survival data" sounds like there is a time component and ignoring that may cause issues (or not depending on your setting). Even if you do not have individual patient records with exact times, there may be methods for taking this into account.