I have a binary classification problem (i.e. is observation positive or negative) and I'm interested in what information I can obtain about observations in my test set. I don't care about the model parameters or observations in my training set. I have plenty of data and would just use an uninformative prior as the task is fairly simple.
With a frequentist logistic regression I can only get a probability estimate of whether the observation is positive/negative, but from what I've heard Bayesian methods can give more information such as an uncertainty estimate. With regards to this I have the following questions:
How does the Bayesian method obtain this extra information? Can you refer to any material on this?
What extra information can be obtained at prediction time? Can one get more information than an uncertainty estimate?