6

I've read here that

... (Bayesian linear regression) is most similar to Bayesian inference in logistic regression, but in some ways logistic regression is even simpler, because there is no variance term to estimate, only the regression parameters.

Why is it the case, why no variance term in Bayesian logistic regression?

Krantz
  • 556
  • 2
  • 18
Patrick
  • 393
  • 1
  • 9

1 Answers1

10

Logistic regression, Bayesian or not, is a model defined in terms of Bernoulli distribution. The distribution is parametrized by "probability of success" $p$ with mean $p$ and variance $p(1-p)$, i.e. the variance directly follows from the mean. So there is no "separate" variance term, this is what the quote seems to say.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • @patrick for linear regression $y = mx + c + \epsilon$, whereas logistic regression p(y=1|x) = logistic(mx +c). – seanv507 Apr 03 '19 at 21:02
  • @seanv507 and would it make sense to have $p(y=1|x)=logistic(mx+c+\epsilon)$ or not? If not, is it because $p()$ is a probability and already includes some uncertainty? – Patrick Apr 03 '19 at 21:37
  • 1
    @Patrick what would this formulation exactly mean? Could you give an example where would you imagine it to be used? – Tim Apr 03 '19 at 21:39
  • @Patrick When you describe the conditional expectation in terms of a distribution there's no error term. – Firebug Apr 03 '19 at 22:12
  • Not really, I was just comparing the linear and the logistic descriptions, first has $\epsilon$, the second one has not. I foolishly believed that we can have an error term in both. @Firebug By distribution you mean the logistic right (which is a cumulative distribution)? – Patrick Apr 04 '19 at 18:16
  • 1
    @Patrick linear regression is $Y \sim \mathcal{N}(X\beta, \sigma)$ while logistic regression is $Y \sim \mathcal{B}(h^{-1}(X\beta))$. Error term is alternative way of writing linear regression, but in both cases you estimate mean of some distribution. So error term *inside* logistic regression would mean that you assume $h(p)$ to be normally distributed. This does not have to be nonsense, but I doubt that this is what you meant. – Tim Apr 04 '19 at 19:12
  • You're right. Thank you again, it's very clear! Don't you think the author meant that there is no error term inside logistic regression (in general)? – Patrick Apr 04 '19 at 20:06