11

From some machine learning notes talking about some discriminative classification methods, in particular logistic regression, where y is the class label (0 or 1) and x is the data, it is said that:

if $x|y = 0 \sim \mathrm{Poisson}(λ_0)$, and $x|y = 1 \sim \mathrm{Poisson}(λ_1)$, then $p(y|x)$ will be logistic.

Why is this true?

Silverfish
  • 20,678
  • 23
  • 92
  • 180
sambajetson
  • 417
  • 2
  • 7

1 Answers1

16

$Y$ has two possible values for any given value of $X$. According to the assumptions,

$$\Pr(X=x|Y=0) = \exp(-\lambda_0) \frac{\lambda_0^x}{x!}$$

and

$$\Pr(X=x|Y=1) = \exp(-\lambda_1) \frac{\lambda_1^x}{x!}.$$

Therefore (this is a trivial case of Bayes' Theorem) the chance that $Y=1$ conditional on $X=x$ is the relative probability of the latter, namely

$$\Pr(Y=1|X=x) = \frac{\exp(-\lambda_1) \frac{\lambda_1^x}{x!}}{\exp(-\lambda_1) \frac{\lambda_1^x}{x!} + \exp(-\lambda_0) \frac{\lambda_0^x}{x!}}= \frac{1}{1 + \exp(\beta_0 + \beta_1 x)}$$

where

$$\beta_0 = \lambda_1 - \lambda_0$$

and

$$\beta_1 = -\log(\lambda_1/\lambda_0).$$

That indeed is the standard logistic regression model.

whuber
  • 281,159
  • 54
  • 637
  • 1,101