3

I'm trying to derive which univariate probability distribution maximizes entropy, assuming finite mean $\mu$ and non-negative support $[0, \infty)$. I know that the answer is the exponential distribution, but I'm struggling to get there.

I start by defining the Lagrangian:

$L(p(x), \lambda_1, \lambda_2) = H[p(x)] + \lambda_1(\int_0^{\infty} p(x) dx - 1) + \lambda_2(\int_0^{\infty} x p(x) dx - \mu)$

Taking the functional derivative with respect to $p(x)$, I find that:

$\log p(x) = -1 + \lambda_1 + \lambda_2 x$

I should be able to solve for $\lambda_1, \lambda_2$ using the two constraints, but I haven't been able to. My most promising attempt involves setting the mean constraint equal to $\mu$ times the normalization constraint:

$\mu \int_0^{\infty} p(x) dx = \int_0^{\infty} x p(x) dx$

But this leaves me with a problem that I don't know how to solve:

$\mu \int_0^{\infty} e^{\lambda_2 x} dx = \int_0^{\infty} x e^{\lambda_2 x} dx$

Sycorax
  • 76,417
  • 20
  • 189
  • 313
Rylan Schaeffer
  • 689
  • 3
  • 10
  • Evaluate the integrals in terms of $\lambda_2.$ – whuber May 13 '19 at 17:25
  • I'm afraid I don't understand – Rylan Schaeffer May 13 '19 at 20:40
  • The integrals are $\int_0^\infty e^{\lambda_2 x}dx$ and $\int_0^\infty x e^{\lambda_2 x}dx.$ Both are numbers that depend on $\lambda_2$ and they are found with a truly elementary calculation. Do that calculation and replace the integrals by what you obtain, then solve. – whuber May 13 '19 at 21:36
  • Can you be less vague? The first integral is equivalent to $\frac{1}{\lambda_2}e^{\lambda_2 x} |_{x=0}^{x=\infty}$, and I can integrate the second integral using integration parts, but I don't see what to do after that. – Rylan Schaeffer May 13 '19 at 23:21
  • 1
    I have been quite specific. Your problem is that you haven't evaluated the integrals correctly. Well, "correctly" isn't quite right--but your expressions aren't *useful.* Get rid of the "$x.$" – whuber May 13 '19 at 23:26
  • I understand that, but I don't understand how to do so. – Rylan Schaeffer May 13 '19 at 23:31

0 Answers0