2

Let $P$ be an infinite partition of the interval $[0,1]$. Let $P$ have elements $I_i$ which has Lebesgue measure $m(I_i)$.

Then the entropy of $P$ is defined by $\sum_i -m(I_i)\log m(I_i)$. Can this be infinite?

leonbloy
  • 60,243
  • 9
  • 68
  • 146
a12345
  • 1,303
  • 10
  • 23
  • Is $m(I_i)$ the measure of the $i$th interval? – Semiclassical Jul 25 '14 at 16:59
  • yes, thanks for your comment – a12345 Jul 25 '14 at 17:03
  • Since $\sum m(I_i) = 1$, the question can be equivalently restated as follows: is there a probability distribution on $\Bbb N$ whose entropy is infinite. As far as I understand, the answer is no as follows from [here](http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution#Discrete_distributions_with_given_mean). – SBF Jul 25 '14 at 17:20
  • Thanks for the link. What is the "mean" that they are talking about? – a12345 Jul 25 '14 at 17:37
  • That I'm not sure I understand: the entropy of a discrete distribution does not depend on its underlying space, so you shall be able to assume the mean to be anything. – SBF Jul 25 '14 at 18:22
  • Related (practically dup) http://math.stackexchange.com/questions/279304 – leonbloy Jul 25 '14 at 18:38

1 Answers1

1

This paper provides an example of a distribution on $\Bbb N$ which has infinite entropy. Thus, in your case $P$ is any partition satisfying $$ m(I_i) = \frac{1}{\lg(i+1)} - \frac{1}{\lg(i+2)}, \qquad i\in \Bbb N, $$ where $\lg i$ is a logarithm base $2$. For example, $I_1 = [1,\frac1{\lg3})$, $I_2 = [\frac1{\lg 3},\frac12)$, $I_3 = [\frac12,\frac1{\lg5})$ etc.

SBF
  • 35,163
  • 4
  • 77
  • 154