0

The maximum entropy probability distribution has entropy at least as great as that of all other members of a specified class of probability distributions (pdf's). Does that mean that the pdf with maximum entropy can only be determined by comparing the entropy values of pdfs 1, 2, 3, 4 and 5 side by side to figure out that pdf 3 has the highest entropy?

Whereas, in the case you are only given pdf 1, you have no evidence to say that it has the maximum entropy of its class?

develarist
  • 3,009
  • 8
  • 31
  • I'm unable to make sense of your question. By "class" I understand you to mean a well-defined set of distributions. When you are given a class and a particular pdf, in what sense do "you have no evidence" to establish that the entropy of that pdf is not exceeded by any of the distributions in the class?? – whuber Jul 29 '20 at 13:38
  • In case 2, say instead you have 1 ceramic bowl in your room. it's not the only ceramic bowl in your house though, so you cannot say that it is the bowl that fits the largest volume since you do not have all the other bowls to compare with in your room... In case 1, on the other hand, you have 5 ceramic bowls with you, and they are all the bowls that exist in your house. Since they all have different widths and depths though, you cannot say which has the largest volume until you take a measurement of each by filling them. by doing so, you acquire evidence that bowl 1 has the largest volume. – develarist Jul 29 '20 at 13:48
  • 1
    Typically, the maximization is done using some mathematical technique like calculus or calculus of variations. For some examples see https://stats.stackexchange.com/questions/361476/what-is-the-maximum-entropy-distribution-given-the-median-instead-of-the-mean/361796#361796, https://stats.stackexchange.com/questions/408125/maximum-entropy-probability-distribution-over-non-negative-support-and-finite-me, https://stats.stackexchange.com/questions/404097/proving-that-shannon-entropy-is-maximised-for-the-uniform-distribution – kjetil b halvorsen Jul 29 '20 at 20:15
  • so the maximization is done on only one distribution, or must be done on all competing distributions within that diatribution's class to ensure that it in fact has the global maximum entropy compared to its competitors? – develarist Jul 30 '20 at 18:04
  • Did you read the linked threads? – kjetil b halvorsen Aug 10 '20 at 22:07

0 Answers0