3

Surprisingly I have not found any literature. I would like to know if it is possible to define a (generalized) mode for general probability distributions, possibly defined on an infinitely dimensional space.

My thought is that it should be related to $ P(B_\theta(r))$, where $B_\theta(r)$ is a ball centered at $\theta$ with radius $r$.

newbie
  • 151
  • 6
  • What are you doing to do with discrete distributions? (and beware, I did not say *discrete distributions on a lattice*...) – Glen_b Oct 12 '16 at 00:18
  • @Glen_b The prior I use is Gaussian prior on function spaces, usually its posterior is equivalent to the prior. The difficulty is that the function spaces seem to be too large for looking into each point, so I am thinking that one might tackle the problem using some concentration inequality, s.a. small ball probability. – newbie Oct 14 '16 at 11:54
  • Small-balls wont work for discrete or mixed distributions (since every atom of probability will become a mode). Since your question seems to be asking about probability distributions in general you'd need a definition that covers these cases. For example, consider a pmf $P(X=i)=2^{-\sqrt{i}}\,,\:i=1,4,9,16,25...$. ... What radius are you going to use? – Glen_b Oct 14 '16 at 21:14
  • @Glen_b Sorry about the ambiguity in the question, what I actually want to settle here is, for example, a Laplace distribution whose density function is set to be zero at zero, can we still define a concept of mode leading to the point zero, intuitively speaking, the small ball around it has the most mass. To be specific, I am interested in a class of measures (diffuse? if I remember correctly) that have no point mass. – newbie Oct 17 '16 at 11:48
  • Are you after the term *continuous*? – Glen_b Oct 17 '16 at 13:04

2 Answers2

1

I guess that practical solution to your problem is simply to use kernel density estimation, where the mode is highest point of the estimated kernel density. It uses kernels $K_\theta$ centered at $\theta$ instead of $B_\theta$. You can easily find a number of papers describing this approach.

Another practical comment is that due to the curse of dimensionality you wouldn't be able to compute such mode and wouldn't be able to obtain meaningful KDE estimates as the number of dimensions grow.

Community
  • 1
Tim
  • 108,699
  • 20
  • 212
  • 390
0

With a continuous distribution mode is usually defined as the point with maximum probability distribution function (fdp). This is closely related to your thougth if it's taken to the limit - that is, for an infinitely small ball.

If $\text{vol}$ means volume (in whichever dimensions), one possible definition of fdp is:

$$fdp(\theta)=\lim_{r\to0}\frac{P(B_\theta(r))}{\text{vol}(B_\theta(r))}$$

Then, one possible definition of mode for a continuous distribution is the point where $P(B_\theta(r))$ per volume unit is maximum.

Furthermore, for certain applications a naive but useful way to estimate the mode from a sample is to fix a ball radius and find the centre of the ball that contains more sample points. For example, geologists used to plot joint orientations in stereographic projection to find the orientations where most points clustered, and therefore estimate the most likely ways a rock slope could fall.

Pere
  • 5,875
  • 1
  • 13
  • 29