In statistics, different orders of moments are tools to characterize a distribution, for example mean, covariance, skewness etc., which also gives an intuitive way to visualize the distribution.
But are there any math tools to account for the "multi-modality" of a distribution?
For example, given a (multimodal) distribution, how to define the information of the modality?
Wikipedia gives several summary statistics for bimodality. I will give some useful examples:
Reminiscent of a proposal by Pearson's, it builds on the idea that bimodal distributions present low kurtosis, high skewness, or both at the same time. $\gamma$ is the skewness while $\kappa$ is kurtosis. $\beta \in [0,1]$. $\beta = 5/9$ for uniform and exponential distributions. Values greater than that indicate bimodality.
$$ \beta = \frac{\gamma^2+1}{\kappa} $$
$D$ measures the degree of separation between two Gaussian components. $D>2$ is an indicator of marked separation between the distributions. You can use it if you have the probability distribution function or if you can model your samples with a bimodal Gaussian mixture.
$$D=\sqrt2\frac{|\mu_1-\mu_2|}{\sqrt{\sigma_1^2+\sigma_2^2}}$$
$A$ can be used to summarize bimodality directly from the samples' histogram. $S$ is the number of categories with non-zero counts, while $K$ is the total number of categories. $U$ is a binary measure of unimodality, and is only equal to one if there's equidistribution of samples across one or more category. $A=-1$ suggests bimodality while $A=1$ indicates unimodality.
$$A = U\left(1-\frac{K-1}{S-1}\right)$$
Two unsupervised learning algorithms come to mind that can help derive information of the individual components of a multimodal distribution. They isolate the individual unimodal densities within a multimodal distribution, and from there, the information and statistics of the isolated unimodal densities can be evaluated independently as you would normally do, without the parent multimodal distribution's complexity getting in the way.
GMM employs clustering in order to identify individual components (child unimodal distributions) from within the parent multimodal distribution. It does this by clustering the two components' samples distinctly from one another via expectation-maximization optimization that iteratively settles on estimates of the mean and variance for each individual component distribution. GMM works best if the components are Gaussian, but is generalized in practice to almost any bell-shaped density.
This measure comes from information theory and exhibits mode-seeking behavior, whereas forward KL-divergence has difficulty detecting modes in a multimodal distribution. By fitting the reverse KL-divergence on a multimodal density, it will eventually identify one of the modes, effectively isolating that mode separately from the parent distribution, so that it can replicate that mode's density as a prediction model for individual analysis independent from the parent distribution.
