Given a measurement that (we know from theory) has a finite expected value, but infinite variance, is it possible to have some statistical information? Can I get the significance of such data? Can methods like the bootstrap or Jackknife be useful to put a bound on the variance?
Asked
Active
Viewed 969 times
3
-
5Your question is somewhat unclear. What is it you intend to actually do by comparing a random variable to measurements? Are you checking whether the measurements follow some distribution with infinite variance? Are you comparing sample mean with some hypothesized mean, or are you asking about something else? When you say "put some bound on the variance", what other variance are you attempting to bound? – Glen_b Feb 20 '16 at 11:55
2 Answers
2
Among many others, Holger Rootzen has noted that once in the presence of distributions with infinite moments, classic statistical analysis is no longer relevant and extreme value theory and analysis must be substituted.
https://lup.lub.lu.se/search/person/mats-hro
Power laws and other extreme value distributions follow a general (p-1) rule in terms of the existence of the moments, where p is the tail exponent or index. As noted in this article,
When 1 < p < 2, the first moment (the mean or average) is infinite, along with all the higher moments. When 2 < p < 3, the first moment is finite, but the second (the variance) and higher moments are infinite
http://tuvalu.santafe.edu/~aaronc/courses/7000/csci7000-001_2011_L2.pdf
This Wiki discussion of Tweedie distributions summarizes the resulting assignments nicely:
The Tweedie distributions ... (are) ... specified by the domain of the tail index parameter, p:
- normal distribution, p = 0,
- Poisson distribution, p = 1,
- Compound Poisson–gamma distribution, 1 < p < 2,
- Gamma distribution, p = 2,
- Positive stable distributions, 2 < p < 3,
- Inverse Gaussian distribution, p = 3,
- Positive stable distributions, p > 3,
- and extreme stable distributions, p = ∞
- For 0 < p < 1 no Tweedie model exists
https://en.wikipedia.org/wiki/Tweedie_distribution
Infinite moments, by definition, are boundless in the population (e.g., the theoretical maximum or supremum) although they will exist in finite data samples. However, the moments of finite data samples from those distributions with infinite moments will be highly unstable and unreliable, easily fooling the unsuspecting regarding their existence.
Gabaix has a good discussion of power laws in finance that reviews many broadly generalizable, invariant mathematical properties such as scale proportionality, fit to many phenomena in nature and business, fractal geometry, use in risk management, etc.
http://pages.stern.nyu.edu/~xgabaix/papers/pl-ar.pdf
There are also many excellent applications of EVT in the real world. Examples include Dutch construction of dyke and levee seawalls against North Sea storm surges to heights protecting them against 1 in 100,000 year events
https://en.wikipedia.org/wiki/Flood_control_in_the_Netherlands
Arthur de Vany's book, Hollywood Economics: How uncertainty shapes the film industry has a rigorous analysis of extreme valued box office revenues.
Now 20 years old, Embrechts, et al's book, Modeling Extremal Events remains a classic, covering all of the essential concepts in EVT
Another, more introductory resource is Kotz' Extreme Value Distributions: Theory and Applications
Basically, there is a wealth of information out there on the "mathematical and statistical information" that can be brought to bear in analyzing EVT distributions.
Mike Hunter
- 9,682
- 2
- 20
- 43
-
1My answer to this question is relevant: http://stats.stackexchange.com/questions/94402/what-is-the-difference-between-finite-and-infinite-variance/100161#100161 – kjetil b halvorsen Feb 20 '16 at 20:01
-
0
Obviously you are interested in testing hypotheses about the expectation value as this is the only parameter you mentioned.
Bootstrap and similar procedures like Jackknife will not work if the variance is infinite. Bootstrap samples always look as if they had finite variance: As your (finite) data sets always have a finite range, the variance must be finite, too. In fact, Mammen (1992) showed, that bootstrap works iff asymptotic normality would work as well. The latter however needs finite variances.
I suggest reading more about nonparametric techniques, e.g. relying on the ranks (the ordering of observed values). Of course, you will loose all the metric information and your assertions will rather be about the median than the expectation. But using appropriate models, you can still infer about the expectation.
More details depend on you either supplying much more information about your particular application or reading more on your own.
Horst Grünbusch
- 5,020
- 17
- 22