Consider the problem of estimating the number of discoverable extra-terrestrial civilizations (in our galaxy, say), or the related problem of estimating the probability of discovering such a civilization in some given time frame.
The proper way to make such estimations is via Bayesian statistics, where you have prior information about the relevant parameters (number of stars, number of inhabitable planets per star, probability they emit detectable radio signals, etc.) and then revise such probabilities as new data are collected (change in number of inhabitable planets, negative results of radio searches, etc.).
Such prior information is generally presented as the Drake Equation, in which such relevant parameters are multiplied together to get the number of potentially discoverable extra-terrestrial civilizations. It can be extended, given some measure of the search effort, volume of space that can be monitored, etc., to yield the probability of finding such extra-terrestrial life.
I have heard astrobiologists at scientific conferences and numerous popularizers on TV (e.g., Neal deGrasse Tyson) quote enormously large numbers for the potentially discoverable civilizations, and how given the short (~50-year) effort of SETI (Search for Extra-Terrestrial Intelligence) searches, the negative results are "like dipping a tea cup into the ocean, finding no whales and erroneously concluding that there are no whales in the ocean."
But is this true?
I have not seen the proper Bayesian statistical method applied to this problem.
Some questions (astronomical but primarily statistical)
- Is there a recent peer-reviewed scientific/statistical paper where the proper Bayesian estimation methods have been applied to this problem? (This 30-year-old paper is the most relevant one I've seen, so far.)
- As astronomers find more and more candidate planets, does that mean to a statistician there is an increase the estimated probability we'll find extra-terrestrial life or decrease that probability, given that decades of negative results then imply life is not discovered on a larger number of planets? (Of course one must make some assumptions about coverage, and so on to answer this.)
- If, as some SETI workers claim, our negative search results don't significantly lower the prior estimation given by the Drake Equation (see the "whale" discussion, above), then will a statistician conclude that doubling, or even increasing our search effort by one or two orders of magnitude similarly not matter, i.e., not change our estimates significantly?
- Given our current estimates for the terms in the Drake Equation and some reasonable extrapolation of search effort (and hence search volume of space and duration of civilization), how long would would a statistician conclude we need to search and at what effort such that the continued negative results would imply the probability of ever finding extra-terrestrial life is negligibly small, say $P \leq 10^{-5}$?