What is the frequentist interpretation of uncertainty vs. variability?

Question

I have been reading Begg, Welsh, and Bratvold (2014), which is an excellent and lucid discussion of the distinction between uncertainty and variability (from a petro/geostatistics perspective). The abstract defines them:

"Uncertainty means we do not know the value (or outcome) of some quantity, ... Variability refers to the multiple values a quantity has at different locations, times or instances"

and describes how they are captured:

"Uncertianty [sic] is quantified by a probability distribution which depends upon our state of information about the likelihood of what the single, true value of the uncertain quantity is. Variability is quantified by a distribution of frequencies of multiple instances of the quantity, derived from observed data.

This makes sense to me. Variability within a population is defined by some frequency mass function (discrete case) or distribution function (continuous case). If we have perfect information about the whole population, there is no uncertainty, and these functions can be exactly specified. If we do NOT have perfect information about the population (e.g. we only have access to a limited sample, then there is some uncertainty about these functions, which can be described as probability distributions on the estimated frequencies or distribution parameters.

My understanding is that this population-level variability collapses to uncertainty when we ask "what is the true value of a particular (unmeasured) element of the population?". There is a semantic distinction between the two sources of uncertainty here - the uncertainty on the population variability is epistemic uncertainty, and the contribution of population variability to sample uncertainty is aleatoric uncertainty.

Although it is not explicitly framed as such, this strikes me as an very Bayesian interpretation of probability, as they describe it as solely interpretable as a personal measure of belief in the truth value of a given well-defined statement.

My understanding of the Frequentist a interpretation of probability is that it is a representation of the true proportion of any given statement, if the whole population could be measured (or the limit of the mean of sample estimates as the number of samples goes to infinity).

I can kind of see how this makes sense (though not as much as the bayesian interpretation), but I'm finding it hard to make a clear distinction between uncertainty and variability in this framework provided by Begg, Welsh, and Bratvold (2014). I think that frequentist probability actually represents population variability here, but in that case, what does uncertainty represent? Just the potential wrongness of sample estimates? Or something else? And how is it quantified? By confidence interval widths? I feel like I am missing some nuance here.

References

• Begg, Steve H., Matthew B. Welsh, and Reidar B. Bratvold. “Uncertainty vs. Variability: What’s the Difference and Why Is It Important?” In Day 1 Mon, May 19, 2014, D011S003R002. Houston, Texas: SPE, 2014. https://doi.org/10.2118/169850-MS.

Answer 1

Uncertainty means we do not know the value (or outcome) of some quantity, eg the average porosity of a specific reservoir (or the porosity of a core-sized piece of rock at some point within the reservoir). Variability refers to the multiple values a quantity has at different locations, times or instances – eg the average porosities of a collection of different reservoirs (or the range of core-plugs porosities at different locations within a specific reservoir).

I think the uncertainty amounts to the epistemic uncertainty(or model uncertainty; caused by parameter uncertainty, and can be reduced by providing more data), and the variability the aleatoric uncertainty(or data uncertainty; inherent uncertainty in the data, but cannot be reduced by providing more data). And their difference is illustrated in this answer.

The frequentist models can only capture the aleatoric uncertainty while the Bayesian both, then the frequentist interpretation of (epistemic) uncertainty would be of no meaning.

The aleatoric uncertainty is expressed in the distribution across the classes, which is zero if one class gets a probability of one. The epistemic uncertainty is expressed in the spread of the predicted probabilities of one class, which is zero if the spread is zero. The non-Bayesian NN can’t express epistemic uncertainty (you can’t get different predictions for the same image), but the BNN can.

The above is a quote from this book: Probabilistic Deep Learning: with Python, Keras and Tensorflow Probability

And how is it quantified?

The aleatoric uncertainty can be measured using entropy, and the spistemic uncertainty can also be quantified as illustrated in 8.5.2 of this book: Probabilistic Deep Learning: with Python, Keras and Tensorflow Probability.

Answer 2

Short answer: Any uncertainty (confidence) we have surrounding an unknown population quantity is due to the sampling variability of the estimation or testing procedure that uses a limited sample size.

Full answer: To the frequentist, population-level quantities (typically denoted by greek characters) are fixed and unknown because we are unable to sample the entire population. If we could sample the entire population we would know the population-level quantity of interest. In practice we have a limited sample from the population, and the only thing one can objectively describe is the operating characteristics of an estimation and testing procedure. Understanding the long-run performance of the estimation and testing procedure is what gives the frequentist confidence in the conclusions drawn from a single experimental result. What the experimenter or anyone else subjectively believes before or after the experiment is irrelevant since this belief is not evidence of anything. Beliefs and opinions are not facts. If the frequentist has historical data ("prior knowledge") this can be incorporated in a meta-analysis through the likelihood and does not require the use of belief probabilities regarding parameters. If fixed population quantities are treated as random variables this can introduce bias in estimation and inference. I find confidence curves to be a particularly useful way to visualize frequentist inference, analogous to Bayesian posterior distributions.