Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [uncertainty]

A broad concept concerning lack of knowledge, especially the absence or imprecision of quantitative information about a process or population of interest.

Uncertainty is a broad concept referring to lack of knowledge and ways to model that lack (such as with probability distributions of model parameters), as well as quantitative evidence thereof, such as measures of variation or dispersion in data.

In one specialized field, uncertainty is defined by the BIPM Guide to the Expression of Uncertainties in Measurement as:

parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand

Uncertainty should not be confused with error. The error is the actual difference between the measured value and the true value. The error is usually not known; if it were known, it could be corrected. The uncertainty is rather an estimate of the statistical distribution of errors around the true value. For example, one might repeat a measurement 100 times and the 100 measurements will have mean a certain mean and standard deviation. The mean could be the reported measurement, and the standard deviation the reported measurement uncertainty.

450 questions
69
votes
4 answers

Assumptions regarding bootstrap estimates of uncertainty

I appreciate the usefulness of the bootstrap in obtaining uncertainty estimates, but one thing that's always bothered me about it is that the distribution corresponding to those estimates is the distribution defined by the sample. In general, it…
user4733
  • 2,494
  • 2
  • 20
  • 31
47
votes
6 answers

Why is softmax output not a good uncertainty measure for Deep Learning models?

I've been working with Convolutional Neural Networks (CNNs) for some time now, mostly on image data for semantic segmentation/instance segmentation. I've often visualized the softmax of the network output as a "heat map" to see how high per pixel…
Honeybear
  • 599
  • 1
  • 6
  • 8
19
votes
1 answer

Can I convert a covariance matrix into uncertainties for variables?

I have a GPS unit that outputs a noise measurement via covariance matrix $\Sigma$: $\Sigma = \left[\begin{matrix} \sigma_{xx} & \sigma_{xy} & \sigma_{xz} \\ \sigma_{yx} & \sigma_{yy} & \sigma_{yz} \\ \sigma_{xz} & \sigma_{yz} & \sigma_{zz}…
Dang Khoa
  • 361
  • 1
  • 2
  • 6
18
votes
3 answers

Finding most likely permutation

[Hoping that this is the right Stackexchange site; inspired from a true story seen at work] Joe has a measuring instrument and $n$ objects to be measured (say, a scale and $n$ weights). He measures each one, obtaining a list of measurements…
GioMott
  • 291
  • 1
  • 5
13
votes
4 answers

Kernel density estimation incorporating uncertainties

When visualising one-dimensional data it's common to use the Kernel Density Estimation technique to account for improperly chosen bin widths. When my one-dimensional dataset has measurement uncertainties, is there a standard way to incorporate this…
Simon Walker
  • 231
  • 2
  • 5
13
votes
4 answers

Classifier for uncertain class labels

Let's say I have a set of instances with class labels associated. It does not matter how these instances were labelled, but how certain their class membership is. Each instancs belongs to exactly one class. Let's say I can quantify the certainty of…
grssnbchr
  • 634
  • 1
  • 7
  • 25
13
votes
3 answers

Calculate uncertainty of linear regression slope based on data uncertainty

How to calculate uncertainty of linear regression slope based on data uncertainty (possibly in Excel/Mathematica)? Example: Let's have data points (0,0), (1,2), (2,4), (3,6), (4,8), ... (8, 16), but each y value has an uncertainty of 4. Most…
bedanec
  • 131
  • 1
  • 1
  • 4
12
votes
3 answers

How best to communicate uncertainty?

A massive problem in communicating the results of statistical calculations to the media and to the public is how we communicate uncertainty. Certainly most mass media seems to like a hard and fast number, even though except in a relatively small…
naught101
  • 4,973
  • 1
  • 51
  • 85
12
votes
3 answers

Reliability of a fitted curve?

I would like to estimate the uncertainty or the reliability of a fitted curve. I intentionally don't name a precise mathematical quantity that I am looking for, since I don't know what it is. Here $E$ (energy) is the dependent variable (response)…
thyme
  • 251
  • 1
  • 5
11
votes
1 answer

What graphical methods are useful for visualising how uncertainties aggregate?

I have a set of systems where uncertainties accumulate within it. These aren't always purely additive - sometimes they are, sometimes they aren't. I've had some success in using fan-charts, bar charts with confidence-intervals, and box plots for…
410 gone
  • 1,028
  • 8
  • 23
11
votes
2 answers

What is the difference between the Monte Carlo (MC) and Monte Carlo Markov Chain (MCMC) method?

The goal of both methods seems to be to derive an estimate of a posterior/target distribution. If a process model exists which links some input parameters (which are themselves uncertain and can be described by a PDF) to an output parameter through…
bdoering
  • 113
  • 1
  • 5
11
votes
2 answers

Voting system that uses accuracy of each voter and the associated uncertainty

Let's say, we have simple "yes/no" question that we want to know answer to. And there are N people "voting" for correct answer. Every voter has a history - list of 1's and 0's, showing whether they were right or wrong about this kind of questions in…
ffriend
  • 9,380
  • 5
  • 24
  • 29
11
votes
0 answers

Empirical Prediction interval for time series forecast based on quantile regression

As Gardner notes "almost all point forecasts are wrong", so prediction intervals (PI) are necessary to quantify uncertainty and help us make informed decisions. There exists theoretical PI, and in some instances they work. In many instances there is…
forecaster
  • 7,349
  • 9
  • 43
  • 81
10
votes
2 answers

SVM confidence according to distance from hyperline

For a probabilistic multi-class classifier we can get probabilities of membership of a new point $x$ to each class $y_i$; in case of 3 classes suppose that we get $P(y_a|x) > P(y_b|x) > P(y_c|x)$, thus the most probable class of x is $y_a$. Now…
shn
  • 2,479
  • 9
  • 31
  • 38
10
votes
2 answers

Predicting Uncertainty in Random Forest Regression

Scenario: I'm trying to build a random forest regressor to accelerate probing a large phase space. I'm using python/scikit-learn to perform the regression, and I'm able to obtain a model that has a reasonably low cross-validation error on known…
Andrew
  • 101
  • 1
  • 4
1
2 3
…
29 30