Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [sensitivity-analysis]

Auxiliary methods intended to check if the outcome of an analysis strongly depends on the model assumptions, preprocessing steps, presence of outliers, etc.

Sensitivity analysis refers to methods to see if violations of assumptions of a model make large differences to results. Examples can include:

  • Cases that violate the assumptions are deleted and the analysis re-run.
  • If you have another measurement (i.e., variable) for a construct, the analysis could be re-run using the alternative measurement in place of the primary measurement.
  • When using Generalized Linear Models, the model could be fit using different link functions.
  • An analysis could be re-run including a very flexible function of a variable, such as a spline function, to assess assumptions about proper functional form.
  • etc. (many other possibilities exist)

In each case, the results are compared to determine if there are important differences in the results due to the specific assumptions made.

132 questions
20
votes
4 answers

Mean and Median properties

Can somebody explain me clear the mathematical logic that would link two statements (a) and (b) together? Let us have a set of values (some distribution). Now, a) Median does not depend on every value [it just depends on one or two middle…
ttnphns
  • 51,648
  • 40
  • 253
  • 462
14
votes
4 answers

ROC and multiROC analysis: how to calculate optimal cutpoint?

I'm trying to understand how to compute the optimal cut-point for a ROC curve (the value at which the sensitivity and specificity are maximized). I'm using the dataset aSAH from the package pROC. The outcome variable could be explained by two…
Tommaso
  • 563
  • 2
  • 5
  • 17
14
votes
1 answer

Sensitivity Analysis in Deep Neural Networks

Following a question already answered (Extracting weight importance from One-Layer feed-forward network) I am looking for inference about relevance of inputs in neural networks. Considering a deep net, where reconstructing the input importance by…
Tommaso Guerrini
  • 1,099
  • 9
  • 27
12
votes
1 answer

Metafor package: bias and sensitivity diagnostics

I am conducting a multi-level meta-analysis that includes some articles with multiple outcomes. Therefore I am using the rma.mv() function. Example code: test.main = rma.mv(yi,vi,random = ~1|ID, data = data) I have two questions: I read in a…
chabeck
  • 336
  • 2
  • 10
10
votes
2 answers

What is the difference between sensitivity analysis and model validation?

I read both wikipedia pages of sensitivity analysis and model validation (here, only linear regression validation) but I don't manage to find a way to separate these two terms. I have the impression that the first one is more used in academia and…
YCR
  • 506
  • 4
  • 20
9
votes
1 answer

Linear Regression: How to favour less "sensitive" parameters?

I have a simple regression model (y = param1*x1 + param2*x2). When I fit the model to my data, I find two good solutions: Solution A, params=(2,7), is best on the training set with RMSE=2.5 BUT! Solution B params=(24,20) wins big on the validation…
elemolotiv
  • 1,048
  • 7
  • 20
6
votes
1 answer

How good is Monte Carlo Simulation when the variable distribution is unknown?

I am reading the book "how to measure everything", there is a chapter when the author encourages the usage of Monte Carlo simulation in simulating the future events in order to get a better understanding of the risk/ uncertainty involved. The theory…
Graviton
  • 845
  • 1
  • 15
  • 28
6
votes
1 answer

Is there a branch of statistics that tries to explain "why" the dataset has certain statistical properties?

Suppose I have a big dataset and I compute some statistical summary of it - e.g., the correlation of one dimension with another. I think a reasonable question to ask would be "what data points explain this result" - e.g., perhaps, is it because…
jkff
  • 221
  • 1
  • 5
6
votes
2 answers

Meta-analysis and homogeneity -- what did these guys do?

I appreciate any insight into this meta-analysis. This is a meta-analysis on alogliptin efficacy and safety. In the 2nd paragraph of the discussion that make this statement: Although heterogeneity testing showed a statistically significant…
Alex
  • 61
  • 1
6
votes
1 answer

Sensitivity Analysis for Missing Not at Random (MNAR) data

I currently have a dataset which contains variables with different degrees of missingingness. One of the key variables for my analysis has about 12% of the values Missing Not at Random (MNAR). From previous research I have done, I gather that most…
RNB
  • 506
  • 5
  • 12
6
votes
1 answer

Sensitivity analysis of machine learning techniques

As you know we can have sensitivity analysis (sensitivity of output(s) based on changing of inputs) in different kinds of regression. Can we have sensitivity analysis for machine learning techniques (Specially Neural network and Support vector…
user2991243
  • 3,621
  • 4
  • 22
  • 48
5
votes
1 answer

What are the proper ways how to add noise to a dataset? (algorithm data sensitivity analysis)

Consider: dataset defined as n datapoints x_i in m-dimensional space. And there is a label y_i defining one of the classes belonging to x_i. There are let's say 5 classes 1,2,3,4,5 (and there is total order among the classes, i.e. 1<2<3<4<5). What…
Marek
  • 395
  • 5
  • 11
5
votes
2 answers

Differences between robustness checks and sensitivity analysis

This is a bit of a terminology question, but what is the difference between a robustness check and a sensitivity analysis? For example, if performing analysis to see how sensitive (or robust) a study's conclusions are to additional variables. Are…
asdf
  • 59
  • 1
  • 3
5
votes
1 answer

Difference between Sobol indices and total Sobol indices?

Given a mathematical model $Y\widetilde Y(X_i)$, where $X_i=x_i^*$ represents a particular point estimate for input variable $X_i$. In sensitivity analysis, Sobol indices explain the importance of an input factor $X_i$ on the variance of the output…
HCAI
  • 737
  • 2
  • 7
  • 23
5
votes
1 answer

Sobol Indices for Discrete Variables

It is well known that the Sobol sensitivity indices are derived from the ANOVA decomposition. Most proofs that I read concerning this subject assume that the original model is of the form $$ Y=f(X_1,X_2,\cdots, X_n)$$ Where $Y$ is the dependent…
Nizar
  • 737
  • 1
  • 6
  • 17
1
2 3
…
8 9