Questions tagged [sensitivity-specificity]

Sensitivity & specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) & the proportion of real negatives that are classified correctly (specificity). They are an integral part of ROC analysis.

Sensitivity and specificity are descriptive statistics that measure the quality of a classification system. They are the proportion of real positives that are classified correctly (sensitivity) and the proportion of real negatives that are classified correctly (specificity). Consider the following four logical possibilities:

                                         Reality:
                                  Positive     Negative
            Classification:    ---------------------------
                              |             |             |
                   'positive' |     TP      |     FP      |
                              |             |             |
                               ---------------------------
                              |             |             |
                   'negative' |     FN      |     TN      |
                              |             |             |
                               ---------------------------

where TP is true positive, FP is false positive, FN is false negative, and TN is true negative. Then:
\begin{align} {\rm Sensitivity} &= \frac{TP}{TP+FN} \\[7pt] {\rm Specificity} &= \frac{TN}{FP+TN} \end{align} They are often used together, and are an integral part of ROC analysis. Sensitivity and specificity are very similar to precision and recall (in fact, recall is equivalent to sensitivity).

177 questions
85
votes
11 answers

What is the best way to remember the difference between sensitivity, specificity, precision, accuracy, and recall?

Despite having seen these terms 502847894789 times, I cannot for the life of me remember the difference between sensitivity, specificity, precision, accuracy, and recall. They're pretty simple concepts, but the names are highly unintuitive to me,…
Jessica
  • 1,781
  • 2
  • 15
  • 17
25
votes
4 answers

Given true positive, false negative rates, can you calculate false positive, true negative?

I have values for True Positive (TP) and False Negative (FN) as follows: TP = 0.25 FN = 0.75 From those values, can we calculate False Positive (FP) and True Negative (TN)?
Simplicity
  • 535
  • 1
  • 7
  • 12
19
votes
3 answers

Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity

I have a test set of 100 cases and two classifiers. I generated predictions and computed ROC AUC, sensitivity and specificity for both classifiers. Question 1: How can I compute p-value to check if one is significantly better than the other with…
14
votes
4 answers

ROC and multiROC analysis: how to calculate optimal cutpoint?

I'm trying to understand how to compute the optimal cut-point for a ROC curve (the value at which the sensitivity and specificity are maximized). I'm using the dataset aSAH from the package pROC. The outcome variable could be explained by two…
Tommaso
  • 563
  • 2
  • 5
  • 17
11
votes
3 answers

Classifier performance measure that combines sensitivity and specificity?

I have 2-classes labelled data on which I'm performing classification using multiple classifiers. And the datasets are well balanced. When assessing the classifiers' performance, I need to take into consideration how accurate the classifier is in…
11
votes
7 answers

Is sensitivity or specificity a function of prevalence?

Standard teaching says that sensitivity and specificity are properties of the test and are independent of prevalence. But isn't this just an assumption? Harrison's principles of internal medicine 19th ed says It has long been asserted that…
10
votes
0 answers

Logistic regression for classification: are there any analytical solutions for the out-of-sample accuracy?

I run a binary logistic regression, with a binary dependent variable and a continuous independent one. Now I want to evaluate the out-of-sample performance of the classification algorithm so obtained. For instance, given a probability threshold, I…
7
votes
2 answers

Is the sum of p-value and specificity 1

When I look at the definition of p value carefully: $$ p = Pr(X
zyxue
  • 847
  • 6
  • 21
6
votes
1 answer

When are ROC curves to compare imaging tests valid? (Focus on the example below)

I would like to ask what criticism could be raised in the following case example: In this paper they test a way of detecting narrowing of the cervical canal on radiographs using a ratio of measurements. In the subjects studied they do have a…
5
votes
2 answers

Origin of terms "sensitivity" and "specificity"

Who coined the terms "sensitivity" and "specificity"—meaning the complements of false positives, and false negatives, respectively in tests and measures—and when did they first do so? Bonus points if there is a motivating etymology.
Alexis
  • 26,219
  • 5
  • 78
  • 131
5
votes
1 answer

Correcting naïve Sensitivity and Specificity for classifier tested against imperfect gold standard

I am writing a supervised classifier for a particular condition. I have two sets of data for my "gold standard", against which I will test my classifier: a Positive set, in which all samples have the condition; a Negative set, in which none have…
4
votes
1 answer

Logistic regression model predicts only one outcome, producing a high specificity but very low sensitivity. How do I improve the model?

I'm designing a logistic regression model to predict hospital mortality. Why? To identify 'adjusted' odds ratios for a variable of interest on mortality. Methods: - set up using a training dataset (75% of total) I have started with 19 variables…
4
votes
2 answers

Comparison of machine learning algorithms

Suppose that I have taken 8 machine learning algorithms which are used by researchers most frequently. I have applied these 8 machine learning algorithms over 8 datasets which are publicly available on internet. I get results like: Random forest…
4
votes
2 answers

Cost functions for weighting sensitivity and specificity in binary classification problem

I'm searching for a combination of sensitivity and specificity cost function because i want have more weight for sensitivity ( sensitivity is more impotent for me rather than specificity). After searching i found this : Final_Cost = ( (Cb/Cg)/(…
user2991243
  • 3,621
  • 4
  • 22
  • 48
3
votes
1 answer

The effect of oversampling on the positive predictive value

I need to calculate the positive predictive value for a validation set for a rare event. The problem is that the validation set was oversampled for the rare event. The event occurs in 5 percent of the population, however the oversampling has…
user43856
  • 31
  • 1
1
2 3
11 12