Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [threshold]

Used (1) for discrete classification (if an instance's predicted probability exceeds a threshold, classify as TRUE, otherwise FALSE), or (2) for discretizing/binning continuous data. *If you are tempted to use this tag, PLEASE read the tag wiki!*

Either usage of thresholds is usually an error for our purposes.

Using thresholds for discrete classification

The statistical part of modeling should extend to outputting predicted class membership probabilities. Deciding whether to treat a particular case as a member of a certain class is based on this probability, but must also consider the costs of misclassification. More information at Classification probability threshold.

Predicted probabilities and thresholds are often used to calculate accuracy. Related to the point above, accuracy is not a good measure for assessing classification models.

Using thresholds to discretize/bin continuous data

Discretizing or binning continuous data typically throws away a lot of information and introduces step changes that are rarely found in actual data generating processes. It is usually better to model nonlinearities using . For more information (despite the title), see What is the benefit of breaking up a continuous predictor variable? (Answer: none.)

231 questions
69
votes
4 answers

Reduce Classification Probability Threshold

I have a question regarding classification in general. Let $f$ be a classifier, which outputs a set of probabilities given some data D. Normally, one would say: well, if $P(c|D) > 0.5$, we will assign a class 1, otherwise 0 (let this be a binary…
sdgaw erzswer
  • 1,199
  • 1
  • 9
  • 13
19
votes
3 answers

Why is the bias term in SVM estimated separately, instead of an extra dimension in the feature vector?

The optimal hyperplane in SVM is defined as: $$\mathbf w \cdot \mathbf x+b=0,$$ where $b$ represents threshold. If we have some mapping $\mathbf \phi$ which maps input space to some space $Z$, we can define SVM in the space $Z$, where the optimal…
Dejan
  • 377
  • 1
  • 2
  • 11
19
votes
1 answer

What is F1 Optimal Threshold? How to calculate it?

I've used h2o.glm() function in R which gives a contingency table in the result along with other statistics. The contingency table is headed "Cross Tab based on F1 Optimal Threshold" Wikipedia defines F1 Score or F Score as the harmonic mean of…
SoakingHummer
  • 351
  • 1
  • 2
  • 9
15
votes
5 answers

Philosophical question on logistic regression: why isn't the optimal threshold value trained?

Usually in logistic regression, we fit a model and get some predictions on the training set. We then cross-validate on those training predictions (something like here) and decide the optimal threshold value based on something like the ROC curve. Why…
StatsSorceress
  • 674
  • 4
  • 17
13
votes
5 answers

Automatic threshold determination for anomaly detection

I am working with a time series of anomaly scores (the background is anomaly detection in computer networks). Every minute, I get an anomaly score $x_t \in [0, 5]$ which tells me how "unexpected" or abnormal the current state of the network is. The…
cryptron
  • 129
  • 1
  • 1
  • 6
10
votes
2 answers

How to change threshold for classification in R randomForests?

All the Species Distribution Modelling literature suggests that when predicting the presence/absence of a species using a model that outputs probabilities (e.g., RandomForests), choice of the threshold probabilitiy by which to actually classify a…
sth
  • 265
  • 1
  • 4
  • 14
9
votes
1 answer

Decision rule as a hyper-parameter in LASSO

I have a question that is related to the following: Is decision threshold a hyperparameter in logistic regression? but would like some clarification. The general consensus is that the decision rule is not a hyper-parameter in the strictest sense…
astel
  • 1,388
  • 5
  • 17
8
votes
1 answer

cost function in logistic regression vs optimization algorithms

I have a table like: tabla <-data.frame(c=c(-3, -2, -1, -1, 1, 2, 3, 0, 0, 5, 4, 3, 0, 8, 9, 10, 6, 6, 7, 3, 0), e=c(0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1)) tabla <-…
GabyLP
  • 641
  • 6
  • 13
7
votes
2 answers

Threshold in precision/recall curve

While I was reading Torgo's Data Mining with R, I found that the description of precision/recall curve was different compared with other approaches. Usually, these curves are based on a threshold that determines which probability value is good…
Robert Smith
  • 3,191
  • 3
  • 30
  • 46
7
votes
3 answers

Are thresholds for logistic regression models prevalence-specific?

I wonder if thresholds for logistic regression models are prevalence-specific. I assume that they are, however, I am not sure about the basic statistical principles behind it and how to deal with the implications for clinical practice. Example: A…
ava
  • 120
  • 7
7
votes
1 answer

Is there a name for the increase in variance upon remeasurement after subsetting with a cut-off value?

Context: My problem relates to estimating effect sizes, such as Cohen's d, when looking at a subset of the population defined by a cut-off threshold. This effect size is the difference in two population means divided by the (assumed equal)…
David Luke Thiessen
  • 1,232
  • 2
  • 15
6
votes
2 answers

Threshold models and flu epidemic recognition

I'm fooling around with threshold time series models. While I was digging through what others have done, I ran across the CDC's site for flu data. http://www.cdc.gov/flu/weekly/ About 1/3 of the way down the page is a graph titled "Pneumonia and…
bill_080
  • 3,458
  • 1
  • 20
  • 21
5
votes
3 answers

How to determine the optimal threshold to achieve the highest accuracy

I have a list of probabilities outputted by a classifier on a balanced dataset. The metric I want to maximize is accuracy ($\frac{TP+TN}{P+N}$). Is there a way to calculate the best threshold (without iterating over many threshold values an…
Shak
  • 153
  • 1
  • 4
5
votes
2 answers

Automatic threshold selection for POT analysis

I have got > 100 river daily flow time series and I need to select a threshold for each of them in order to perform a Peak Over Threshold extreme value analysis. I know that I can select a threshold manually with specific R packages but I would like…
aaaaa
  • 274
  • 4
  • 15
5
votes
0 answers

Unsupervised Anomaly Detection Threshold Selection

If we have a data set that contains only positive examples I am wondering how we can effectively choose a threshold for an anomaly detection technique. Are there anomaly detection techniques that can select an appropriate threshold given only…
mino
  • 265
  • 3
  • 16
1
2 3
…
15 16