Questions tagged [bayes-optimal-classifier]
29 questions
12
votes
4 answers
Calculating the error of Bayes classifier analytically
If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes classifier for them theorically?
Also suppose the…
Isaac
- 973
- 1
- 9
- 20
9
votes
1 answer
Bayes optimal classifier vs Likelihood Ratio
I am getting slightly confused by all the probabilistic classifiers.
The bayes optimal classifier is given as $ max (p(x|C)p(C)) $ and if all classes have equal prior then it reduces to $ max (p(x|C)) $
The likelihood ratio is given as $…
RuiQi
- 545
- 1
- 6
- 13
7
votes
0 answers
Help with a proof of Bayes classifier optimality
I have a class assignment to provide a proof that Bayes classifier for the two label version is optimal in that it's error rate is always ${\le}$ any other classifier.
I've never worked through a proof before, or written one. I have a proof I've…
Zack Newsham
- 675
- 1
- 8
- 12
6
votes
1 answer
Empirical Risk Minimization: Rewriting the expected loss using Bayes' rule and the definition of expectation
I am currently studying Transfer Learning by Qiang Yang, Yu Zhang, Wenyuan Dai, and Sinno Jialin Pan. Chapter 2.2 Instance-Based Noninductive Transfer Learning says the following:
As mentioned earlier, in noninductive transfer learning, the source…
The Pointer
- 1,064
- 13
- 35
3
votes
1 answer
This decision is the best we can do if we have no prior information about the probabilities of the two classes?
I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm says the following:
We are now in the position…
The Pointer
- 1,064
- 13
- 35
3
votes
0 answers
Question about using Bayesian rule as a classification for continuous data set
Please note that my question is not about coding.
I am now learning Bayesian classification and I think I understand it in a discrete case. I have trouble understanding it for multivariate continuous data. My problem is in calculating the…
Mary
- 116
- 1
- 9
2
votes
1 answer
How is this a "Bayes classifier"?
I am currently studying the textbook Learning with kernels: support vector machines, regularization, optimization and beyond by Schölkopf and Smola. Chapter 1.2 A Simple Pattern Recognition Algorithm says the following:
We are now in the position…
The Pointer
- 1,064
- 13
- 35
2
votes
2 answers
Improve Adaboost that using weighted logistic regression instead of decision trees
I implemented Adaboost that using weighted logistic regression instead of decision trees and I managed to get to 0.5% error, I'm trying to improve it for days with no success and I know it possible to get with him to 0% error, hope you guys could…
Oran Sherf
- 21
- 3
2
votes
1 answer
Use different Naive Bayes classifiers to target different data
I am practicing using the Naive Bayes classifier to predict whether people get a stroke or not, but, I am confused with two classifiers. One is categorical Naive Bayes, another is Gaussian Naive Bayes.
For example, in the dataset, there are several…
Woden
- 167
- 8
2
votes
1 answer
LDA and Fisher LDA - are their weight vectors always equivalent?
Linear Discriminant Analysis (LDA) and Fisher Linear Discriminant Analysis (FLDA) both project high-dimensional observations to univariate classification scores using different rationals and assumptions. For simplicity, I'm here considering the…
Trisoloriansunscreen
- 1,669
- 12
- 25
2
votes
1 answer
Combining Classifiers with different Precision and Recall values
Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is high and the recall is low, whereas the precision…
user8115948
- 21
- 3
2
votes
1 answer
Why classifiers report the class with maximum posterior probability as the predicted class?
When we train a classifier to predict $y \in \{1, \dots, K\}$ given an input $x$, classification is done by reporting the class with the highest posterior probability as the prediction; that is:
$$
\hat{y}(x) = \arg\max_{\hspace{-0.75cm} y \in \{1,…
Sobi
- 2,141
- 11
- 22
1
vote
0 answers
Derive the criterion for minimizing the expected loss when there is a general loss matrix and general prior probabilities for the classes
In the book "Pattern Recognition and Machine Learning" I am trying to do exercise 1.23 (p.63):
Derive the criterion for minimizing the expected loss when there is a general
loss matrix and general prior probabilities for the classes.
On p.42 it…
Slim Shady
- 203
- 9
1
vote
0 answers
What is the relation between Linear Classifier and Linear Decission Boundary (or Non Linear Decision Boundary)?
As we know (Wikipedia Definition): Linear Classifier makes a classification decision based on the linear combination of the feature vectors.
Mathematically : $y = f(\sum w_i x_i)$
So , $f$ is our linear classifier (which may be logistic or any…
Girish Kumar Chandora
- 281
- 1
- 10
1
vote
2 answers
How would you find a p threshold for a binary classification prediction?
Lets say that there's a binary classification problem where $X$ ∈ $R_p$ and $Y ∈ \{0,1\} $ and $Pr(Y = 1 | X = x) = p$ for $p$ in $[0,1]$. There is a loss function $L_{falseneg} > 0$ for false prediction of Y = 0 when the outcome is Y = 1, and vice…
321ahno
- 11
- 1