1

As we know (Wikipedia Definition): Linear Classifier makes a classification decision based on the linear combination of the feature vectors.
Mathematically : $y = f(\sum w_i x_i)$
So , $f$ is our linear classifier (which may be logistic or any other function). Now this linear classifier creates a decision boundary.

Now, for example consider only two features(X1, X2) : If the decision boundary is straight line then we say its linear decision boundary otherwise non linear decision boundary.

So, my question :
(1) If a classifier is a linear then it creates a linear decision boundary and vice versa.
(2) Non linear classifier always creates non linear decision boundary

Does the above statements are true if not then please explain? I have seen so many examples , like for SVM classifier, we transform the data to higher dimension and get the hyperplane in feature space but in input space it has non linear decision boundary.
So, what is the exact relation between a classifier and decision boundary, especially in the linear case?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Girish Kumar Chandora
  • 281
  • 1
  • 10
  • 3
    Neither statement is generally true. The subtlety is that a nonlinear classifier may, by accident, create a linear boundary. Thus, the appearance of a linear boundary in one specific application does not determine whether the classifier *as a procedure* is linear or not. Also, the appearance of a linear manifold at some point in executing a procedure does not necessarily make the *entire* procedure a linear one. – whuber Jul 06 '21 at 15:16
  • @whuber I got your explanation. So its all depend on the situations , both linear and non linear classifier can create any type of decision boundaries. Even thought logistic regression can create non linear decission boundaries too( if extra features are added) . Pls correct me if some thing is not true. – Girish Kumar Chandora Jul 06 '21 at 15:31
  • @whuber Pls refer to these [question](https://stats.stackexchange.com/questions/178522/why-knn-is-a-non-linear-classifier), the explanations are conflicting. Still I am not able to understand the concept of linear classifier. How do we know a classifier is linear ? – Girish Kumar Chandora Jul 07 '21 at 04:13
  • The answer you reference is a little sloppy because it lacks a suitable quantifier. You have to understand it as meaning *all* boundaries are linear, no matter what the inputs might be. – whuber Jul 07 '21 at 13:49
  • @whuber Your time and help would be highly appreciated if you share any blog or reading material through which I gain these concept- Linear and Non Linear Classifier . Linear and Non Decision Boundaries. Even today I asked the question in different style on Data Science Community, but didnt get response. Have a look on [question](https://datascience.stackexchange.com/questions/97553/does-linear-classifier-creates-linear-decision-boundary-in-the-input-feature-spa?noredirect=1#comment101823_97553) – Girish Kumar Chandora Jul 07 '21 at 13:56

0 Answers0