1

On the XOR problem, there is no linear decision boundary (linear in $x_1$ and $x_2$) that will be able to perfectly classify all 4 points. The weights and bias of a decision boundary that classifies the most points correctly is:

weight = $[1, 1]$ bias = $-1.5$

Producing the decision boundary:

Best linear decision boundary on XOR

Is there any linear model (of the features $x_1$ and $x_2$ only) that will converge to the decision boundary that classifies the points as best as is possible? If so, what is the model and why does it converge to this decision boundary?

jet457
  • 11
  • 2
  • You need to be specific about your models. After all, one can achieve perfect classification with a simple linear model of the form $E[h(Y)]=\beta_0+\beta_1 x_1 x_2.$ – whuber Dec 06 '19 at 21:14
  • I'll update the question to specify I mean linear models of the features $x_1$ and $x_2$ only. – jet457 Dec 06 '19 at 21:17
  • 1
    Thank you. You seem to have some misperceptions about logistic regression that the thread at https://stats.stackexchange.com/questions/127042/why-isnt-logistic-regression-called-logistic-classification might clear up. – whuber Dec 06 '19 at 21:25
  • 1
    Logistic regression is not attempting to classify the points at all; see https://www.fharrell.com/post/classification/ . This is important in the context of your optimal decision boundary above; sure, you got two out of three in the lower left group correct, but that third one is very, very wrong as measured by criteria that penalize you based on how far from the decision boundary you are. – jbowman Dec 06 '19 at 21:25
  • I'm voting to close this question as off-topic because it appears to involve a confusion about what linear models do. – Peter Flom Dec 07 '19 at 11:56
  • Wait what about the follow up question? I understand now why logistic regression won't produce the answer I want, but I'm also curious to know if there are any linear models in $x_1$ and $x_2$ that can produce the decision boundary with optimal classification error – jet457 Dec 08 '19 at 00:10
  • Since you've drawn one in the plot above, the answer would appear to be "yes". – jbowman Dec 08 '19 at 00:35
  • I know this linear decision boundary exists. I'm asking if there are any models that will converge to that decision boundary or something similar, and if so, what are they? Or at the very least what is the mathematical description of a model that would converge to that decision boundary. – jet457 Dec 08 '19 at 01:45

0 Answers0