0

I'm trying to interpret the weights of a linear svm which I use to classify elements in my dataset of patients into two classes: alzheimer and non-alzheimer. From this post I understand that the value of each weight can be interpreted as: "how much a feature contributes in the classification". I'm interest in understanding how to correlate the sign of the weights to the classification label: in what way do I infer that positive weights are associated to the alzheimer label and negative weights to the non-alzheimer label, and not vice-versa? I also read this post but it seems to me his reasoning is based on having two numerical labels, one positive and one negative...

pls78
  • 3
  • 2

1 Answers1

0

Positive and negative weights are both associated with AD and non AD labels. If you code AD as 1 and non AD as 0, then positive weights are associated positively and negative negatively with the AD label and vice versa with the non AD label. I.e., positive weight means that the larger that variable is, the higher chance that a subject will be classified as AD, and negative weight means that the lower that variable is, the higher the chance of AD classification.

That being said I am not a huge fan of interpreting weights from ML models, see for example Haufe et al., On the interpretation of weight vectors of linear models in multivariate neuroimaging https://www.sciencedirect.com/science/article/pii/S1053811913010914

rep_ho
  • 6,036
  • 1
  • 22
  • 44
  • Thank you for your answer. So, if I get it right, the only information I can infer from inspecting the weights is how much a feature contributes to the classification but one can't tell whether it contributes to one class or the other. Namely, no information regarding which label a feature is contributing to can be derived from the sign of its associated weight, e.g. positively weighed features are not necessarily associated with AD and negatively weighed features with non AD. Is it correct? – pls78 Nov 04 '21 at 21:12
  • Yes, you cannot say that, but that is not because of SVM or any model. That's just how it is when you are classifying something. In no situation do you have a variable that is only associated with one class, is height associated with being a male or female? Both, positively with being male and negatively with being female. Obesity is positively associated with heart attacks but also negatively associated with healthy hearth – rep_ho Nov 04 '21 at 21:35