1

I am working on a multi-label classification problem. Each sample is capable of taking more than a single label. Sometimes samples don't have any labels associated with them.

My dataset has 50% samples with 1 or more labels associated with them. The remaining have no labels at all. I am sure, among the future "test" samples, there will be a population that has no labels attached.

So far, I've been dropping the 50% samples with no labels and training a multilabel classifier. Recently, I realized that this model will end up predicting labels for a sample even when none of the labels seem appropriate for it. This leaves me with 2 options -

  1. Add a new label called "NONE", which is equal to 1 for samples with no labels and 0 for label-annotated samples.
  2. Simply train the multilabel classifier on all the standard labels. Let the model figure out on its own which combination of features qualify for no labels at all.

Am I thinking in the right direction? I'd also like to know your suggestions on this problem.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Aishwarya A R
  • 133
  • 3
  • 1
    Similar Qs have been asked before: https://stats.stackexchange.com/questions/228853/classification-with-partially-labelled-data-potential-positives, https://stats.stackexchange.com/questions/450395/semi-supervised-learning-with-partially-unobservable-labels, https://stats.stackexchange.com/questions/169887/classification-with-partially-unknown-data, https://stats.stackexchange.com/questions/517/unsupervised-supervised-and-semi-supervised-learning and search this site! – kjetil b halvorsen Aug 23 '20 at 19:20
  • Thank you! I'll check them out. – Aishwarya A R Aug 24 '20 at 04:36

1 Answers1

1

Let $n$ be the number of distinct labels. The problem with your first proposed solution is that your multi-label method now has to learn that label "NONE" never co-occurs with other labels. If the multi-label method assumes nothing about the distribution of labels, then it has to learn that all $2^n-1$ combinations of labels in which "NONE"=1 and at least one other label is 1, never occurs. It also, does not prevents predicting all zeroes.

As your problem has a lot of samples with no labels at all, a simple and effective solution is to build your own hierarchical classifier. Make two classifiers: The first one is a binary classifier that just detects if all labels are zero or not. To train this binary classifier, just transform your samples with no labels to label "A" and all other labels to label "B". That is, a "A" from this binary classifier means no labels at all and a "B" means that exists at least one label. The second classifier is any multi-label classifier you want, but only trained on samples with at least one label. In prediction/test phase, this second classifier is only called if the first binary classifier predicts label "B" (at least one label). Details on more elaborated hierarchical classifiers can be found in: https://www.researchgate.net/publication/306040749_Consistency_of_Probabilistic_Classifier_Trees

Other common solutions are using one of this four multi-label methods in combination with a multi-class classifier (ex: K-nn and SVM): Binary Relevance, Classifier Chain and Label Powerset. Scikit-learn implement this methods. I suggest Classifier Chain, which take dependencies among labels into account, as it seems from your question that you want the algorithm to predict pretty well when there is no labels at all. Label Powerset is also a good solution, except if you have a "lot" of labels ($n\geq 20$) and not sufficient data.

lhsmello
  • 26
  • 5