2

I'm working on a classification problem with a very imbalanced dataset. Many papers mention a "weighted cross-entropy loss function" or "focal loss with balancing weights". I can't find any of those in tensorflow (tf.keras to be precise) but there is a class_weight parameter in model.fit(). Is there a difference between those two things or is this just the way tensorflow implements weighted loss functions?

Sebastian E
  • 181
  • 1
  • 6

1 Answers1

1

The weighted cross-entropy and focal loss are not the same. By setting the class_weight parameter, misclassification errors w.r.t. the less frequent classes can be up-weighted in the cross-entropy loss. The focal loss is a different loss function, its implementation is available in tensorflow-addons.

sebp
  • 1,787
  • 13
  • 24
  • 1
    Sorry, that maybe was a bit misleading, I know that those are different functions. My question is if the class weight parameter is just the way of passing weights to the loss function in tf.keras or if there are also special loss functions that directly take weights. And if the second case was true, wat would be the difference of those options. – Sebastian E Dec 27 '19 at 23:11
  • Yes that's correct, in Keras you use the `class_weight` parameter, which under the hood gets converted to per-sample weights, depending on `class_weight`, which are used to compute a weighted loss across all samples. – sebp Dec 28 '19 at 16:00