How to create Hinge loss function in python from scratch?

Question

I am learning the math behind popular loss functions by trying to hard code all loss functions from scratch. I tried to code hinge loss

My code

def hinge_fun(actual, predicted):
    # replacing 0 = -1
    new_predicted = np.array([-1 if i==0 else i for i in predicted])
    
    # calculating hinge loss
    hinge_loss = np.mean([max(0, 1-x*y) for x, y in zip(actual, new_predicted)])
    return hinge_loss

hinge_fun(actual, predicted)

I converted all 0's to -1 because I read here that hinge loss expects target values as [-1, 1]

Now my real question here is that when I tested my function with sklearn's hinge loss function, it is not always the same. For example:

import numpy as np
from sklearn.metrics import hinge_loss

# case 1
actual = np.array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1])
predicted = np.array([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1])

hinge_loss(actual, predicted) # sklearn function output 0.7
hinge_fun(actual, predicted) # my function output 0.7

# case 2
actual = np.array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1])
predicted = np.array([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1])

hinge_loss(actual, predicted) # sklearn function output 0.8333333333333334
hinge_fun(actual, predicted) # my function output 0.9166666666666666

Can you see the difference in the second case, I just showed two cases but the output is not always the same. I don't know what I am missing in my function, can you help me please?

Siong Thye Goh · Accepted Answer · 2021-08-08T13:17:33.377

First, for your code, besides changing predicted to new_predicted. You forgot to change the label for actual from $0$ to $-1$.

Also, when we use the sklean hinge_loss function, the prediction value can actually be a float, hence the function is not aware that you intend to map $0$ to $-1$. To achieve the same result, you should pass new_predicted to the sklearn function.

import numpy as np
from sklearn.metrics import hinge_loss

def hinge_fun(actual, predicted):
    # replacing 0 = -1
    new_predicted = np.array([-1 if i==0 else i for i in predicted])
    new_actual = np.array([-1 if i==0 else i for i in actual])

    # calculating hinge loss
    hinge_loss = np.mean([max(0, 1-x*y) for x, y in zip(new_actual, new_predicted)])
    return hinge_loss


# case 1
actual = np.array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1])
predicted = np.array([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 1])
new_predicted = np.array([-1 if i==0 else i for i in predicted])
print(hinge_loss(actual, new_predicted)) # sklearn function output 0.4
print(hinge_fun(actual, predicted)) # my function output 0.4

# case 2
actual = np.array([1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1])
predicted = np.array([0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1])
new_predicted = np.array([-1 if i==0 else i for i in predicted])
print(hinge_loss(actual, new_predicted)) # sklearn function output 0.5
print(hinge_fun(actual, predicted)) # my function output 0.5

How to create Hinge loss function in python from scratch?

1 Answers1