How to validate your loss function when it is not a simple regression or classification?

Question

Assuming I have loss function f(y_pred,y_target) that I will use to train my neural network. In this case the loss function is a regression, and let's say it should work on something that is not directly solvable by something like a mean squared error loss function. An example would be an angle, which gives the rotation of an object, due to the definition of the circle**.

I am not asking for a loss function for this problem, I want to know if there is a good way to test, if your loss function is correct.

I made two assumption which must be true for a loss function for work:

1. The higher the discontent with the result, the higher the loss
2. The derivation of the loss function should only be zero when the loss function itself is zero

One could now test this mathematically, however I am looking for a more programmable approach, like using a stochastic input and an error with a certain distribution added and expecting the same distribution in the loss function.

So, is there a typical approach to this? Some kind of "roadmap" to test the loss function itself to guarantee, that it will do the right job?

** a simple regression will not work, because 0 and 2pi,4pi and so on will mean the same

Answer 1

On the paper "Outline of forecast theoryusing generalized cost functions" 1 Clive Granger (a nobel prize winner no less!) gives the following properties of a loss function $L(x)$:

1. $L(0) = 0$ (no error, no loss).

2. $\min L(x) = 0$, so $L(x)\geq 0$.

3. $L(x)$ is monotonically non-decreasing as $x$ moves away from zero. (similar to your first requirement)

You could write your code to check these conditions

Also interesting is the paper "Loss Functions in Time Series Forecasting" 2 by Lee

Links: