Why doesn't mean square error work in case of angular data?

Question

Suppose, the following are the first few lines from a dataset for solving a regression problem:

H   7.042 5.781 5.399  -9.118   5.488  7.470     0 0 0 1 0 1 1 3 0 0 0 1 0 0 0 0 0 0 0 0    2.144    
H   5.781 5.399 5.373   5.488   5.166  6.452     0 1 1 3 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0    2.268
H   5.399 5.373 5.423   5.166   4.852  6.069     0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    2.101 
H   5.373 5.423 5.247   4.852   5.164  6.197     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    2.222 
H   5.423 5.247 5.485   5.164   4.943  6.434     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    2.416
C   5.247 5.485 6.675   4.943   8.103  8.264     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    3.028
C   5.485 6.675 6.372   8.103  -9.152  9.047     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    -1.235
C   6.675 6.372 5.669  -9.152  -8.536 11.954     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    -0.953 
H   6.372 5.669 5.304  -8.536   5.433  6.703     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    2.233 
H   5.669 5.304 5.461   5.433   4.924  6.407     0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0    2.313

The left-most column is the class data. The right-most column is Y data. The rest of the features are all angular data.

.

My initial setup for the model was as follows:

def create_model(n_hidden_1, n_hidden_2, num_features):
    # create the model
    model = Sequential()
    model.add(tf.keras.layers.InputLayer(input_shape=(num_features,)))
    model.add(tf.keras.layers.Dense(n_hidden_1, activation='relu'))
    model.add(tf.keras.layers.Dense(n_hidden_2, activation='relu'))
    model.add(tf.keras.layers.Dense(1))

    # instantiate the optimizer
    opt = keras.optimizers.Adam(learning_rate=LEARNING_RATE)

    # compile the model
    model.compile(
         loss="mean_squared_error",
         optimizer=opt,
         metrics=["mean_squared_error"]
    )

    # return model
    return model

This model didn't produce the correct outcome. The expected output should be around $0.20$. However, the current setup is producing around $0.80$. These are radian values.

Someone told me that MSE doesn't work in the case of angular data. So, I need to use a custom output layer and a custom error function.

Why doesn't mean square error work in the case of angular data?

How can I solve this issue?

@J.Delaney, The expected output should be around $0.20$. However, the current setup is producing around $0.80$. These are radian values. — user366312, Feb 19 '22 at 19:25
You mean the output for some single line? Is the Y data also in radians? Why are some of the angular features greater than 2$\pi$? Try to explain your problem more clearly — J. Delaney, Feb 19 '22 at 19:47
@J.Delaney, I have edited the data. 1st column is class, the next 6 columns are the angles of adjacent atoms of an alpha-carbon in amino acid, the next 20 columns are # of neighboring atoms, the last column is Y. — user366312, Feb 19 '22 at 20:01
This data is generated by my P.I. So, it's hard to describe for me. — user366312, Feb 19 '22 at 20:07
if Y is in radians (?) then MSE can obviously be problematic because for example $-\pi$ and $+\pi$ are the same angle and that is not reflected in your loss function. — J. Delaney, Feb 19 '22 at 20:18
The natural thing would be to change the loss function for example to be square of the angular difference (see e.g. [https://stackoverflow.com/questions/1878907/how-can-i-find-the-difference-between-two-angles]). I don't know if something like that already exists or you will have to write your own custom function. — J. Delaney, Feb 19 '22 at 20:37
Since this seems to data for modeling a protein, it may also be worth asking questions on [Bioinformatics](https://bioinformatics.stackexchange.com/) or [Matter Modeling](https://mattermodeling.stackexchange.com/) about the underlying data and what property you are trying to calculate. — Tyberius, Feb 21 '22 at 14:51

Sycorax · Answer 1 · 2022-02-27T19:48:45.270

The problem with using MSE directly on angle values is that values can be very close together on the circle, but will have a large square error. For example, 359 degrees and 1 degrees are very close together on a circle, but very far apart in terms of squared difference.

To rectify this, you should use a distance metric that reflects the fact that you're working with angles.

The Euclidean distance between two points on a circle is the length of the chord between the two points. So a simple and direct example of a distance considers the angle $\alpha$ as a point on the unit circle $(\cos \alpha, \sin \alpha)$, then computes the Euclidean distance in the ordinary way.

Elementary trigonometric identities simplify the expression.

$$\begin{align} d_E^2 (\theta, \phi) &= (\cos \theta - \cos \phi)^2 + (\sin \theta - \sin \phi)^2 \\ &= \cos^2 \theta - 2 \cos \theta \cos \phi + \cos^2 \phi + \sin^2 \theta - 2 \sin \theta \sin \phi + \sin^2 \phi \\ d_E (\theta, \phi) &= \sqrt{2 - 2 \cos (\theta - \phi) } \end{align}$$

By inspection, we can see that angles that are very close together have $\cos (\theta - \phi)$ close to 1, so the distance is close to 0. On the other hand, angles that are very far apart have $\cos (\theta - \phi)$ close to 0, so the distance is close to 1. This satisfies our intuitive notions of distance.

Of course, this is not the only option. This distance and some alternatives are discussed in these related threads:

Why doesn't mean square error work in case of angular data?

1 Answers1