3

Heads up: I'm not sure if this is the best place to post this question, so let me know if there is somewhere better suited.

I am trying to train a simple neural network to learn a simple quadratic function of the form: $f(x) = 5 - 3x + 2x^2$

I set up a single-layered network with a single neuron. The input is a 2d array of the form $(x, x^2)$ and I don't use an activation function. I expect that the weights and biases I extract from the network will correspond to the coefficients in the function $f(x)$.

I randomly generate some training points and labels, as well as a validation data set, and train my model using the Keras sequential model called from TensorFlow. 

import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt

def fTest(x_arg):
    return 5 - 3*x_arg + 2*(x_arg)**2 

# training data
t = np.random.choice(np.arange(-10,10, .01),5000 )
t1 = []
for i in range(len(t)):
    t1.append([t[i], t[i]**2])
s = []
for i in range(len(t)):
    s.append(fTest(t[i]))
t1 = np.array(t1)
s = np.array(s)

# validation set
v = np.random.choice(np.arange(-10,10, .01),5000 )
v1 = []
for i in range(len(v)):
    v1.append([v[i], v[i]**2])
u = []
for i in range(len(v)):
    u.append(fTest(v[i]))
v1 = np.array(v1)
u = np.array(u)

model = keras.Sequential([
    keras.layers.Dense(1, input_shape=(2,) , use_bias=True),
])

model.compile(optimizer='adam', 
              loss='mean_squared_logarithmic_error',
              metrics=['mae','accuracy'])

model.fit(t1, s, batch_size=50, epochs=2000, validation_data=(v1,u))

The model seems to train, but very poorly. The 'accuracy' metric is also zero, which I am very confused about. 

Epoch 2000/2000
200/200 [==============================] - 0s 23us/step - loss: 0.0018 - mean_absolute_error: 1.0144 - acc: 0.0000e+00 - val_loss: 0.0014 - val_mean_absolute_error: 1.0276 - val_acc: 0.0000e+00

Visually, the predictions of the model seem to be reasonably accurate

enter image description here

I've tried other loss-functions but none of them seem to work any better. I'm fairly new to to using TF/Keras so is there something obvious that I'm missing?

Edit: corrected training output

nonreligious
  • 131
  • 4
  • 2
    accuracy doesn't make sense as a metric for regression tasks. – shimao Apr 22 '19 at 15:31
  • The term 'accuracy' is applicable to situations where you need to make class/category predictions where you have a success or failure on the label allocation, because you may be off by 0.1, and the `RMSE` (root mean square error) is therefore a more suitable measure to track – Vass Apr 22 '19 at 15:40
  • I see, my mistake! Is there a way I can fix this by rounding/ using a few significant figures? Thanks. – nonreligious Apr 22 '19 at 15:44
  • @nonreligious Accuracy isn't appropriate for this task, so I don't see how rounding would help you. What problem are you trying to solve? I don't think you'll find a square wheel terribly practical. – Sycorax Apr 22 '19 at 15:46
  • I was hoping that I could get an estimate of how accurate the net was training by determining what percentage of predictions were correct - to, say 2 d.p. – nonreligious Apr 22 '19 at 15:53
  • 1
    i would suggest to also plot "residuals" for your result, that is difference between prediction and true data. That is much better way to visualize error than just data – aaaaa says reinstate Monica Apr 22 '19 at 16:35

1 Answers1

5

You're misunderstanding what , in the sense Keras implements, measures. Accuracy measures the proportion of samples correctly predicted, such as allocating images into classes (e.g. "dog," "cat," or "fish").

In your problem, you're not trying to infer class membership, you just want the predictions to be close to the true values. It's not surprising that accuracy is poor, because accuracy doesn't measure anything relevant about this problem.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • I see, my mistake! Is there a way I can improve the training to reduce the loss further? It seems pretty poor at the moment. – nonreligious Apr 22 '19 at 15:47
  • What is poor? `loss: 4.5276e-13` is at the edge of machine epsilon. – Sycorax Apr 22 '19 at 15:51
  • Sorry - I have been going round in circles and think I pasted the wrong thing. I tried this again and found `Epoch 2000/2000 200/200 [==============================] - 0s 23us/step - loss: 0.0018 - mean_absolute_error: 1.0144 - acc: 0.0000e+00 - val_loss: 0.0014 - val_mean_absolute_error: 1.0276 - val_acc: 0.0000e+00` but I think this can be improved with more rounds of training. Thanks for your help, I will re-evaluate what I am asking. – nonreligious Apr 22 '19 at 15:59
  • Glad to help. We also have a thread about how to go about debugging and improving a network to fit the training data well. Perhaps you would find it helpful : https://stats.stackexchange.com/questions/352036/what-should-i-do-when-my-neural-network-doesnt-learn/352037#352037 – Sycorax Apr 22 '19 at 16:02
  • Cheers, this looks very useful! – nonreligious Apr 22 '19 at 16:06