0

I am trying to learn a time series (Mackey-Glass) using a neural net. In order to see if there has been success in the learning process, I am looking at the correlations between the predicted and real values.

The following table shows these correlations:

Correlations

The correlation between the predicted value and the actual real value real_lag_0 is 0.986. Now comparing the predicted value with the real value shifted backwards by one real_lag_1 yields 0.993.

It seems like the highest correlation occurs at lag 1. Compare this source

If the lag is 1, then the neural net is worthless?

How could I improve the model to obtain the highest correlation for actual values?

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
Andy
  • 103
  • 3
  • 1
    What do you mean by "what is the lag in this case"? The lag is the lag, i.e., the time series shifted by one period. It looks like your forecasts are better at predicting lagged values, rather than the actual values you are trying to predict. If these are one-step-ahead forecasts, then your forecast is a better estimate of the last observation than of the future value, which is not very surprising. – Stephan Kolassa Mar 21 '19 at 06:29
  • @StephanKolassa Thank you for your answer! I meant the lag where the peak of correlation occurs, as mentioned in this source: https://medium.com/neuri/being-bayesian-and-thinking-deep-time-series-prediction-with-uncertainty-25ff581b056c ... I was not sure, since the values are quite similar. How could I improve the model to obtain the highest correlation for actual values? – Andy Mar 21 '19 at 19:23
  • Thank you for your clarification. I'll nominate for reopening, though I'm afraid we won't be able to help you much. – Stephan Kolassa Mar 22 '19 at 10:35

1 Answers1

2

It looks like your forecasts are better at predicting lagged values, rather than the actual values you are trying to predict.

If these are one-step-ahead forecasts, then the value with lag 1 is the last historical observation. In this case, your forecast is a better estimate of the last observation than of the future value. If you think about it, this is not very surprising. After all, the last observation should carry a lot of information about your future time series, and in contrast to the actual future, it is known.

I suspect this is your key question:

How could I improve the model to obtain the highest correlation for actual values?

Unfortunately, we can't tell you that. You will need to dig into your data and understand whether there are any unmodeled drivers you could include. Or conversely, perhaps you have modeled weak drivers, which may make your forecast worse through the bias-variance tradeoff.

This earlier thread may be helpful: How to know that your machine learning problem is hopeless?

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
  • Is driver a synonym for feature? So, in the model I used the last 10 values as input for predicting the next value. Maybe some of them are weak drivers and I should use only e.g. the last 5 values? – Andy Mar 22 '19 at 19:20
  • For instance. Or you may have external drivers. Not every time series is driven only by itself. For instance, a key driver for retail sales are promotions and price changes. And to understand what might drive your series, you need subject matter expertise. – Stephan Kolassa Mar 22 '19 at 19:50