0

I have been searching through the whole CrossValidated but couldn't find the answer.

I want to test out-of-sample the volatility forecasts (if it means something ARCH-like ones, MSGARCH, Multifractal models). I want to use MSE and MAE metric but am not sure what to take for actual volatility.

I know that probably the best I can do is to take realized volatility of that day based on the sum of the intraday returns but I don't have access to that data.

So questions are the following:

  1. Can I compare it to squared returns as written here On forecasting, the mean squared error and realized volatility In this case my problem is that I feel like couple of outliers will influence score much more than most of the data set.
  2. Can I use absolute returns similarly as to in the point one?
  3. Is there any better reference than this?
  4. Is there any access to intraday data of S&P500 for example and if not is there maybe estimated volatility from such a data available?

Reference to any such answer is very welcome

Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Selena Pepic
  • 21
  • 2
  • Some references: Patton & Sheppard ["Evaluating Volatility and Correlation Forecasts"](https://link.springer.com/chapter/10.1007/978-3-540-71297-8_36) (2009) and Patton ["Volatility forecast comparison using imperfect volatility proxies"](https://www.sciencedirect.com/science/article/pii/S030440761000076X) (2011). – Richard Hardy May 16 '21 at 16:59
  • Maybe ["On forecasting, the mean squared error and realized volatility"](https://stats.stackexchange.com/questions/143999) can help, too. – Richard Hardy May 16 '21 at 17:10
  • As far as I remember from Patton's papers, MAE does not work well, while MSE and Qlike are OK for evaluating volatility forecasts. – Richard Hardy May 16 '21 at 17:41
  • Thank you, this helps a lot. According to Patton, your last comment is correct. Sorry if the following question doesn't make sense, I am fairly new to this but what I don't understand is the following: I have let's say last 365 returns of S&P500. I train the model on them and predict one step ahead volatility. During the Corona time S&P500 dropped by 12%, this means because I train on 100*returns, my biggest returns^2 is around 144. My biggest predicted volatility is around 6-7. Should I have trained on returns instead of returns*100 or did MSE on variance instead of sqrt of it? – Selena Pepic May 16 '21 at 19:19
  • It is normal that the underlying variance varies less than realizations of a time series. You may simulate a true GARCH process with known parameter values and see for yourself. – Richard Hardy May 16 '21 at 20:12
  • Sorry, one last thing to confirm. You say that I should use https://drive.google.com/file/d/1opaZfiwVbdMzBQ0QnADpT3ZZA205JGgE/view this as representative and not https://drive.google.com/file/d/1kSU5mQFJ6olzC4s2IzyTp6F6YsKVlCcX/view even though squared returns are in a way proportional to squared volatility? – Selena Pepic May 16 '21 at 21:47
  • I might have misunderstood you. It makes sense to compare $r_t^2$ with $\hat\sigma_t^2$, not with $\hat\sigma_t$. – Richard Hardy May 17 '21 at 05:20

0 Answers0