1

I wanna evaluate a simple GARCH(1,1) model for the conditional variance. Firstly, I understand that the conditional variance is unobserved and that is really the crux of the issue.

Out-of-sample, I have created a loop that re-estimates, for each new observation, and forecasts one period ahead. This works just fine, yet I am unsure of what I should evaluate this against. I first though the squared residuals, but then again? Of what model should these come from? The model with same specifications, but for the whole sample?

Edit: If any one else finds this question, I just wanted to say briefly what I found. When evaluating forecasts of different models, one can use squared returns (if working with stock returns). This might not give correct results (it actually almost surely will, because realized vol is not the same as squared returns), yet the ranking will as if the exercise had been done with the (unobservable) conditional variance.

pApaAPPApapapa
  • 213
  • 1
  • 7
  • 1
    Related questions: [1](https://stats.stackexchange.com/questions/225877/interpreting-qlike-and-mse-loss-function-patton-2011), [2](https://stats.stackexchange.com/questions/143999/on-forecasting-the-mean-squared-error-and-realized-volatility), [3](https://stats.stackexchange.com/questions/231429/construction-of-several-forecast-appraisal-techniques) and the link in [2] to Patton & Sheppard ["Evaluating volatility and correlation forecasts"](http://link.springer.com/chapter/10.1007/978-3-540-71297-8_36) (2009). – Richard Hardy Nov 25 '18 at 10:17
  • If you believe GARCH is a good model for the underlying truth then your best evaluation method is the log likelihood of the GARCH evaluated at out of sample points. – Cagdas Ozgenc Nov 25 '18 at 10:26
  • @RichardHardy Thanks for the response. You are always helpful. See your name around a lot, and some incredibly helpful comments among them. – pApaAPPApapapa Nov 25 '18 at 14:23
  • @CowboyTrader, a more efficient alternative to that would be AIC, hence your suggestion is not "the best". But in any case, what do you mean by "best"? Unless the likelihood coincides with your loss function, how relevant is it to begin with? – Richard Hardy Nov 25 '18 at 14:50
  • @RichardHardy OP wants to evaluate performance with out-of-sample data points. AIC is not relevant in this case as the error estimates are not biased. OP hasn't specified a loss function. Log likelihood is a very good surrogate and supported by old and new research in terms of bounding other loss functions. – Cagdas Ozgenc Nov 25 '18 at 16:48
  • @CowboyTrader, no, he does not. Read this: *I wanna evaluate a simple GARCH(1,1) model for the conditional variance*. Next, given that a model is good for the underlying data, splitting the sample into training and testing is a fundamentally inefficient strategy. However, the point that likelihood is a good surrogate is relevant. If this is true and if there is no information about the OP's loss function, using likelihood (more precisely, AIC) could be a good idea. The problem is that not knowing the loss function is unsatisfactory in itself. So better find out what it is and use it directly. – Richard Hardy Nov 25 '18 at 17:23
  • This discussion is interesting. I have though of simply using MSE for the out-of-sample. Is that incorrect? – pApaAPPApapapa Nov 28 '18 at 14:55

0 Answers0