3

I am very confused in testing regressions and know that there are many explanations available online, but I am still not getting anything it in my mind. Suppose I have daily data for past 100 days, I run a simple linear regression estimate the parameters. Now there are three things:

1) If I just use the first 75 days of data and rerun the regression, I'll get slightly different parameters and then I can forecast the 76th day value of dependent variable through it and check the error comparing it with the original 76th day dependent variable value. Is this out of sample testing?

2) If the above is out of sample then what is in-sample testing? To be specific what would I use in regression and what to estimate?

3) If I use the original regression and estimate the 101th day dependent value then would it be a forecasting or another form of out of sample testing? Also for every next forecast, how would I know if I need to rerun the model adding more recent data or should I continue with the parameters derived from the first regression.

Dawny33
  • 2,239
  • 1
  • 21
  • 37
Meesha
  • 133
  • 1
  • 1
  • 5

1 Answers1

5
  1. Yes
  2. In-sample testing is looking at the errors of the first 75 days. Obviously the regression is already fitted to that data. If those errors are similar to the out of sample errors, it might be a good indicator that the model generalizes well.
  3. If you don't have the y data for the 101th day, it's forecasting. If you do have the y data, it's out of sample testing. Ideally you should be rerunning the regression every day, either using a rolling or expanding window.

See here to learn more about walk-forward optimization.

JPN
  • 746
  • 1
  • 6
  • 16
  • I know this is obvious, but to the beginner it could be confusing: In (2) you could also write 100 instead of 75. I.e. in-sample testing means that you use an observation for testing that was part of the estimation sample. – sheß Sep 02 '15 at 10:19
  • I'm not sure I understand. In part #1 of the question, @Meesha states that the regression was run on the first 75 days. – JPN Sep 02 '15 at 13:08
  • I know, I'm just saying that "Estimating with 100 days and looking at the error of the $100^{ths}$ day" would also be in-sample testing. Quite obvious, I know. But maybe it helps understanding your answer if part #2 of the question is considered separate from part #1 – sheß Sep 02 '15 at 13:11
  • Ok. I think I see your point. You're saying that there might be a case where you want to look at the distributions for the whole data set, both in-sample and out of sample data. In which case, that is considered in-sample testing? – JPN Sep 02 '15 at 13:13