1

I have a real-world time series process with an dependent variable Y responds to multiple independent variables (not necessary a linear response but can assume so if necessary). A non-statistical model was built to predict the process with the same set of the independent variables. I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables the same way as of the real-world process.

I could build another statistical model for the residuals (real Y - modeled Y) and look at the model coefficients and their statistical significance. But there are some issues associated with the approach. I may not be able to flag model biases or noises. Just wondering whether there is more statistically rigorous approach for this problem? Thanks and any hint may help.

athlonshi
  • 111
  • 2

1 Answers1

1

You asked : "I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables"

I have implemented a test for the constancy of parameters over time (extending the revered Chow Test) to test for break-points in time where the parameters of the model changed significantly.

In this way one can assess the collective homogeneity of model parameters over tine.

IrishStat
  • 27,906
  • 5
  • 29
  • 55
  • Telling us you have an answer is not providing an answer! It's just teasing... . – whuber Nov 28 '18 at 20:00
  • 1
    I should have referred to https://stats.stackexchange.com/questions/342512/could-adding-more-history-expanding-the-training-sample-reduce-forecast-accuracy has a discussion of the CHOW Test as applied to testing the constancy of parameters for different time slices (groups) – IrishStat Nov 28 '18 at 21:55