12

I want to implement (in R) the following very simple Dynamic Linear Model for which I have 2 unknown time varying parameters (the variance of the observation error $\epsilon^1_t$ and the variance of the state error $\epsilon^2_t$).

$ \begin{matrix} Y_t & = & \theta_t + \epsilon^1_t\\ \theta_{t+1} & = & \theta_{t}+\epsilon^2_t \end{matrix} $

I want to estimate these parameters at each time point, without any look ahead bias. From what I understand, I can use either a MCMC (on a rolling window to avoid the look ahead bias), or a particle filter (or Sequential Monte Carlo - SMC).

Which method would you use, and
What are the pros and cons of these two methods?

Bonus question: In these methods, how do you select the speed of change of the parameters? I guess we have to input an information here, because there is a bargain between using a lot of data to estimate the parameters and using less data to react more quickly to a change in the parameter?

Taylor
  • 18,278
  • 2
  • 31
  • 66
RockScience
  • 2,731
  • 4
  • 27
  • 46
  • My question is a bit similar to http://stats.stackexchange.com/questions/2149/what-is-the-difference-between-a-particle-filter-sequential-monte-carlo-and-a-k. I have re opened a question on purpose as the situation is a bit different and I would like different opinions. (The answer by gd047 was mainly focusing on unscented Kalman filter (UKF) ) – RockScience Nov 29 '10 at 07:54
  • Weird that my bounty doesn't help... Is my question badly formulated.... No one has a piece of answer? Or a question on my question? – RockScience Dec 03 '10 at 07:52
  • The way it's posed, this seems like a degenerate problem -- the errors could equally be attributed to the observation noise or the process noise. Are there more constraints? Is the state one dimensional? – IanS Dec 04 '10 at 21:48
  • @lanS. All objects have indeed here only one dimension. Can you develop a bit more on the fact that the errors that can be either observation or noise. It is exactly what I would like to achieve. I would like to get a rolling estimation of the signal to noise ratio by estimating the sd of the 2 time varying noises.... – RockScience Dec 05 '10 at 15:34
  • Maybe should I start by fixing the sd of the process noise for a start and see how the sd of the observation noise reacts? – RockScience Dec 05 '10 at 15:36

3 Answers3

5

If you have time varying parameters and want to do things sequentially (filtering), then SMC makes the most sense. MCMC is better when you want to condition on all of the data, or you have unknown static parameters that you want to estimate. Particle filters have issues with static parameters (degeneracy).

Darren Wilkinson
  • 351
  • 3
  • 8
4

Have a look at the dlm package and its vignette. I think you might find what you are looking for from the vignette. The package authors have also written a book Dynamic Linear Models with R.

whuber
  • 281,159
  • 54
  • 637
  • 1,101
Matti Pastell
  • 306
  • 1
  • 3
  • @ Matti Pastell: I have this book. It is very good indeed. My question is about the difference between the particule filter (which from what I understand is a sequential version of MCMC), and a MCMC on a rolling window (in the latter, we re run the optimisation process on a rolling window). Which method should be prefered, and why? – RockScience Dec 08 '10 at 07:53
  • Also, I don't really find easy to build this time varying model with dlm. Honestly the package is very easy to use for non time varying models, but it starts to be more tricky for everything else. Edit: By more tricky I mean that there is no function to solve the problem. You need to code yourself the script. – RockScience Dec 08 '10 at 07:56
  • 1
    OK, I also have the book but I haven't had time to read it yet. Sorry that it doesn't help with your problem. – Matti Pastell Dec 08 '10 at 08:01
  • 1
    Thank you anyway, it is a good book, it deserves to be cited here – RockScience Dec 08 '10 at 10:03
3

I've read Dynamic Linear Models with R (good book), the final chapter deals with sequential Monte Carlo / particle filtering. It also includes some R code; however, in the concluding remarks of chapter 5 they explicitly warn that SMC becomes increasingly unreliable as additional time passes because the errors accumulate. Thus, they recommend "refreshing" the particle filter with the posterior distribution from a full MCMC sample every $T$ periods. Perhaps I misread their warnings, but this would seem to imply that you are better off with the rolling window MCMC. However, I would think there are substantial computer processing constraints with that method. For example, assuming you had 1,000 different univariate time series with 50 observations each and it took you 10 minutes to run a full MCMC Gibbs sampler. Then, it would take you 340 days ($(1000 \times (50-1) \times 10) \div 60 \div 24$) of continuous processing to estimate the parameters without look-ahead bias. Maybe my estimate of the time it takes to run the MCMC is wildly off, but I think it's a conservative but reasonable estimate.

It's been several years since you asked the question, so I'd be curious if you yourself have an answer now.

Ursus Frost
  • 190
  • 7