0

My problem is extremely simple. But due to lack of proper documentation, I'm getting stuck.

I'm using UCM from statsmodel of python. I'm trying this example.

I want to create a dataset that has actual series, predicted series, level, trend and cycle component. I tried to use output_res.predict and output_res.fittedvalues option to get the predicted series. But I'm always getting 0 for the first observation.

Can someone please guide me how to get the final dataset?

Thanks!

Beta
  • 5,784
  • 9
  • 33
  • 44
  • Looks like this is essentially about using Python and as such off topic. – Richard Hardy Dec 18 '19 at 14:59
  • Its actually not completely off topic. This is because anyone wanting to use UCM using stat models want to create this dataset. And the doc does not explicity telling how to do it. Also, why when we are using `predict` option, we are getting `0` in the first value. Both of this is relevant. If I post it in stackoverflow, most probably no one will able to answer it. – Beta Dec 18 '19 at 15:02
  • 1
    I am not saying I find your question uninteresting or generally irrelevant; quite to the contrary. This is why I read it and commented under it. My perspective on the off-topicness remains, though. – Richard Hardy Dec 18 '19 at 16:26

1 Answers1

1

The short answer is: for non-stationary models you should ignore the first few observations (usually the number of elements of the state vector). It would probably be better if this value should be reported as np.nan instead of as zero.

Details:

First, notice that for the first prediction of the model you haven't observed any data set, so it is just the unconditional expectation $E[y_1]$. The observation equation of the state space model is written $y_t = Z \alpha_t + \varepsilon_t$, so $E[y_1] = Z E[\alpha_1]$. However, most unobserved components models are non-stationary, which means that $E[\alpha_1]$ does not exist.

To handle this case, Statsmodels uses a diffuse initialization, which implies setting $\alpha_1 \sim N(0, \kappa I)$ and letting $\kappa \to \infty$. But for any particular $\kappa$, then $E[\alpha_1] = 0$, and that's why you're getting the result that the first predicted value is always equal to zero.

cfulton
  • 1,193
  • 1
  • 6
  • 10