Learning a mapping from one time series to another with a Kalman Filter

Question

I am interested in finding the relation between two (possibly multi dimensional) time series $x_{1:T}$ and $y_{1:T}$. I wonder how I can do that with a linear dynamical system/Kalman filter.

My approach would be sth like estimating the parameters $\theta$ with EM for the joint probability $p(x_{1:T}, y_{1:T}, h_{1:T})$ where $h_{1:T}$ is my hidden state. Then say that I have observed a sequence $x'_{1:T}$, I want to determine the most likely sequence $y'_{1:T}$ by finding $p(y'_{1:T}|x'_{1:T})$.

Does anyone know whether this is the standard approach for this problem? If not, what is the standard approach? Does anyone know a source where this is described?

Answer 1

Since I believe to have found the solution myself, I am gonna post it here.

We are being given a hidden Markov model (as in HMM or LDS) with estimated transition and emission probabilities $p(h_t|h_{t-1})$ and $p(v_t|h_{t})$. In our case, $v_t$ is a concatenation of unobserved variables $u_t$ and observed ones $o_t$. The goal is to determine $p(u_t|o_{1:t})$. First we decompose this formula to go via $h_{1:t}$:

$$ p(u_t|o_{1:t}) = \sum_{h_{t}} p(u_t|h_t, o_{1:t})p(h_t| o_{1:t}) \\ = \sum_{h_{t}} p(u_t|h_t)p(h_t| o_{1:t}). \\ $$

We can find $p(u_t|h_t)$ by marginalizing over $o_t$. For $p(h_t| o_{1:t})$ we notice that it is similar to the filtering recursion:

$$ p(h_t| o_{1:t}) = \sum_{h_{t-1}} p(o_t|h_t)p(h_t|h_{t-1})p(h_{t-1}|o_{1:t-1}). $$

The first factor can be obtained by marginalization over $u_t$, the second is given, the third is a recursion.

For a linear dynamical system, $p(v_t|h_t)$ is Gaussian and thus marginalization is straightforward.

Answer 2

You should look at Shumway and Stoffer, Time Series Analysis and Its Applications, Chapter 6. And the code on their website.http://www.stat.pitt.edu/stoffer/tsa2/chap6.htm