Suppose that we have a time-homogeneous discrete-time Markov chain $(X_n)$. We want to estimate the transition probabilities $p_{ij} = \mathbb{P}[X_{n+1} = j \mid X_n = i]$.
In the case when we have full information, e.g. we observe $(X_1, X_2, \ldots, X_N)$, the maximum likelihood estimator is $$ \hat{p}_{ij} = \frac{ n_{ij} }{ \sum_{k=1}^{m} n_{ik} }, $$ where $n_{ik}$ is the number of times that the process moved from state $i$ to $k$. (I took the expression above from https://stats.stackexchange.com/a/14361/22409)
Unfortunately, in my case we are not able to observe all the transitions. For example, we may observe $X_1, X_6, X_{11}, X_{13}, X_{14}, X_{15}, \ldots$, that is some of the states, but the states that we observe are not regularly spaced.
I have two related questions:
- Mathematically, how do we estimate the transition probabilities given such data observations?
- What statistical software is available for estimating the transition probabilities?
