2

Full Problem: A continuous-time Markov chain has generator matrix

$$Q= \begin{pmatrix} -1 & 1 & 0 \\ 1 & -2 & 1 \\ 2 & 2 & -4 \\ \end{pmatrix} $$

(i) Exhibit the transition matrix of the embedded Markov chain.

(ii) Exhibit the holding time parameters for each state.

OK, I must be misunderstanding something. I have the following for the embedded chain transition probabilities:

$$P_{ij} = \frac{q_{ij}}{q_i}$$

where $q_i$ is the parameters of the exponential length of time that the process stays in i.

The problem is that I know $q_i$ is the minimum with exponential distribution and parameter $\sum_k q_{ik}$.

But when I calculate $P_{11}$ that gives me $-1$ which is impossible..

Obviously there is an error in my thinking but can anyone shine light on what I'm misunderstanding? Thank you greatly in advance.

Nicklovn
  • 741
  • 5
  • 18
  • I discovered my mistake. The diagonals are always zero. I'll leave this here in case anyone finds themselves equally confused. – Nicklovn Apr 23 '18 at 14:44
  • It's unclear what you mean by "the diagonals are always zero," because that's not the case either with $Q$ or with the answer to (i). – whuber Apr 23 '18 at 15:41
  • No. The diagonals of the transition matrix of the embedded Markov chain are supposed to be zero because there is no self-transition in a continuous state markov chain. Also, I don't consider this a duplicate. It is a different method of deriving the transition matrix of the embedded M.C. that does not require diagonalization. – Nicklovn Apr 23 '18 at 22:05
  • 1
    The only way your readers would understand that would be for you to edit your post to elaborate on these requirements and restrictions. – whuber Apr 23 '18 at 22:14
  • Is there a case where transition to the same state is possible in continuous time Markov Chains? My understanding was that you cannot have self-transition trivially explained by the fact that a same state transition is simply the state staying in the starting state for exponentially distributed time. – Nicklovn Apr 23 '18 at 22:21
  • 1
    It seems like you might be confusing a generator with the transition matrix itself. I'm not reading anything that distinguishes your question (in its current form) from the duplicate. – whuber Apr 23 '18 at 22:28
  • 1
    No no. I see where the confusion stems from. The question you linked to mine is asking for $P(t)$ which is the transition function. I was asking how to find the embedded chain transition matrix. I.e. the transition function with 0 waiting time. – Nicklovn Apr 23 '18 at 22:33
  • Is this not a duplicate of https://stats.stackexchange.com/questions/340948/continuous-markov-chains-deriving-the-transition-matrix-from-the-generator-matr ? – Math1000 May 10 '18 at 09:37

1 Answers1

2

Indeed $P_{ij} = \frac{q_{ij}}{q_j}$ for $i\ne j$, so the transition matrix for the embedded Markov chain is $$ P=\pmatrix{ 0&1&0\\ \frac12&0&\frac12\\ \frac12&\frac12&0 }. $$ The holding time for state $0$ has $\mathsf{Exp}(1)$ distribution since $q_{01}=1$ and $q_{02}=2$. The holding time for state $1$ has $\mathsf{Exp}(2)$ since $q_{10}=q_{12}=1$ and the minimum of two independent exponential random variables is again exponential with rate equal to the sum of the two. A similar argument shows that the holding time for state $2$ has $\mathsf{Exp}(4)$ distribution.

Math1000
  • 649
  • 4
  • 11