1

The definition of the Markov property is typically that the next state depends only on the present state and no past states. However, the mathematical definition I usually see (e.g. https://stats.stackexchange.com/a/2463/225553) says that P(X_t = x | X_{t-1}, X_{t-2}, X_{t-3}...X_t0) = P(X_t | X_{t-1}). That seems to say that the present state depends on only the immediate prior state. Are these equivalent definitions or am I misunderstanding the notation?

coderunner
  • 113
  • 2

1 Answers1

1

You correctly wrote... $$P(X_t = x | X_{t-1}, X_{t-2}, X_{t-3},...X_{t0}) = P(X_t | X_{t-1})$$ However this is equivalent to the following by virtue of shifting the time index by 1. $$P(X_{t+1} = x | X_{t}, X_{t-1}, X_{t-2},...X_{t0}) = P(X_{t+1} | X_{t})$$

In words: Given the present, the future is independent of the past.

Hope this helps. If not, feel free to comment and ask for clarification.

SecretAgentMan
  • 1,463
  • 10
  • 30
  • Thanks for answering. The shifted time index makes sense. I'm still a bit confused. Does this mean that $$P(X_t = x | X_{t-1}, X_{t-2}, X_{t-3},...X_{t0}) = P(X_t | X_{t-1})$$ $$X_{t-1}$$ is considered the "present"? I am still unsure if the statements "The future state depends only on the present state" is equivalent to "The present state depends only on the immediate past state". – coderunner Nov 02 '18 at 05:29
  • @coderunner, if you were at $t-1$ in time then at that moment it would be the present, right? Similarly, if you were at $t-5$ in time, $t-6$ would be the immediate past for the process and $t-4$ would be the future. I'm not sure if that helps... – SecretAgentMan Nov 02 '18 at 05:33
  • e.g. We have states A and B that directly lead to state C. Is it still correct to say the present state C depends only on the immediate prior state(s) A and B? If no, I think I am confusing myself from translating the math notation to english words. – coderunner Nov 02 '18 at 05:35
  • sorry about that. Had to add another comment because the first was running too long... – coderunner Nov 02 '18 at 05:35
  • The Markov property is talking about the *state of the process in time*, not just the states themselves. – SecretAgentMan Nov 02 '18 at 05:37
  • Ahh, I think that statement clears it up quite a bit for me. Thank you! – coderunner Nov 02 '18 at 05:43