I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author presents the following example:
Example 2.2. (Machine Reliability). The Depend-On-Us company manufactures a machine that is either up or down. If it is up at the beginning of a day, then it is up at the beginning of the next day with probability $.98$ (regardless of the history of the machine), or it fails with probability $.02$. Once the machine goes down, the company sends a repair person to repair it. If the machine is down at the beginning of a day, it is down at the beginning of the next day with probability $.03$ (regardless of the history of the machine), or the repair is completed and the machine is up with probability $.97$. A repaired machine is as good as new. Model the evolution of the machine as a DTMC.
Let $X_n$ be the state of the machine at the beginning of day $n$, defined as follows:
$$X_n = \begin{cases} 0 & \text{if the machine is down at the beginning of day $n$,} \\ 1 & \text{if the machine is up at the beginning of day $n$.} \end{cases}$$
The description of the system shows that $\{ X_n, n \ge 0 \}$ is a DTMC with state space $\{0, 1\}$ and the transition probability matrix
$$P = \begin{bmatrix} .03 & .97 \\ .02 & .98 \end{bmatrix} \tag{2.7}$$
Now suppose the company maintains two such machines that are identical and behave independently of each other, and each has its own repair person. Let $Y_n$ be the number of machines in the “up” state at the beginning of day $n$. Is $\{ Y_n, n \ge 0 \}$ a DTMC?
First we identify the state space of $Y_n, n \ge 0$ to be $\{ 0, 1, 2 \}$. Next we see if the Markov property holds; that is, we check if $P(Y_{n + 1} = j \mid Y_n = i, Y_{n - 1}, \dots, Y_0)$ depends only on $i$ and $j$ for $i, j = 0, 1, 2$. For example, consider the case $Y_n = i = 1$ and $Y_{n + 1} = j = 0$. Thus, one machine is up (and one down) at time $n$. Since both machines are identical, it does not matter which is up and which is down. In order to move to state $0$ at time $n + 1$, the down machine must stay down and the up machine must go down at the beginning of the next day. Since the machines are independent, the probability of this happening is $.03*.02$, independent of the history of the two machines. Hence we get
$$P(Y_{n + 1} = 0 \mid Y_n = 1, Y_{n - 1}, \dots, Y_0) = .03 * .02 = .0006 = p_{1, 0}$$
Proceeding in this fashion, we construct the following transition probability matrix:
$$P = \begin{bmatrix} .0009 & .0582 & .9409 \\ .0006 & .0488 & .9506 \\ .0004 & .0392 & .9604 \end{bmatrix}. \blacksquare \tag{2.8}$$
In the example, the author says the following:
Next we see if the Markov property holds; that is, we check if $P(Y_{n + 1} = j \mid Y_n = i, Y_{n - 1}, \dots, Y_0)$ depends only on $i$ and $j$ for $i, j = 0, 1, 2$.
But what I don't understand is where in this example it was verified that the Markov property holds? Some calculations were performed based on the fact that the random variables representing the two machines are independent, but it doesn't seem that any calculations were performed to prove that the Markov property holds?
I would greatly appreciate it if people would please take the time to clarify this.
