Problem 1: On any given day, a student is either healthy or ill. Of the students who are healthy today, 95% will be healthy tomorrow. Of the students who are ill today, 55% will still be ill tomorrow. a. What is the stochastic matrix for this situation? b. Suppose 20% of the students are ill on Monday. What fraction or percentage of the students are likely to be ill on Tuesday? On Wednesday? c. If a student is healthy today, what is the probability that he or she will be healthy two days from now? d. Find the steady state vector for the Markov chain. e. What is the probability that after many days a specific student is ill? Does it matter if that person is ill today?
Solution: a. The stochastic matrix is P=\begin{bmatrix}95\% & 45\% \\ 5\% & 55\%\end{bmatrix}. b. The probability vector for Monday is \mathbf{x_0}=\begin{bmatrix}80\%\\ 20\%\end{bmatrix} and the probability for Tuesday is going to be P\mathbf{x_0}=\begin{bmatrix}85\%\\ 15\%\end{bmatrix}. c. Because we only consider if one specific student is healthy or not, the probability vector for today is just \mathbf{y_0}=\begin{bmatrix}100\%\\ 0\%\end{bmatrix}. Then two days from now, the probability vector is going to be P^2\mathbf{y_0}=\begin{bmatrix}92.5\%\\ 7.5\%\end{bmatrix}. d. The steady state vector for the Markov chain is given by P\mathbf{x}=\mathbf{x}. The solution is x_1=9x_2. Noting that x_1+x_2=1, x_1=90\%, x_2=10\%. e. The long-term behavior of Markov chain does not depend on its initial probability vector. In fact, after many days, a specific student is ill at a 10% chance and it does not matter if he or she is ill today.