Homework 4

Due: 2/6 (fri.)

Assignment

• Problem 1 (2 pts). Consider the Markov chain with states {0, 1, 2} and one step transition probability matrix:

        +-             -+
        |  0   0.8  0.2 |
  P  =  | 0.3  0.4  0.3 |
        | 0.5  0.1  0.4 |
        +-             -+
  

  1. Draw the state transition probability diagram
  2. Does the Markov chain have a unique steady-state probability distribution?
  3. What is P[ X_5 = 2 | X_2 = 0]? (Here, X_i means that "i" is a subscript. Hint: Compute P^3, where "^3" means "to the power 3".)
• Textbook Problem 3.5 (1 pt)
• Textbook Problem 3.6 (1 pt)
• Textbook Problem 3.11 (2 pt, the most difficult problem.)

Remember: No class next week 2/9 - 2/13