Question

Consider a Markov chain on S = {0,1,....} with the following transition probabilities

p00 = 0,p01=1 pi0= i/i+1, pi,i+1 =1/i+1, i_>1

a) Find the stationary distribution of the chain.

b) Interpret the stationary distribution.

Answer 1

The stationary distribution of the Markov chain is π = {1, 1/2, 1/3, ...}.

Step-by-step explanation:

To find the stationary distribution of the Markov chain, we need to solve the equation π = πP, where π is the stationary distribution and P is the transition probability matrix. Let's find the values of π for each state:

• For state 0, we have π0 = π0 * p00 + π1 * p10, which simplifies to π0 = π1.
• For state i, where i > 0, we have πi = πi-1 * pi-1,i + πi * pi,i+1, which simplifies to πi = πi-1 / i and πi = πi+1 * (i+1).

From these equations, we can see that π0 = π1 = π2 = ... = 1, and πi = 1/i. Therefore, the stationary distribution is π = {1, 1/2, 1/3, ...}.