Question

A particle moves on a circle through points which been marked 0,1,2,3,4 (in a clockwise order). At each step it has a probability p of moving to the right (clockwise) and 1- p to the left (counterclockwise). Let Xn denote its location on the circle after nth step. The process { Xn, n>= 0} is a Markov Chain a. Find the transition probability matrix. b. Calculate the limiting probabilities.

Answer 1

Explanation:

Given data:

SS={0,1,2,3,4}

Let probability of moving to the right be = P

Then probability of moving to the left is =1-P

The transition probability matrix is:

`\left[\begin{array}{ccccc}1&P&0&0&0\\1-P&1&P&0&0\\0&1-P&1&P&0\\0&0&1-P&1&P\\0&0&0&1-P&1\end{array}\right]`

Calculating the limiting probabilities:

π0=π0+Pπ1 eq(1)

π1=(1-P)π0+π1+Pπ2 eq(2)

π2=(1-P)π1+π2+Pπ3 eq(3)

π3=(1-P)π2+π3+Pπ4 eq(4)

π4=(1-P)π3+π4 eq(5)

π0+π1+π2+π3+π4=1

π0-π0-Pπ1=0

→π1 = 0

substituting value of π1 in eq(2)

(1-P)π0+Pπ2=0

from

π2=(1-P)π1+π2+Pπ3

we get

(1-P)π1+Pπ3 = 0

from

π3=(1-P)π2+π3+Pπ4

we get

(1-P)π2+Pπ4 =0

from π4=(1-P)π3+π4

→π3=0

substituting values of π1 and π3 in eq(3)

→π2=0

Now

π0+π1+π2+π3+π4=0

π0+π4=1

π0=0.5

π4=0.5

So limiting probabilities are {0.5,0,0,0,0.5}