Question

A Markov chain has the transition probability matrix

0 1 2
0||0.7 0.2 0.1||
1||0.3 0.5 0.2||
2|| 0 0 1 ||

The Markov chain starts at time zero in state X ₀=0 let
T=min {n≥0;X ₙ=2}

be the first time that the process reaches state 2 . Eventually, the process will reach and be absorbed into state 2. If in some experiment we observed such a process and noted that absorption had not yet taken place, we might be interested in the conditional probability that the process in in state 0 (or 1 ), given that absorption had not yet taken place. Determine Pr X ₀ T>3.

Hint : the event {T>3} is exactly the same as the event { X ₃ = 1}

Answer 1

The problem involves calculating the conditional probability of being in state 0 at time 3 given non-absorption by time 3 in a Markov chain. we utilize the given transition probability matrix and the law of total probability to solve it.

Step-by-step explanation:

The student's question involves finding the conditional probability that the Markov chain is in state 0 at time 3 given that absorption has not occurred by that time. the transition probability matrix is given and we are told that the process starts at state 0 with the goal of reaching state 2 for the first time, denoted as T. absorption occurs when the process reaches state 2. We use the fact that {T > 3} is the same as {X₃ = 1} to focus on states 0 and 1 only, as state 2 cannot have been reached.

To calculate Pr T > 3, we can set up the total probabilities of being in state 0 or state 1 at time 3 and then use these probabilities to find the desired conditional probability. we use the transition probability matrix and the law of total probability to calculate the probabilities of being in each state at time 3 without having been absorbed and then apply the conditional probability formula.