Question

an individual has three umbrellas, some at her office, and some at home. if she is leaving home in the morning (or leaving work at nigh) and it is raining she will take an umbrella, if one is there. otherwise, she gets wet. assume that independent of the past, it rains on each trip with probability 0.2. to formulate a markov chain, let xn be the number of umbrellas at her current location. (a) find the transition probability for this markov chain. (b) calculate the limiting fraction

Answer 1

(a) The probability of getting wet is 0.2 (rain), and the probability of staying dry is 0.8 (no rain).

(b) The limiting fraction (stationary distribution) for this Markov chain is
`\([0, 0, 0, 1]\)`.

How to find probability and limiting fraction?

To determine the transition probabilities:

(a) Transition Probabilities:

If Xₙ = 0, she is without an umbrella. The probability of getting wet is 0.2 (rain), and the probability of staying dry is 0.8 (no rain). So, the transition probabilities are:

`\[P(X_(n+1) = 0 | X_n = 0) = 0.8\]`

`\[P(X_(n+1) = 1 | X_n = 0) = 0.2\]`

If Xₙ = 1, she has one umbrella. The probability of getting wet is 0.2 (rain), and the probability of staying dry is 0.8 (no rain). The transition probabilities are:

`\[P(X_(n+1) = 0 | X_n = 1) = 0.2\]`

`\[P(X_(n+1) = 1 | X_n = 1) = 0.8\]`

`\[P(X_(n+1) = 2 | X_n = 1) = 0.2\]`

If Xₙ = 2, she has two umbrellas. The probability of getting wet is 0.2 (rain), and the probability of staying dry is 0.8 (no rain). The transition probabilities are:

`\[P(X_(n+1) = 1 | X_n = 2) = 0.2\]`

`\[P(X_(n+1) = 2 | X_n = 2) = 0.8\]`

`\[P(X_(n+1) = 3 | X_n = 2) = 0.2\]`

If Xₙ = 3, she has three umbrellas. The probability of getting wet is 0.2 (rain), and the probability of staying dry is 0.8 (no rain). The transition probabilities are:

`\[P(X_(n+1) = 2 | X_n = 3) = 0.2\]`

`\[P(X_(n+1) = 3 | X_n = 3) = 0.8\]`

(b) Limiting Fraction:

To calculate the limiting fraction, find the stationary distribution. This involves solving the system of equations πP = π,

where π = stationary distribution vector and

P = transition probability matrix.

The limiting fraction is the stationary distribution vector π. Solving the system of equations:

The transition probability matrix P is as follows:

`\[ P = \begin{bmatrix} 0.8 & 0.2 & 0 & 0 \\ 0.2 & 0.8 & 0.2 & 0 \\ 0 & 0.2 & 0.8 & 0.2 \\ 0 & 0 & 0.2 & 0.8 \end{bmatrix} \]`

Let
`\( \pi = [p_0, p_1, p_2, p_3] \)`,

where π is the limiting fraction for state i.

The system of equations is given by:

`\[ p_0 \cdot 0.8 + p_1 \cdot 0.2 = p_0 \]`

`\[ p_0 \cdot 0.2 + p_1 \cdot 0.8 + p_2 \cdot 0.2 = p_1 \]`

`\[ p_1 \cdot 0.2 + p_2 \cdot 0.8 + p_3 \cdot 0.2 = p_2 \]`

`\[ p_2 \cdot 0.2 + p_3 \cdot 0.8 = p_3 \]`

Now, solve this system of equations. Starting with the first equation:

`\[ 0.8p_0 + 0.2p_1 = p_0 \]`

`\[ 0.2p_1 = 0.2p_0 \]`

`\[ p_1 = p_0 \]`

Moving on to the second equation:

`\[ 0.2p_0 + 0.8p_1 + 0.2p_2 = p_1 \]`

`\[ 0.2p_0 + 0.8p_0 + 0.2p_2 = p_0 \]`

`\[ 1.0p_0 + 0.2p_2 = p_0 \]`

`\[ 0.2p_2 = 0.0 \]`

`\[ p_2 = 0.0 \]`

Now, the third equation:

`\[ 0.2p_1 + 0.8p_2 + 0.2p_3 = p_2 \]`

`\[ 0.2p_0 + 0.8p_0 + 0.2p_3 = 0.0 \]`

`\[ 1.0p_0 + 0.2p_3 = 0.0 \]`

`\[ 0.2p_3 = -1.0p_0 \]`

`\[ p_3 = -5.0p_0 \]`

Finally, the fourth equation:

`\[ 0.2p_2 + 0.8p_3 = p_3 \]`

`\[ 0.2 \cdot 0 + 0.8 \cdot (-5.0p_0) = -5.0p_0 \]`

`\[ -4.0p_0 = -5.0p_0 \]`

`\[ p_0 = 0 \]`

Now, since
`\( p_0 = 0 \)`,
`\( p_1 = 0 \)`, and
`\( p_2 = 0 \)`, normalize the vector to make it a valid probability distribution:

`\[ \pi = [0, 0, 0, 1] \]`

So, the limiting fraction (stationary distribution) for this Markov chain is
`\([0, 0, 0, 1]\)`.