Final answer:
The visits to a transient state in a Markov chain form a geometric progression because each visit can be framed as a 'success' in a geometric distribution with the probability of the first visit being p and subsequent visits being reduced by the probability of failure q, or (1 - p), each time.
Step-by-step explanation:
To understand how the number of visits to a transient state in a Markov chain follows a geometric progression, we need to delve into the concept of geometric distribution, which is intimately related to the Bernoulli trials and can be applied to Markov chains.
A transient state in a Markov chain is one that, once left, may never be returned to. In other words, it's not an absorbing state. The ability to return to a transient state or not, and the number of times this can happen, if at all, adhere to a rule of probability that can be described by a geometric distribution. By definition, a geometric distribution models the number of trials up to and including the first success in a series of independent Bernoulli trials. Notably, each trial has only two possible outcomes: 'success' or 'failure' and the probability of success, denoted by p, remains constant for each trial.
The occurrence of visiting a transient state can be framed as a 'success' in a geometric distribution. The probability of visiting the state for the first time is p, and, similarly, the probability of not visiting the state in a given step is q where q = 1 - p. Hence, if the probability of visiting a transient state at the first step is p, then the probability of visiting it at the second step for the first time (having failed to visit in the first step) would be p*q, for the third step it is p*q^2, and so on, forming a geometric progression. Gradually, as we extend this scenario, we see that the chances of visiting the state decrease exponentially - characteristic of a geometric progression.
The properties of geometric distribution
Geometric distribution has three main characteristics:
- Trials are independent Bernoulli trials with one success, which is the last one.
- The trials can theoretically continue indefinitely.
- The probabilities of success, p, and failure, q, are constant for each trial.
The mean (μ) and standard deviation (σ) of such a distribution can be calculated using the formulas μ = (1/p) and σ = √(q/p^2), respectively. These formulas give us further insight into the nature of the geometric distribution and its reliance on the probability p.