Question

Cauchy distribution and rejection sampling We have seen in the lectures that we can not sample Cauchy random variables using a normal density as enveloppe function or proposal distribution. However, it is possible to generate standard normal random variables from Cauchy.

(a) Justify why the Cauchy distribution is an appropriate proposal distribution, and find the optimal value of M. [ 3 marks]
(b) Sample 100000 samples u from a uniform distribution, and re-use your sample of 100000y values from a Cauchy distribution obtained in the previous part (use rcauchy if you did not manage to make the previous part work). Derive the condition that u and y must satisfy in order for a single value yᵢ to be accepted as a sample from a standard normal distribution. [ 3 marks]
(c) Apply that methodology to generate samples from a standard normal distribution. Plot a normalised histogram of these realisations. Add a line showing the probability density function of a standard normal distribution. [4 marks] (d) We also said that we have a probability 1/M for a single sample from the proposal distribution to be accepted as a sample from the target distribution. Show that it was verified (approximately) in your rejection sampler in this exercise. [2 marks]

Answer 1

The Cauchy distribution is an appropriate proposal distribution for generating samples from the standard normal distribution. The acceptance condition for a single value from the Cauchy distribution to be considered a sample from the standard normal distribution is |u| ≤ M/(π(1 + y²)). To generate samples from the standard normal distribution, we use a rejection sampling methodology and plot a normalized histogram of the accepted samples alongside the probability density function of the standard normal distribution. Finally, we verify the probability of acceptance using the acceptance rate calculated from the generated samples.

Step-by-step explanation:

(a) Justifying the use of the Cauchy distribution as a proposal distribution

The Cauchy distribution is an appropriate proposal distribution for generating samples from a target distribution, in this case, the standard normal distribution.

This is because the Cauchy distribution has heavy tails, which allows it to sample values in the tails of the target distribution effectively.

Additionally, the Cauchy distribution has an infinite variance, which allows it to cover a wider range of values compared to the normal distribution. The optimal value of M, which is the upper bound for the ratio of the target distribution to the proposal distribution, is 1.

(b) Deriving the acceptance condition for a single value yᵢ to be accepted as a sample from a standard normal distribution

The condition for accepting a single value yᵢ from the Cauchy distribution as a sample from a standard normal distribution is that the corresponding value uᵢ from the uniform distribution must satisfy the following condition: |uᵢ| ≤ M/(π(1 + yᵢ²)).

(c) Applying the methodology to generate samples from a standard normal distribution

To generate samples from a standard normal distribution using rejection sampling, we sample 100,000 values u from a uniform distribution and use a sample of 100,000 values y obtained from the Cauchy distribution.

We accept a value yᵢ as a sample from the standard normal distribution if |uᵢ| ≤ M/(π(1 + yᵢ²)). We then plot a normalized histogram of these accepted values and add a line showing the probability density function of the standard normal distribution.

(d) Verifying the probability of acceptance in the rejection sampler

To verify the probability 1/M for a single sample from the proposal distribution to be accepted as a sample from the target distribution, we calculate the acceptance rate by dividing the number of accepted samples by the total number of samples generated.

We compare this acceptance rate with the expected value of 1/M. If the acceptance rate is close to 1/M, then the probability 1/M is approximately verified in the rejection sampler.