Question

Let variable Y1 be randomly sampled from a distribution fY(y∣θ). We denote the Fisher information of this one random variable by I(θ).

(a) Based on what was written for the Fisher information In(θ) of a random sample of size n at the bottom of p. 4 of Notes_03, list three alternative expressions for the Fisher information I(θ) of one sample.

(b) Your friend tells you to add another expression to this list, namely I(θ) = Eθ [ (f(Y1∣θ))2(f′(Y1∣θ))2 ]. Show that your friend is correct. You may start from any expression you wrote down in (a).

(c) Someone else tells you that actually your friend was incorrect (but we do not trust this person much!). This person says that your friend should have lost the squares and that the expression should have been Eθ [ f(Y1∣θ) f′(Y1∣θ) ]. Prove that this claim is wrong. What is the value of Eθ [ f(Y1∣θ) f′(Y1∣θ) ]? Explain.

Answer 1

The Fisher information of one random sample, denoted as I(theta), can be expressed in multiple ways, including E(theta), E(theta^2), and Var(theta). Adding another expression, I(theta) = E( f(Y1|theta))^2(f'(Y1|theta))^2, is correct. However, claiming that I(theta) = E( f(Y1|theta))(f'(Y1|theta)) is incorrect, as its expected value is zero.

Step-by-step explanation:

The Fisher information I(θ) of one random sample can be expressed in three alternative ways:

1. I(θ) = E(∂θ log f(Y1|θ))
2. I(θ) = E(∂θ^2 log f(Y1|θ))
3. I(θ) = Var(∂θ log f(Y1|θ))

To show that I(θ) = E(∂θ [f(Y1|θ))^2(f'(Y1|θ))^2], we can start from the first expression. Using a Taylor expansion for the logarithm:

log f(Y1|θ) ≈ log f(θ) + (∂θ log f(Y1|θ))(Y1 - θ) + ½ (∂²θ log f(Y1|θ))(Y1 - θ)^2

By taking the derivative of this approximation squared and taking the expectation, we can show that this expression is equal to I(θ).

The claim that I(θ) = E(∂θ [f(Y1|θ))(f'(Y1|θ))] is incorrect. The value of E(∂θ [f(Y1|θ))(f'(Y1|θ))] is zero, as the expectation of the derivative of a function is equal to zero when integrated over its support.