Question

Problem 3 (Is the MLE for i.i.d exponential data asymptotically normal?). Let Xi,1 Sisn, be i.i.d exponential with parameter > 0. (a) Does the support of this distribution depend on ? (b) Compute the maximum likelihood estimate for 1, î. = (c) Consider the function g(x) = 1/x. Construct a second order Taylor expansion of this function around the value 1/1 (why?), similar to the more general case you considered in problem on Taylor expansions from the previous homework. = (d) Suppose the true value of l is l = le. Use this Taylor expansion to determine the asymptotic distribution of Valî - do) (e) Compute the Fisher information I(10) and determine whether your answers to the previous part agree with the asymptotic normality results we described in class.

Answer 1

Okay, here are the steps to solve this problem:

(a) The support of an exponential distribution with parameter λ is (0, ∞). It does not depend on λ.

(b) The MLE for λ is the inverse of the sample mean:

î = n/∑Xi

(c) We will Taylor expand g(x) = 1/x around x = 1/λ0, where λ0 is the true value of λ.

g'(x) = -1/x2

g"(x) = 2/x3

Taylor expansion at x = 1/λ0:

g(x) ≈ g(1/λ0) + g'(1/λ0)(x - 1/λ0) + g"(1/λ0)(x - 1/λ0)2/2

1/x ≈ 1/λ0 - (1/λ02)(x - 1/λ0) + (2/λ03)(x - 1/λ0)2/2

(d) Plug in x = 1/î:

1/î ≈ 1/λ0 - (1/λ02)(1/î - 1/λ0) + (2/λ03)(1/î - 1/λ0)2/2

λ0î ≈ λ0 - (λ0)2(λ0 - î) + 2(λ0)3(λ0 - λ0)2/2

λ0î + (λ0)2(λ0 - î) - 2(λ0)2 ≈ 0

Solving for î - λ0 gives:

î - λ0 ≈ - (λ0)2/(2(n - λ0))

So (î - λ0) is asymptotically N(0, (λ0)2/(2(n)).

(e) The Fisher information is:

I(λ0) = E[-∂2/∂λ2 log L(X|λ) | λ = λ0]

= 2n/λ02

Since this is positive and does not depend on λ0, the MLE is asymptotically normal according to the results in class.

Does this look correct? Let me know if you have any other questions!

Answer 2

The student's question deals with exponential distributions and maximum likelihood estimation. The support of the exponential distribution does not depend on the parameter, the MLE for 1/λ is the reciprocal of the sample mean, and the estimator's distribution becomes normal as the sample size increases, which is confirmed using the Fisher information.

Step-by-step explanation:

Maximum Likelihood Estimate for Exponential Distribution

The question deals with the properties of the exponential distribution and the maximum likelihood estimation (MLE) for a dataset consisting of independent and identically distributed (i.i.d) variables from an exponential distribution with an unknown parameter λ. The solutions to the problems involve several steps:

• (a) The support of the exponential distribution does not depend on λ since it is always x ≥ 0.
• (b) The MLE for 1/λ, denoted by î, is calculated as the reciprocal of the sample mean of the data.
• (c) The Taylor expansion of g(x) = 1/x around the value 1/λ helps to linearize the function, which is useful for further calculations on the distribution of the estimator.
• (d) The asymptotic distribution of √(n)(î - λ₀) will have a normal distribution based on the properties of the MLE and the central limit theorem.
• (e) The Fisher information I(λ₀) is calculated to confirm that the asymptotic distribution of the estimator is indeed normal as seen in class.

The overall understanding of these concepts helps in statistical inference and understanding the behavior of estimators for large sample sizes.