Question

Suppose that the multiple linear regression model yi=xiTβ+ei holds for i=1,…,n, where the ei are independent errors, with E(ei∣X=xi)=0 but, unlike the standard setting, with different variances Var(ei∣X=xi)=σi2. a) Show that the OLS estimate β^=(XTX)−1XTY is still an unbiased estimate of β. That is, show that E[β^∣X]=β under the model assumptions. b) Write the variance matrix of the errors ei as a diagonal matrix, say Σ, with diagonal elements (σ12,…,σn2). Show that Var(β^)=(XTX)−1XTΣX(XTX)−1.

Answer 1

a) The OLS estimate β^=(XTX)−1XTY is still an unbiased estimate of β. b) The variance matrix of the errors ei can be written as a diagonal matrix Σ with diagonal elements (σ12,…,σn2).

Step-by-step explanation:

a) To show that the OLS estimate β^=(XTX)−1XTY is an unbiased estimate of β, we need to show that E[β^∣X]=β. Using the definition of the OLS estimate, we have:

β^=(XTX)−1XTY

Taking the expectation, we get:

E[β^∣X]=E[(XTX)−1XTY∣X]

=E[(XTX)−1XT(Xβ+e)∣X]

=E[(XTX)−1XTXβ∣X]+E[(XTX)−1XTe∣X]

=β+E[(XTX)−1XTe∣X]

Since E[(XTX)−1XTe∣X] is equal to zero under the given assumptions (independence and zero conditional mean), we have:

E[β^∣X]=β

b) To write the variance matrix of the errors ei as a diagonal matrix Σ, we use the definition of Σ and the given information:

Σ=diag(σ12,…,σn2)

To find the variance of β^, we use the formula:

Var(β^)=(XTX)−1XTΣX(XTX)−1