Question

Let A be an m×n matrix of rank n and let P=A(A

T
A)
−1
A
T
. (a) Show that Pb=b for every b∈C(A). Explain this property in terms of projections. (b) If b∈C(A)
⊥
, show that Pb=0. (c) Give a geometric illustration of part (a) and (b) if C(A) is a plane through the origin in R
3
.

Answer 1

(a) To show that Pb = b for every b ∈ C(A), we need to demonstrate that the projection matrix P projects any vector b onto the column space of A, resulting in b itself.

First, let's consider the expression Pb. We have:

Pb = A(AᵀA)⁻¹Aᵀb

Since b ∈ C(A), there exists a vector x such that Ax = b. Substituting this into the equation, we get:

Pb = A(AᵀA)⁻¹Aᵀ(Ax)

= A(AᵀA)⁻¹(AᵀA)x

= Ax

= b

Therefore, Pb = b for every b ∈ C(A).

In terms of projections, this property means that the projection of any vector b onto the column space of A will result in the vector b itself if b is already in the column space of A. This is because the projection matrix P projects vectors onto the column space of A, and if b is already in that column space, the projection will preserve b.

(b) Now, let's consider the case when b ∈ C(A)⊥, which means b is orthogonal to the column space of A. We want to show that in this case, Pb = 0.

For b ∈ C(A)⊥, there exists a vector y such that Aᵀy = b. Let's substitute this into the expression for Pb:

Pb = A(AᵀA)⁻¹Aᵀb

= A(AᵀA)⁻¹Aᵀ(Aᵀy)

= A(AᵀA)⁻¹(AᵀA)y

= Ay

Since y is orthogonal to the column space of A, Ay = 0.

Therefore, Pb = Ay = 0 when b ∈ C(A)⊥.

This property means that if b is orthogonal to the column space of A, the projection of b onto the column space using the matrix P will result in the zero vector.

(c) Geometrically, in the case where C(A) is a plane through the origin in R³, part (a) can be illustrated as follows: If b is in the plane defined by C(A), projecting b onto the column space of A will simply result in b itself, as it is already in the plane.

In part (b), when b ∈ C(A)⊥, it means b is orthogonal to the plane defined by C(A). The projection of b onto the column space of A will result in the zero vector because b is perpendicular to the entire plane.

Visually, part (a) shows that the projection of a vector onto a plane it lies on results in the vector itself, while part (b) demonstrates that the projection of a vector orthogonal to the plane onto the plane results in the zero vector.

Explanation: