Question

The goal of this exercise is to find a least square solutions of a matrix equation Ax=b

[0 1] [-1]
where A= [1 1] b= [-2]
[-1 -1] [-1]
by calculating the projection of the vector b onto the column space of the matrix A.
Compute an orthogonal basis of the matrix A and use to compute the vector projcol(a)b , i.e. the orthogonal projection of b onto the column space of A.

Answer 1

projcol(A)b = 1/2 (-b1 + b2, -b1 - b2) = (1, -3)

1. Orthogonal Basis of A:

First, we need an orthogonal basis for A. We can use Gram-Schmidt Orthogonalization:

Start with the first column of A: (0, 1). This is already a basis vector.

Subtract the projection of the second column onto the first to get the second basis vector: (1, 1) - ((1, 1) . (0, 1))/(0, 1).(0, 1))*(0, 1) = (-1, -1).

Therefore, our orthogonal basis for A is { (0, 1), (-1, -1) }.

2. Projection of b onto Col(A):

The projection of b onto the column space of A is given by:

projcol(A)b = (b . v1 / ||v1||^2) * v1 + (b . v2 / ||v2||^2) * v2

where v1 and v2 are the basis vectors from step 1.

Calculating the components:

b . v1 = (-1, -2) . (0, 1) = -2

b . v2 = (-1, -2) . (-1, -1) = 3

||v1||^2 = (0, 1).(0, 1) = 1

||v2||^2 = (-1, -1).(-1, -1) = 2

Therefore, the projection becomes:

projcol(A)b = (-2/1) * (0, 1) + (3/2) * (-1, -1) = (1, -3)

3. Least Squares Solution:

The least squares solution for Ax = b is given by:

x = (A^T A)^(-1) A^T b

Calculating the inverse and multiplying, we get:

x = ((1/2, -1/2), (-1/2, 1/2)) * ((-1, 1), (-2, -2)) * (-1, -2) = (1, -3)

This confirms that the projection of b onto Col(A) is also the least squares solution.

Therefore, the projection of b onto the column space of A and the least squares solution for Ax = b are both (1, -3).