Skip to main content

# Section6.3Orthogonal Projection¶ permalink

##### Objectives
1. Understand the orthogonal decomposition of a vector with respect to a subspace.
2. Understand the relationship between orthogonal decomposition and orthogonal projection.
3. Understand the relationship between orthogonal decomposition and the closest vector on / distance to a subspace.
4. Learn the basic properties of orthogonal projections as linear transformations and as matrix transformations.
5. Recipes: orthogonal projection onto a line, orthogonal decomposition by solving a system of equations, orthogonal projection via a complicated matrix product.
6. Pictures: orthogonal decomposition, orthogonal projection.
7. Vocabulary words: orthogonal decomposition, orthogonal projection.

Let be a subspace of and let be a vector in In this section, we will learn to compute the closest vector to in The vector is called the orthogonal projection of onto This is exactly what we will use to almost solve matrix equations, as discussed in the introduction to Chapter 6.

# Subsection6.3.1Orthogonal Decomposition

We begin by fixing some notation.

##### Notation

Let be a subspace of and let be a vector in We denote the closest vector to on by

To say that is the closest vector to on means that the difference is orthogonal to the vectors in

In other words, if then we have where is in and is in The first order of business is to prove that the closest vector always exists.

##### Definition

Let be a subspace of and let be a vector in The expression

for in and in is called the orthogonal decomposition of with respect to and the closest vector is the orthogonal projection of onto

Since is the closest vector on to the distance from to the subspace is the length of the vector from to i.e., the length of To restate:

##### Closest vector and distance

Let be a subspace of and let be a vector in

• The orthogonal projection is the closest vector to in
• The distance from to is

Now we turn to the problem of computing and Of course, since really all we need is to compute The following theorem gives a method for computing the orthogonal projection onto a column space. To compute the orthogonal projection onto a general subspace, usually it is best to rewrite the subspace as the column space of a matrix, as in this important note in Section 2.6.

##### Proof

Let be the orthogonal decomposition with respect to By definition lies in and so there is a vector in with Choose any such vector We know that lies in which is equal to by this important note in Section 6.2. We thus have

and so

This exactly means that is consistent. If is any solution to then by reversing the above logic, we conclude that

##### Example(Orthogonal projection onto a line)

Let be a line in and let be a vector in By the theorem, to find we must solve the matrix equation where we regard as an matrix (the column space of this matrix is exactly ). But and so is a solution of and hence

To reiterate:

##### Recipe: Orthogonal projection onto a line

If is a line, then

for any vector

When is a matrix with more than one column, computing the orthogonal projection of onto means solving the matrix equation In other words, we can compute the closest vector by solving a system of linear equations. To be explicit, we state the theorem as a recipe:

##### Recipe: Compute an orthogonal decomposition

Let be a subspace of Here is a method to compute the orthogonal decomposition of a vector with respect to

1. Rewrite as the column space of a matrix In other words, find a a spanning set for and let be the matrix with those columns.
2. Compute the matrix and the vector
3. Form the augmented matrix for the matrix equation in the unknown vector and row reduce.
4. This equation is always consistent; choose one solution Then

In the context of the above recipe, if we start with a basis of then it turns out that the square matrix is automatically invertible! (It is always the case that is square and the equation is consistent, but need not be invertible in general.)

##### Proof

We will show that which implies invertibility by the invertible matrix theorem in Section 5.1. Suppose that Then so by the theorem. But (the orthogonal decomposition of the zero vector is just so and therefore is in Since the columns of are linearly independent, we have so as desired.

Let be a vector in and let be a solution of Then so

The corollary applies in particular to the case where we have a subspace of and a basis for To apply the corollary, we take to be the matrix with columns

# Subsection6.3.2Orthogonal Projection

In this subsection, we change perspective and think of the orthogonal projection as a function of This function turns out to be a linear transformation with many nice properties, and is a good example of a linear transformation which is not originally defined as a matrix transformation.

We compute the standard matrix of the orthogonal projection in the same way as for any other transformation: by evaluating on the standard coordinate vectors. In this case, this means projecting the standard coordinate vectors onto the subspace.

In the previous example, we could have used the fact that

forms a basis for so that

by the corollary. In this case, we have already expressed as a matrix transformation with matrix See this example.

Let be a subspace of with basis and let be the matrix with columns Then the standard matrix for is

We can translate the above properties of orthogonal projections into properties of the associated standard matrix.

We emphasize that the properties of projection matrices would be very hard to prove in terms of matrices. By translating all of the statements into statements about linear transformations, they become much more transparent. For example, consider the projection matrix we found in this example. Just by looking at the matrix it is not at all obvious that when you square the matrix you get the same matrix back.