In this chapter, it will be necessary to find the closest point on a subspace to a given point, like so:
The closest point has the property that the difference between the two points is orthogonal, or perpendicular, to the subspace. For this reason, we need to develop notions of orthogonality, length, and distance.
The basic construction in this section is the dot product, which measures angles between vectors and computes the length of a vector.
The dot product of two vectors in is
Thinking of as column vectors, this is the same as
For example,
Notice that the dot product of two vectors is a scalar.
You can do arithmetic with dot products mostly as usual, as long as you remember you can only dot two vectors together, and that the result is a scalar.
Let be vectors in and let be a scalar.
The dot product of a vector with itself is an important special case:
Therefore, for any vector we have:
This leads to a good definition of length.
The length of a vector in is the number
It is easy to see why this is true for vectors in by the Pythagorean theorem.
For vectors in one can check that really is the length of although now this requires two applications of the Pythagorean theorem.
Note that the length of a vector is the length of the arrow; if we think in terms of points, then the length is its distance from the origin.
If is a vector and is a scalar, then
This says that scaling a vector by scales its length by For example,
Now that we have a good notion of length, we can define the distance between points in Recall that the difference between two points is naturally a vector, namely, the vector pointing from to
The distance between two points in is the length of the vector from to
Vectors with length are very common in applications, so we give them a name.
A unit vector is a vector with length
The standard coordinate vectors are unit vectors:
For any nonzero vector there is a unique unit vector pointing in the same direction. It is obtained by dividing by the length of
Let be a nonzero vector in The unit vector in the direction of is the vector
This is in fact a unit vector (noting that is a positive number, so ):
In this section, we show how the dot product can be used to define orthogonality, i.e., when two vectors are perpendicular to each other.
Two vectors in are orthogonal or perpendicular if
Notation: means
Since for any vector the zero vector is orthogonal to every vector in
We motivate the above definition using the law of cosines in In our language, the law of cosines asserts that if are two nonzero vectors, and if is the angle between them, then
In particular, if and only if which happens if and only if Therefore,
To reiterate:
