In Section 4.1 we learned to multiply matrices together. In this section, we learn to “divide” by a matrix. This allows us to solve the matrix equation in an elegant way:

One has to take care when “dividing by matrices”, however, because not every matrix has an inverse, and the order of matrix multiplication is important.

Subsection4.5.1Invertible Matrices

The reciprocal or inverse of a nonzero number is the number which is characterized by the property that For instance, the inverse of is We use this formulation to define the inverse of a matrix.

Definition

Let be an (square) matrix. We say that is invertible if there is an matrix such that

In this case, the matrix is called the inverse of and we write

We have to require and because in general, matrix multiplication is not commutative. However, we will show in the invertible matrix theorem that if and are matrices such that then automatically

There exist non-square matrices whose product is the identity. Indeed, if

then However, so does not deserve to be called the inverse of

One can show using the ideas in this subsection that if is an matrix for then there is no matrix such that and For this reason, we restrict ourselves to square matrices when we discuss matrix invertibility.

Facts about invertible matrices

Let and be invertible matrices.

is invertible, and its inverse is

is invertible, and its inverse is (note the order).

Proof

The equations and at the same time exhibit as the inverse of and as the inverse of

We compute
Here we used the associativity of matrix multiplication and the fact that This shows that is the inverse of

Why is the inverse of not equal to If it were, then we would have

But there is no reason for to equal the identity matrix: one cannot switch the order of and so there is nothing to cancel in this expression.

More generally, the inverse of a product of several invertible matrices is the product of the inverses, in the opposite order; the proof is the same. For instance,

Subsection4.5.2Computing the Inverse Matrix¶ permalink

So far we have defined the inverse matrix without giving any strategy for computing it. We do so now, beginning with the special case of matrices.

Suppose that Define Then
The reader can check that so is invertible and

Suppose that Let be the matrix transformation Then
If is the zero matrix, then it is obviously not invertible. Otherwise, one of and will be a nonzero vector in the null space of Suppose that there were a matrix such that Then
which is impossible as Therefore, is not invertible.

There is an analogous formula for the inverse of an matrix, but it is not as simple, and it is computationally intensive. The interested reader can find it in subsection in Section 5.2.

The following theorem gives a procedure for computing in general.

Theorem

Let be an matrix, and let be the matrix obtained by augmenting by the identity matrix. If the reduced row echelon form of has the form then is invertible and Otherwise, is not invertible.

First suppose that the reduced row echelon form of does not have the form This means that fewer than pivots are contained in the first columns (the non-augmented part), so has fewer than pivots. It follows that (the equation has a free variable), so there exists a nonzero vector in Suppose that there were a matrix such that Then

which is impossible as Therefore, is not invertible.

Now suppose that the reduced row echelon form of has the form In this case, all pivots are contained in the non-augmented part of the matrix, so the augmented part plays no role in the row reduction: the entries of the augmented part do not influence the choice of row operations used. Hence, row reducing is equivalent to solving the systems of linear equations where are the standard coordinate vectors:

The columns of the matrix in the row reduced form are the solutions to these equations:

The advantage of solving a linear system using inverses is that it becomes much faster to solve the matrix equation for other, or even unknown, values of For instance, in the above example, the solution of the system of equations

where are unknowns, is

Subsection4.5.4Invertible linear transformations¶ permalink

As with matrix multiplication, it is helpful to understand matrix inversion as an operation on linear transformations. Recall that the identity transformation on is denoted

Definition

A transformation is invertible if there exists a transformation such that and In this case, the transformation is called the inverse of and we write

The inverse of “undoes” whatever did. We have

for all vectors This means that if you apply to then you apply you get the vector back, and likewise in the other order.

To say that is one-to-one and onto means that has exactly one solution for every in

Suppose that is invertible. Then always has the unique solution indeed, applying to both sides of gives

and applying to both sides of gives

Conversely, suppose that is one-to-one and onto. Let be a vector in and let be the unique solution of Then defines a transformation from to For any in we have because is the unique solution of the equation for For any in we have because is the unique solution of Therefore, is the inverse of and is invertible.

Suppose now that is an invertible transformation, and that is another transformation such that We must show that i.e., that We compose both sides of the equality on the left by and on the right by to obtain

We have and so the left side of the above equation is Likewise, and so our equality simplifies to as desired.

If instead we had assumed only that then the proof that proceeds similarly.

It makes sense in the above definition to define the inverse of a transformation for to be a transformation such that and In fact, there exist invertible transformations for any and but they are not linear, or even continuous.

If is a linear transformation, then it can only be invertible when i.e., when its domain is equal to its codomain. Indeed, if is one-to-one, then by this note in Section 4.2, and if is onto, then by this note in Section 4.2. Therefore, when discussing invertibility we restrict ourselves to the case

Suppose that is invertible. Let be the inverse of We claim that is linear. We need to check the defining properties in Section 4.3. Let be vectors in Then

Now that we know that is linear, we know that it has a standard matrix By the compatibility of matrix multiplication and composition in Section 4.4, the matrix for is But is the identity transformation and the standard matrix for is so One shows similarly that Hence is invertible and

This subsection consists of a single important theorem, with many equivalent conditions for a matrix to be invertible. This is one of the most important theorems in this textbook. We will append two more criteria in Section 6.1.

Invertible Matrix Theorem

Let be an matrix, and let be the matrix transformation The following statements are equivalent:

is invertible.

is invertible.

The reduced row echelon form of is the identity matrix

The only reduced row echelon form matrix with pivots is

The first three are translations of each other, because is the set of solutions of and is the dimension of The equivalence of 5 and 8 results from this theorem in Section 3.5, and the equivalence of 5 and 10 results from this theorem in Section 4.2. The basis theorem in Section 3.9 implies the equivalence of 8 and 9, since any set of linearly independent vectors in forms a basis.

Assertions 13, 14, and 17 are translations of each other, because is the span of the columns of which is equal to the range of The equivalence of 11 and 13 follows from this theorem in Section 3.3. Since the only -dimensional subspace of is all of so 14, 15, and 16 are equivalent.

Since has rows and columns, it has a pivot in every row (resp. column) if and only if it has pivots. By this theorem in Section 3.5, there is a pivot in every column if and only if the columns are linearly independent, and by this theorem in Section 3.3, there is a pivot in every row if and only if the rows span

By this proposition, the transformation is invertible if and only if it is both one-to-one and onto, so 2 is equivalent to (10 and 17). We have already shown so and are equivalent to each other; thus they are both equivalent to 2.

At this point, we have demonstrated the equivalence of assertions 1–17.

Invertibility of means there exists a matrix such that and so 1 implies 18. Now we prove directly that is onto. Let be a vector in and let Then

so has at least one solution.

Invertibility of means there exists a matrix such that and so 1 implies 19. Now we prove directly that is one-to-one. Suppose that and are vectors such that

Therefore, so is one-to-one.

Since 10 and 17 both imply 1, we have finished the proof.

There are two kinds of square matrices:

invertible matrices, and

non-invertible matrices.

For invertible matrices, all of the statements of the invertible matrix theorem are true.

For non-invertible matrices, all of the statements of the invertible matrix theorem are false.