Subsection Prepare
Weβll start by seeing how performing an elementary row operation can be done via matrix multiplication.
Example 2.5.1. Row operation and matrix multiplication.
Consider the matrix
\(\begin{bmatrix} 1 \amp 2 \\ 3 \amp 5 \end{bmatrix}\text{.}\)
The first step in Gauss-Jordan elimination is \(-3R_1+R_2 \rightarrow R_2\text{,}\)
\begin{align*}
\amp \amp \amp \begin{bmatrix} 1 \amp 2 \\ 3 \amp 5 \end{bmatrix}\\
-3R_1+R_2 \amp\rightarrow R_2 \amp \amp \begin{bmatrix} 1 \amp 2 \\ 0 \amp -1 \end{bmatrix}\text{.}
\end{align*}
However, we want to perform this row operation by finding a matrix to multiply by \(A\text{.}\) If we instead perform that same row operation on the \(2\times 2\) identity matrix, we get
\begin{align*}
\amp \amp \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix}\\
-3R_1+R_2 \amp\rightarrow R_2 \amp \amp \begin{bmatrix} 1 \amp 0 \\ -3 \amp 1 \end{bmatrix}\text{.}
\end{align*}
Call this matrix, \(E_1\text{,}\) so \(E_1=\begin{bmatrix} 1 \amp 0 \\ -3 \amp 1 \end{bmatrix}\text{.}\)
Now check that multiplying \(E_1\) by \(A\) gives us
\begin{align*}
E_1A\amp =\begin{bmatrix} 1 \amp 0 \\ -3 \amp 1 \end{bmatrix}\begin{bmatrix} 1 \amp 2 \\ 3 \amp 5 \end{bmatrix}\\
\amp = \begin{bmatrix} 1 \amp 2 \\ 0 \amp -1 \end{bmatrix} \text{,}
\end{align*}
which is the exact same effect as performing \(-3R_1+R_2\rightarrow R_2\) on \(A\) directly.
We just turned an operation on rows of a matrix into a matrix multiplication. You might be wondering, βDoes this work for other elementary row operations?β The answer is, βYes!β We can turn row operations into matrix multiplication by performing the row operation on the appropriately sized identity matrix.
Activity 2.5.2. Row operations and matrix multiplication.
There is a name for the kinds of matrices we have been considering.
Definition 2.5.3. Elementary Matrices.
Matrices which are obtained by performing one elementary row operation to the identity matrix are called
elementary matrices.
Example 2.5.4. Continuing Gauss-Jordan with matrix multiplication.
Letβs return to the matrix in
ExampleΒ 2.5.1 and continue our row reducing process. We already defined
\(E_1\) and calculated
\begin{equation*}
E_1A = \left[\begin{array}{rrr} 1 \amp 2 \\ 0 \amp -1 \end{array}\right]
\end{equation*}
The next step in Gauss-Jordan elimination is \(-R_2\rightarrow R_2\text{.}\) Doing this operation to the \(2 \times 2\) identity matrix yields
\begin{align*}
\amp \amp \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix}\\
-R_2 \amp\rightarrow R_2 \amp \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp -1 \end{bmatrix}\text{.}
\end{align*}
If we let \(E_2=\begin{bmatrix}1 \amp 0 \\ 0 \amp -1 \end{bmatrix}\text{,}\) and multiply \(E_2\) by the matrix we already had after multiplying by \(E_1\text{,}\) we get
\begin{align*}
E_2 (E_1A) \amp = \begin{bmatrix}1 \amp 0 \\ 0 \amp -1 \end{bmatrix} \begin{bmatrix} 1 \amp 2 \\ 0 \amp -1 \end{bmatrix} \\
\amp = \begin{bmatrix} 1 \amp 2 \\ 0 \amp 1 \end{bmatrix} \text{.}
\end{align*}
Thereβs one more row operation we need to do to finish row reducing \(A\text{.}\) Letβs create \(E_3\) by performing \(-2R_2+R_1 \rightarrow R_1\) on the \(2\times 2\) identity matrix
\begin{align*}
\amp \amp \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix}\\
-2R_2+R_1 \amp\rightarrow R_1 \amp \amp \begin{bmatrix} 1 \amp -2 \\ 0 \amp 1 \end{bmatrix}\text{,}
\end{align*}
and multiplying \(E_3\) by the matrix we had gives us
\begin{align*}
E_3 (E_2E_1A) \amp = \begin{bmatrix}1 \amp -2 \\ 0 \amp 1 \end{bmatrix} \begin{bmatrix} 1 \amp 2 \\ 0 \amp 1 \end{bmatrix} \\
\amp = \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix} \text{.}
\end{align*}
To recap what we just did: we row reduced the matrix \(A\) by multiplying on the left by three matrices in turn. In this case, the reduced row echelon form was the identity matrix, so we also found matrices whose product is the identity. Associativity means that
\begin{equation*}
(E_3E_2E_1)A=I\text{,}
\end{equation*}
which means that \(A\) is invertible and \(A^{-1}=E_3E_2E_1\text{.}\)
This is kind of a big deal. We said in
Calculating inverses of any size that in order to calculate inverses we augment by the identity matrix and then row reduce. If a matrix row reduces to
\(I\) on the left then the inverse appears on the right. Now we see another connection between inverses and row reducing: if a matrix row reduces to the identity, the inverse is the product of the elementary matrices which perform the row reduction.
It is possible to show that since the product
\(E_3E_2E_1\) is invertible (and it is since its inverse is
\(A\)) it is the case that
\(E_3\text{,}\) \(E_2\text{,}\) and
\(E_1\) are all invertible, too. Instead, letβs try to see why that has to be the case based on properties of elementary matrices.
Activity 2.5.5. Inverses of elementary matrices.
There are 3 kinds of elementary row operations and so there are 3 kinds of elementary matrices: ones which result from 1) scaling a row, 2) from interchanging two rows, and 3) from adding a multiple of a row to another row.
(a) Scaling operation inverses.
Drag each matrix below to its inverse. You can do this by multiplying and seeing which pairs have
\(I\) as their product, or by using an inverse solver, but pay attention to the patterns. What is true about the inverse of a matrix which scales a row by a number
\(c\text{?}\)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp \frac{1}{2} \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp -2 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ \frac{1}{2} \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rr} \frac{1}{5} \amp 0\\ 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rr} -5 \amp 0 \\ 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rr} -\frac{1}{5} \amp 0 \\ 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp \frac{1}{3} \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 3 \end{array}\right] \)
(b) Scaling inverse generally.
Say in your own words what the inverse (or opposite) is of scaling a row by a number
\(c\text{.}\)
(c) Interchanging operation inverses.
Drag each matrix below to its inverse. You can do this by multiplying and seeing which pairs have
\(I\) as their product, or by using an inverse solver, but pay attention to the patterns. What is true about the inverse of a matrix which swaps two rows?
- \(\left[\begin{array}{rrr} 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 0 \amp -1 \amp 0 \\ -1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rrr} 0 \amp -1 \amp 0 \\ 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 0 \amp 0 \amp 1 \\ 0 \amp 1 \amp 0 \\ 1 \amp 0 \amp 0 \end{array}\right] \)
(d) Interchanging inverse generally.
Say in your own words what the inverse is of interchanging two rows.
(e) Multiply-and-add operation inverses.
Drag each matrix below to its inverse. You can do this by multiplying and seeing which pairs have
\(I\) as their product, or by using an inverse solver, but pay attention to the patterns. What is true about the inverse of a matrix which multiplies a row by a number
\(c\) and adds the result to another row?
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ -2 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 2 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ \frac{1}{2} \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right]\)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 1 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp -1 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp \frac{1}{2} \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
- \(\left[\begin{array}{rrr} 1 \amp 0 \amp -\frac{1}{2} \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{array}\right] \)
(f) Multiply-and-add inverse generally.
Say in your own words what the inverse is of multiplying a row by
\(c\) and adding to another row.
Letβs return to our running example of this section. We started with
\begin{equation*}
A=\begin{bmatrix} 1 \amp 2 \\ 3 \amp 5 \end{bmatrix}
\end{equation*}
and through row reducing by multiplying by elementary matrices we found that
\begin{equation*}
E_3E_2E_1A=I\text{.}
\end{equation*}
Weβve seen that elementary row operations can be undone by another elementary row operation, which means that every elementary matrix is invertible, and the inverse is another elementary matrix. Thus we can multiply both sides of the above equation by \(E_3^{-1}\) and get that
\begin{align*}
E_3^{-1}(E_3E_2E_1A)\amp = E_3^{-1}I \\
(E_3^{-1}E_3)E_2E_1A\amp = E_3^{-1} \\
E_2E_1A\amp = E_3^{-1} \text{.}
\end{align*}
Next we undo \(E_2\) by multiplying both sides by \(E_2^{-1}\) to get
\begin{align*}
E_2^{-1}(E_2E_1A)\amp = E_2^{-1}E_3^{-1} \\
E_1A\amp = E_2^{-1}E_3^{-1} \text{.}
\end{align*}
Finally, we undo \(E_1\) in the same way and get
\begin{align*}
E_1^{-1}E_1A\amp = E_1^{-1}E_2^{-1}E_3^{-1} \\
A\amp = E_1^{-1}E_2^{-1}E_3^{-1} \text{.}
\end{align*}
Since the inverse of an elementary matrix is also an elementary matrix, we just wrote
\(A\) as a product of elementary matrices. Moreover, there wasnβt anything special about this example; the fact that the reduced row echelon form of
\(A\) is
\(I\) meant that we could multiply
\(A\) by elementary matrices to get
\(I\text{,}\) which meant that
\(A\) is invertible and also that
\(A\) can be expressed as the product of elementary matrices.
We have now seen a lot of different things that are equivalent to a matrix having an inverse (and there will be still more in the future). Letβs collect what we know so far.
Theorem 2.5.6. Equivalent to being invertible.
Let \(A\) be an \(n\times n\) matrix. The following are equivalent:
-
-
The reduced row echelon form of
\(A\) is
\(I\text{.}\)
-
There is exactly one solution (not 0 and not infinitely many) to any linear system that has
\(A\) as its coefficient matrix.
-
For any column matrix
\(X\) with
\(n\) rows, if
\(AX=\mathbf{0}\text{,}\) then
\(X=\mathbf{0}\text{.}\)
-
\(A\) is the product of elementary matrices.
A
theorem in math is a statement which has a proof that itβs true. We donβt get into full proofs, but we can still explore and understand some of the reasons for each of the above equivalences. We did a little of that just above the theorem, when we were seeing how row reducing to the identity led to invertibility and being able to be written as a product of elementary matrices, and weβll explore more during the Participate activities.
Reading Questions Reading Questions
1. Order of multiplication.
If you have found elementary matrices
\(E_1\) and
\(E_2\) which perform row operations, and you want to perform
\(E_1\) first on a matrix
\(M\) and then to the result perform
\(E_2\text{,}\) which product do you calculate?
\(E_1E_2M\)
-
Just like with functions \(f(g(x))\text{,}\) the \(g(x)\) comes second as we read left-to-right, but that means perform \(g(x)\) first and plug the result into \(f(x)\text{.}\)
So too with matrix multiplication. By associativity, \(E_1E_2M\) is the same as \(E_1(E_2M)\text{,}\) which means multiply \(E_2M\) and then multiply \(E_1\) by the result.
\(E_2E_1M\)
Correct!
2. Elementary matrices and invertibility.
Every elementary matrix is invertible, and the inverse of an elementary matrix is also an elementary matrix.
True.
The inverse of scaling is scaling by the reciprocal; the inverse of swapping two rows is swapping the same two rows again; the inverse of multiplying a row by \(c\) and adding to another row is multiplying the row by \(-c\) and adding.
False.
The inverse of scaling is scaling by the reciprocal; the inverse of swapping two rows is swapping the same two rows again; the inverse of multiplying a row by \(c\) and adding to another row is multiplying the row by \(-c\) and adding.
3. Reflection.
(a)
Reflect on your confidence level.
(b)
Ask a question about the material. What additional information do you think someone would need to become more confident in their understanding?