Similar to the properties we have seen for vector operations (in Lecture 6) and matrix addition and scalar multiplication of matrices (in Lecture 21), we have some properties of matrix multiplication.
Theorem. Let \(A, B\), and \( C \) be matrices, and let \(r\) be a scalar. Whenever the following expressions are defined,
| \( A(BC) = (AB)C \) | Associativity |
| \( A(B+C) = AB+AC \) | Distributivity |
| \( (A+B)C = AC+BC \) | Distributivity |
| \( r(AB) = (rA)B = A(rB) \) | Compatibility |
| \( I_m A = A = A I_n \) | Multiplicative Identity |
Proof of Associativity. Write \( \bbm c_i \) for the \( i^{\rm th} \) column of \( C\). Then the \( i^{\rm th}\) column of \( BC \) is \( B\bbm c_i \). So, the \( i^{\rm th} \) column of \( A(BC) \) is \( A(B\bbm c_i) \). Now, the \( i^{\rm th} \) column of \( (AB)C \) is \( (AB)\bbm c_i = A(B\bbm c_i) \). Thus, since \( A(BC) \) and \( (AB)C \) have the same columns, they are equal.
The other properties can be proved similarly. \( \Box \)
We have already seen that it is possible to have two matrices \( A \) and \( B \) for which the product \( AB \) is defined but \( BA \) is undefined. We have also seen examples where both \( AB \) and \( BA \) are defined but the two products are different sizes. However, even when \( AB \) and \( BA \) are both defined and both the same size, we still sometimes have \( AB \ne BA \).
Example 1. Let \( A = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 1 & 2 \\ 1 & 0 & -1 \end{bmatrix} \) and \( B = \begin{bmatrix} 0 & -1 & 1 \\ 1 & 0 & -2 \\ 3 & 2 & 5 \end{bmatrix} \). Compute \( AB\) and \( BA\).
Using the definition of matrix multiplication we learned in the previous lecture, we compute \[ AB = \begin{bmatrix} 1 & 2 & 1 \\ 2 & 1 & 2 \\ 1 & 0 & -1 \end{bmatrix} \begin{bmatrix} 0 & -1 & 1 \\ 1 & 0 & -2 \\ 3 & 2 & 5 \end{bmatrix} = \begin{bmatrix} 5 & 1 & 2 \\ 7 & 2 & 10 \\ -3 & -3 & -4 \end{bmatrix} \quad \mbox{and} \quad BA = \begin{bmatrix} 0 & -1 & 1 \\ 1 & 0 & -2 \\ 3 & 2 & 5 \end{bmatrix} \begin{bmatrix} 1 & 2 & 1 \\ 2 & 1 & 2 \\ 1 & 0 & -1 \end{bmatrix} = \begin{bmatrix} -1 & -1 & -3 \\ -1 & 2 & 3 \\ 12 & 8 & 2 \end{bmatrix}.\ \Box \]
All of these examples show that, in general, matrix multiplication is not commutative. We must exercise caution when doing algebra involving matrix multiplication. Many of the algebraic rules we are accustomed to do not work with matrices. For example, it is also not possible in general to cancel matrices; that is, if \( AB = AC \), then we cannot conclude that \( B=C \). Also, if \( AB = O \) (the zero matrix), then we cannot conclude that either \( A=O \) or \(B=O \).
Definition. A matrix is square if it has an equal number of rows and columns.
When \( A \) is an \( n\times n \) square matrix, we can multiply \( A \) by itself to obtain another \( n\times n\) matrix. We can do this repeatedly, creating the powers of \( A \).
Definition. Let \( A \) be an \( n\times n \) square matrix and let \( k \ge 0 \) be an integer. Then the \( k^{\rm th} \) power of \( A \), written \( A^k \), is the result of multiplying \( A \) by itself \( k \) times: \[ A^k = \underbrace{A\cdot A \cdot A \cdots A}_{k\ \rm times} \] We define \( A^0 = I_n \).
We briefly touched on the idea of repeatedly multiplying a matrix by itself in our discussion of Markov chains in Lecture 12. Later, in Lecture 35, we will focus on finding ways to compute powers of a matrix efficiently.
Note again that we must be careful not to use algebraic rules for matrices when they do not apply.
Example 2. Let \( A = \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \) and \( B = \begin{bmatrix} 0 & 2 \\ 1 & -1 \end{bmatrix} \). Compute both \( A^2 B^2 \) and \( (AB)^2 \). Are these matrices equal?
We compute \[ A^2 B^2 = \left( \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \right) \left( \begin{bmatrix} 0 & 2 \\ 1 & -1 \end{bmatrix} \begin{bmatrix} 0 & 2 \\ 1 & -1 \end{bmatrix} \right) = \begin{bmatrix} 2 & -2 \\ -5 & 7 \end{bmatrix} \] \[ (AB)^2 = \left( \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} 0 & 2 \\ 1 & -1 \end{bmatrix} \right) \left( \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} 0 & 2 \\ 1 & -1 \end{bmatrix} \right) = \begin{bmatrix} 2 & -6 \\ -3 & 11 \end{bmatrix} \]
We see that \( A^2B^2 \ne (AB)^2 \). \( \Box \)
This result may seem counterintiuitive until we realize that \( (AB)^2 \) means \( ABAB \). Since matrix multiplication is not commutative, we cannot swap the positions of \( A \) and \( B \) to turn this expression into \( AABB \).
Definition. Given an \( m\times n\) matrix \(A\), the transpose of \( A\), written \(A^T\), is the \( n\times m\) matrix whose columns are the rows of \( A\), and vice versa.
Example 3. Let \( A = \begin{bmatrix} 7 & 0 & -4 \\ 1 & -2 & 5 \end{bmatrix} \). Compute \( A^T \).
Since \( A\) has two rows, we construct \( A^T \) as a matrix with two columns with the same numbers as the rows of \( A\) in the same order: \( A^T = \begin{bmatrix} 7 & 1 \\ 0 & -2 \\ -4 & 5 \end{bmatrix} \). \( \Box \)
The transpose operation has some algebraic properties of its own. Whenever the following expressions are defined,
Example 4. Let \( A = \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \) and \( B = \begin{bmatrix} 0 & -1 & 1 \\ 3 & 2 & 5 \end{bmatrix} \). Verify that \( (AB)^T = B^T A^T \).
We compute \[ (AB)^T = \left( \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix} \begin{bmatrix} 0 & -1 & 1 \\ 3 & 2 & 5 \end{bmatrix} \right)^T = \begin{bmatrix} 0 & -1 & 1 \\ 3 & 3 & 4 \end{bmatrix}^T = \begin{bmatrix} 0 & 3 \\ -1 & 3 \\ 1 & 4 \end{bmatrix} \] \[ B^TA^T = \begin{bmatrix} 0 & -1 & 1 \\ 3 & 2 & 5 \end{bmatrix}^T \begin{bmatrix} 1 & 0 \\ -1 & 1 \end{bmatrix}^T = \begin{bmatrix} 0 & 3 \\ -1 & 2 \\ 1 & 5 \end{bmatrix} \begin{bmatrix} 1 & -1 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 0 & 3 \\ -1 & 3 \\ 1 & 4 \end{bmatrix} \]
Note that, while you might think that \( (AB)^T \) should equal \( A^T B^T \), the expression \( A^T B^T \) isn't even defined in the situation of Example 4. Order matters with matrix operations!
Since we now have a matrix multiplication, we might consider whether there is any such thing as "matrix division." In algebra, we typically think of division in terms of multiplicative inverses. For example, the multiplicative inverse of the number 3 is the number 1/3, since \( 3\cdot \frac 13 = 1 \) and 1 is our multiplicative identity. So, division by 3 is really just multiplication by 1/3. Not every number has a multiplicative inverse: there is no number that can be multiplied by zero to result in 1.
Do multiplicative inverses of matrices exist? Sometimes, they do! Since our multiplicative identity is \( I \), an inverse for a matrix \( A \) would look like a matrix \( C \) for which \( CA = I \), or a matrix \( D \) for which \( AD = I \). Since our multiplication is not commutative, we can imagine situations where only one of these "one-sided" inverses exists. The next two theorems tell us more about this possibility.
Theorem (Left-Side Inverse). Let \( A \) be an \( m\times n \) matrix and suppose there exists an \( n\times m \) matrix \( C\) for which \( CA = I_n \). Then \( A \) has a pivot in every column, and \( A \) cannot have more columns than rows.
Proof. Let \( \bbm u \) be a solution of the equation \( A\bbm x = \bbm 0\). Now, \( C(A\bbm u) = (CA)\bbm u = I_n \bbm u = \bbm u \) but also \( C(A\bbm u) = C(\bbm 0) = \bbm 0 \). So, \( \bbm u = \bbm 0 \) is the only solution of \( A\bbm x = \bbm 0 \). By the Linearly Independent Columns Theorem, \( A \) has a pivot in every column. Since each pivot is in a different row, \( A \) cannot have more columns than rows. \( \Box \)
Theorem (Right-Side Inverse). Let \( A \) be an \( m\times n \) matrix and suppose there exists an \( n\times m \) matrix \( D\) for which \( AD = I_m \). Then \( A \) has a pivot in every row, and \( A \) cannot have more rows than columns.
Proof. Let \( \bbm b \in \mathbb R^m \) be any vector and consider the equation \( A\bbm x = \bbm b \). We have \( A(D\bbm b)= (AD)\bbm b = I_m \bbm b = \bbm b \). So, \( A\bbm x = \bbm b\) has at least one solution for every \( \bbm b \in \mathbb R^m \). By the Spanning Columns Theorem, \( A\) has a pivot in every row. Since each pivot is in a different column, \( A \) cannot have more rows than columns. \( \Box \)
We will study inverses in much more detail in the next few lectures.
« Lecture 21 Back to Top Lecture 23 »