If \( H \) is a subspace of \( \mathbb R^n \), recall from Lecture 30 that a basis for \( H \) is a linearly independent set of vectors that spans \( H \).
Definition. Let \( H \subseteq \mathbb R^n \) be a subspace other than \( \{ \bbm 0 \} \). The dimension of \( H \) is the number of vectors in any basis for \( H \). The dimension of \( \{ \bbm 0 \} \) is defined to be zero.
In mathematics, we want our definitions to be unambiguous; that is, we don't want a definition to be interpreted as meaning more than one thing. While this definition for "dimension" may seem reasonable at first, it begs an important question: How do we know that we can't have two different bases for subspace with different numbers of vectors? We need to show that "dimension" is well-defined by showing that this situation is impossible.
Theorem (Dimension is Well-Defined). Let \( H \subseteq \mathbb R^n \) be a subspace other than \( \{ \bbm 0 \} \). If \( \cal B \) and \( \cal C \) are two bases for \( H \), then \( \cal B \) and \( \cal C \) have the same number of vectors.
Before we can prove this theorem, we need another theorem, which is related to the More Vectors Than Entries Theorem from Lecture 16.
Theorem (More Vectors Than a Basis). Let \( H \subseteq \mathbb R^n \) be a subspace and let \( {\cal B} = \{ \bbm b_1, \ldots, \bbm b_p \} \) be a basis for \( H \). Any set of \( q \) vectors with \( q \gt p \) is linearly dependent.
Proof of the More Vectors Than a Basis Theorem. Let \( \bbm v_1, \bbm v_2, \ldots, \bbm v_q \in H \). Consider the vector equation \[ x_1 [\bbm v_1]_{\cal B} + x_2 [\bbm v_2]_{\cal B} + \cdots + x_q [\bbm v_q]_{\cal B} = \bbm 0 \]
This vector equation corresponds to a coefficient matrix with \( p \) rows and \( q \) columns. Since \( q \gt p \), this matrix cannot have a pivot in every column. Therefore, this vector equation has a nontrivial solution, which gives a dependence relation among the vectors \( [\bbm v_i]_{\cal B} \): \[ c_1 [\bbm v_1]_{\cal B} + c_2 [\bbm v_2]_{\cal B} + \cdots + c_q [\bbm v_q]_{\cal B} = \bbm 0 \] for some scalars \( c_1, \ldots, c_q \) that are not all zero.
As we discussed in the previous lecture, the correspondence \( \bbm x \leftrightarrow [\bbm x]_{\cal B} \) is a one-to-one and onto linear transformation, which means that we have \[ [c_1 \bbm v_1 + c_2 \bbm v_2 + \cdots + c_q \bbm v_q]_{\cal B} = \bbm 0 \]
The only vector that has a zero coordinate vector is the zero vector, so \( c_1 \bbm v_1 + c_2 \bbm v_2 + \cdots + c_q \bbm v_q = \bbm 0 \). This proves that the vectors \( \bbm v_1, \ldots, \bbm v_q \) are linearly dependent. \( \Box \)
Proof of the Dimension is Well-Defined Theorem. Write \( p \) and \( q\) for the numbers of vectors in \( \cal B \) and \( \cal C \), respectively. If \( q \gt p \), then \( \cal C \) is linearly dependent by the previous theorem, which is a contradiction since \( \cal C \) is a basis. Similarly, if \( p \gt q \), then \( \cal B \) is linearly dependent, which is also a contradiction. Hence we must have \( p = q \). \( \Box \)
We can think of the dimension of a subspace as being the "right" number of linearly independent vectors you need to form a basis for \( H \).
The Basis Theorem. Let \( H \) be a subspace of \( \mathbb R^n \) with dimension \( p > 0 \). Any linearly independent set of \( p \) vectors in \( H \) is a basis for \( H \).
Proof. Let \( \{ \bbm v_1, \ldots, \bbm v_p \} \) be a linearly independent set of vectors in \( H \). We must prove that \( \{ \bbm v_1, \ldots, \bbm v_p \} \) spans \( H \). Let \( \bbm w \in H \).
Consider the set \( \{ \bbm v_1, \ldots, \bbm v_p, \bbm w \} \). This set has \( p+1 \) vectors, so it must be linearly dependent by the More Vectors Than a Basis Theorem. From the Characterization of Linearly Dependent Sets Theorem (from Lecture 16), we know that one of the vectors in this set must be a linear combination of the preceding vectors. However, it cannot be any of the \( \bbm v_i \) since the original set \( \{ \bbm v_1, \ldots, \bbm v_p \} \) is linearly independent.
Thus, it must be that \( \bbm w\) is a linear combination of the vectors \( \bbm v_1, \ldots, \bbm v_p \), which shows that these vectors span \( H \). Therefore, \( \{ \bbm v_1, \ldots, \bbm v_p \} \) is a basis for \( H \). \( \Box \)
Now we turn our attention to the dimensions of the null space and column space of a matrix.
Definition. Given a matrix \( A \), the nullity of \( A \) is the dimension of \( \Nul A \).
Definition. Given a matrix \( A \), the rank of \( A \) is the dimension of \( \Col A \).
Theorem (Rank-Nullity). Given an \( m\times n\) matrix \( A\), we have \( {\rm rank}(A) + {\rm nullity}(A)= n\).
Proof. We learned in Lecture 29 that the pivot columns of \( A \) form a basis for \( \Col A \). So, the rank of \( A \) equals the number of pivot columns of \( A \). We learned in Lecture 28 that \( \Nul A \) has one basis vector for each free variable in the equation \( A \bbm x = \bbm 0 \). This means that the nullity of \( A \) equals the number of non-pivot columns in \( A \). The number of pivot columns of \( A \) plus the number of non-pivot columns of \( A \) equals the total number of columns, \( n \). \( \Box \)
Example. Let \( A = \begin{bmatrix} -1 & 2 & 3 & -4 \\ -2 & 4 & 1 & 2\\ 0 & 0 & -1 & 2 \end{bmatrix} \). Compute the rank and nullity of \(A\) and verify the Rank-Nullity Theorem for \( A \).
We row-reduce \( A \): \[ \begin{bmatrix} -1 & 2 & 3 & -4 \\ -2 & 4 & 1 & 2\\ 0 & 0 & -1 & 2 \end{bmatrix} \longrightarrow \begin{bmatrix} 1 & -2 & 0 & -2 \\ 0 & 0 & 1 & -2 \\ 0 & 0 & 0 & 0 \end{bmatrix} \]
We see that \( A \) has two pivot columns, so \( {\rm rank}(A) = 2 \). Since \( A \) has two non-pivot columns, \( {\rm nullity}(A) = 2 \). Finally, we verify that \( {\rm rank}(A) + {\rm nullity}(A) = 2+2 = 4 \), which equals the number of columns of \( A \). \( \Box \)
Our study of null spaces and column spaces give us additional criteria that can be used to determine whether a square matrix is invertible. We can include these in an expanded version of the Invertible Matrix Theorem from Lecture 25:
The Invertible Matrix Theorem. Let \( A \) be a square \( n\times n\) matrix. The following statements are equivalent: