Recall that, in Lecture 6, we discussed how vectors in \( \mathbb R^2 \) and \( \mathbb R^3 \) can be thought of as "arrows." In this lecture, we'll learn how to apply geometric concepts such as length, distance, and perpendicularity to vectors.
Definition. Given two vectors \( \bbm u, \bbm v \in \mathbb R^n \) with entries \( u_i \) and \( v_i \), respectively, the inner product (dot product) of \( \bbm u \) and \( \bbm v\), written \( \bbm u \cdot \bbm v \) is \[ \bbm u \cdot \bbm v = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n . \]
Note that \( \bbm u \cdot \bbm v \) is a scalar. We can also write \( \bbm u \cdot \bbm v = \bbm u^T \bbm v \), where we think of the product \( \bbm u^T \bbm v\) as a scalar rather than as a \( 1\times 1\) matrix.
Example 1. Let \( \bbm u = \vecthree {-2} 1 5 \) and \( \bbm v = \vecthree 6 {-7} 2 \). Compute \( \bbm u \cdot \bbm v \).
We compute \( \bbm u \cdot \bbm v = (-2)(6) + (1)(-7) + (5)(2) = -9 \). \( \Box \)
Much like the other algebraic operations we have learned about, the inner product has several nice properties.
Theorem (Properties of Inner Product). Let \( \bbm u, \bbm v, \bbm w\in \mathbb R^n \) and let \( c\) be a scalar.
Proof of (1). Write \( u_i \) and \( v_i \) for the entries of \( \bbm u \) and \(\bbm v \), respectively. Then, \[ \bbm u \cdot \bbm v = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = v_1 u_1 + v_2 u_2 + \cdots + v_n u_n = \bbm v \cdot \bbm u. \] The other parts of the theorem can be proved in a similar way. \( \Box \)
Definition. The length (norm) of a vector \( \bbm v = \vecfour {v_1} {v_2} \vdots {v_n} \in \mathbb R^n \), written \( \| \bbm v \| \), is \( \| \bbm v \| = \sqrt{ \bbm v \cdot \bbm v } = \sqrt{ v_1^2 + v_2^2 + \cdots + v_n^2 }. \)
In two or three dimensions, the norm of a vector is just the distance between the origin and the head of \( \bbm v\).
Definition. A unit vector is a vector whose length is equal to 1.
The standard basis vectors \( \bbm e_1, \bbm e_2, \ldots, \bbm e_n \) are all unit vectors. We can "normalize" a vector by dividing it by its own length: \( \frac 1 {\| \bbm v \|} \bbm v \) is a unit vector that points in the same direction as \( \bbm v\).
Example 2. Let \( \bbm v = \vecthree {-4} 5 0 \) and \( \bbm w = \vecthree 1 {-1} 2 \). Find a unit vector that points in the same direction as \( \bbm v+ \bbm w \).
First, we compute \( \bbm v+ \bbm w = \vecthree {-3} 4 2 \). The length of this vector is \( \| \bbm v + \bbm w \| = \sqrt{(-3)^2 + 4^2 + 2^2} = \sqrt{29}\). So, the unit vector is \[ \frac{1}{\| \bbm v + \bbm w \|} (\bbm v+\bbm w) = \frac{1}{\sqrt{29}} \vecthree {-3} 4 2 = \vecthree {-3/\sqrt{29}} {4/\sqrt{29}} {2/\sqrt{29}}.\ \Box \]
Given two vectors \( \bbm u \) and \( \bbm v \) in standard position, the vector \( \bbm u - \bbm v \) points from the head of \( \bbm v \) to the head of \( \bbm u \). The distance between the heads of two vectors in standard position is thus the norm of this difference: \( \| \bbm u - \bbm v \| \).
This way of calculating distance gives us a way to investigate when two vectors are perpendicular. We say that two vectors in \( \mathbb R^2 \) or \( \mathbb R^3 \) are "perpendicular" if they form a right angle when placed tail-to-tail. However, this notion of "perpendicular" doesn't easily extend to four or more dimensions. Another way of thinking about perpendicularity that does extend to higher dimensions is this: Two vectors \( \bbm u \) and \( \bbm v\) are perpendicular when the distance between \( \bbm u \) and \( \bbm v \) is the same as the distance between \( \bbm u \) and \( -\bbm v\). That is, when \( \| \bbm u - \bbm v \| = \| \bbm u - (-\bbm v) \| \). In this case, we say that the vectors are orthogonal.
The equation \( \| \bbm u - \bbm v \| = \| \bbm u - (-\bbm v) \| \) can be simplified by squaring both sides. We get \[ \| \bbm u - \bbm v \|^2 = (\bbm u - \bbm v)\cdot (\bbm u - \bbm v) = \bbm u \cdot \bbm u - 2 \bbm u \cdot \bbm v + \bbm v \cdot \bbm v \quad \mbox{and} \quad \| \bbm u + \bbm v \|^2 = (\bbm u + \bbm v)\cdot (\bbm u + \bbm v) = \bbm u \cdot \bbm u + 2 \bbm u \cdot \bbm v + \bbm v \cdot \bbm v \]
Setting these equal and simplifying gives \( 4\bbm u \cdot \bbm v = 0\), which is equivalent to \( \bbm u \cdot \bbm v = 0\).
Definition. Let \( \bbm u, \bbm v \in \mathbb R^n \). The vectors \(\bbm u\) and \(\bbm v\) are orthogonal if \(\bbm u\cdot\bbm v = 0\).
We use the word "orthogonal" rather than "perpendicular" because the word "perpendicular" has a strong connection to 90-degree angles. Since "angles" don't really make sense in higher dimensions, the word "orthogonal" is used instead. In two or three dimensions, "perpendicular" and "orthogonal" can be used interchangeably.
Example 3. Show that the vectors \( \bbm u = \vecfour 3 {-1} 0 2 \) and \( \bbm v = \vecfour {-2} 2 7 4 \) are orthogonal.
We compute \( \bbm u \cdot \bbm v = (3)(-2) + (-1)(2) + (0)(7) + (2)(4) = 0 \). \( \Box \)
In Lecture 29, we discussed the column space of a matrix. We can also think of the rows of a matrix as vectors and consider the "row space" of a matrix.
Definition. Let \( A \) be an \( m\times n\) matrix and write \( \bbm u_i^T \) for the \( i^{\rm th} \) row of \( A \). The row space of \( A \) is \( \mbox{Row } A = \mbox{Span} \{ \bbm u_1, \bbm u_2, \ldots, \bbm u_m \} \).
Theorem. If \( A \) is an \( m\times n\) matrix, then \( \mbox{Row } A \) is a subspace of \( \mathbb R^n \).
Proof. This follows immediately from the Subspace Spanned By a Set Theorem from Lecture 27.
When we have a subset \( H \) of \( \mathbb R^n \), we will often want to think about the collection of vectors that are perpendicular to every vector in \( H \). This turns out to be a subspace of \( \mathbb R^n \), even if \( H \) is not!
Definition. Let \( H \subseteq \mathbb R^n \). A vector \( \bbm z \in \mathbb R^n \) is orthogonal to \( H \) if \( \bbm z \) is orthogonal to every vector in \( H \).
Definition. Let \( H \subseteq \mathbb R^n \). The orthogonal complement of \( H \), written \( H^\perp \), is the set of all vectors in \( \mathbb R^n \) that are orthogonal to \( H \).
Theorem. If \( H \) is a subset of \( \mathbb R^n \), then \( H^\perp \) is a subspace of \( \mathbb R^n \).
Proof.
This leads to an interesting relationship between column spaces, null spaces, transposes, and row spaces:
Theorem. Let \( A \) be an \( m\times n\) matrix. Then,
Proof. Since \( \Col A \) is the same as \( \mbox{Row } A^T \), it suffices to prove the second bullet point. Write \( \bbm u_i^T \) for the rows of \( A \), and note that for any vector \( \bbm x \in \mathbb R^n \) we have \[ A\bbm x = \vecfour {\bbm u_1^T} {\bbm u_2^T} \vdots {\bbm u_m^T} \bbm x = \vecfour {\bbm u_1^T \cdot \bbm x} {\bbm u_2^T \cdot \bbm x} \vdots {\bbm u_m^T \cdot \bbm x}. \]
We see that \( A\bbm x = \bbm 0 \) exactly when \( \bbm x\) is orthogonal to each row of \( A \). Hence the elements of \( \Nul A \) are exactly the elements of \( (\mbox{Row } A)^\perp \). \( \Box \)
Example 4. Let \( \bbm v_1 = \vecfour 1 {-1} 0 1 \) and \( \bbm v_2 = \vecfour {-2} 0 1 {-1} \), and let \( H = {\rm Span}\{ \bbm v_1, \bbm v_2 \} \). Find a basis for \( H^\perp \).
We can think of \( H \) as the column space of the matrix \( A \) that has \( \bbm v_1 \) and \( \bbm v_2 \) as its columns. Since \( (\Col A)^\perp = \Nul A^T \), we row reduce \( A^T \) to write the solution of \( A^T \bbm x = \bbm 0 \) in parametric form: \[ \begin{bmatrix} 1 & -1 & 0 & 1 \\ -2 & 0 & 1 & -1 \end{bmatrix} \longrightarrow \begin{bmatrix} 1 & 0 & -1/2 & 1/2 \\ 0 & 1 & -1/2 & -1/2 \end{bmatrix}. \]
The solution in parametric form is \[ \bbm x = \vecfour {x_1} {x_2} {x_3} {x_4} = x_3 \vecfour {1/2} {1/2} 1 0 + x_4 \vecfour {-1/2} {1/2} 0 1. \]
Thus, a basis for \( H^\perp \) is \( \left\{ \vecfour {1/2} {1/2} 1 0, \vecfour {-1/2} {1/2} 0 1 \right\} \). \( \Box \)
« Lecture 36 Back to Top Lecture 38 »