Lecture 39 - Orthonormal Sets

Learning Objectives

• Determine whether a given set of vectors is orthonormal
• Given a subspace \( H \) with orthonormal basis \( \cal B \) and a vector \( \bbm x\in H \), find the coordinate vector \( [\bbm x]_{\cal B} \)

Definition and Examples

In the previous lecture, we learned about orthogonal sets. An orthonormal set is an orthogonal set where every vector is a unit vector.

Definition. A set of vectors \( \{ \bbm u_1, \bbm u_2, \ldots, \bbm u_p \} \) in \( \mathbb R^n \) is an orthonormal set if \( \bbm u_i \cdot \bbm u_j = 0 \) for all \( i \neq j \) and \( \bbm u_i \cdot \bbm u_i = 1 \) for all \( i \).

Example 1. Let \( \bbm v_1 = \vecthree {3/\sqrt{11}} {1/\sqrt{11}} {1/\sqrt{11}} \), \( \bbm v_2 = \vecthree {-1/\sqrt{6}}{2/\sqrt{6}}{1/\sqrt{6}} \), and \( \bbm v_3 = \vecthree{-1/\sqrt{66}}{-4/\sqrt{66}}{7/\sqrt{66}} \). Show that \( \{ \bbm v_1, \bbm v_2, \bbm v_3 \} \) is an orthonormal set.

We compute the following dot products:

Since \( \bbm v_i \cdot \bbm v_j = 0 \) for all \( i \neq j \) and \( \bbm v_i \cdot \bbm v_i = 1 \) for all \( i \), the set is orthonormal. \( \Box \)

Matrices With Orthonormal Columns

The pattern of dot products in Example 1 helps us understand the following theorem, which characterizes matrices whose columns form an orthonormal set.

Theorem (Characterization of Matrices With Orthonormal Columns). An \( m\times n\) matrix \( U \) has orthonormal columns if and only if \( U^T U = I_n \).

Proof. Let \( \bbm u_1, \bbm u_2, \ldots, \bbm u_n \in \mathbb R^m \) be the columns of \( U \). We have \[ U^T U = \vecfour {\bbm u_1^T} {\bbm u_2^T} \vdots {\bbm u_n^T} [{\bbm u_1}\ {\bbm u_2}\ \cdots\ {\bbm u_n}] = \begin{bmatrix} \bbm u_1^T \bbm u_1 & \bbm u_1^T \bbm u_2 & \cdots & \bbm u_1^T \bbm u_n \\ \bbm u_2^T \bbm u_1 & \bbm u_2^T \bbm u_2 & \cdots & \bbm u_2^T \bbm u_n \\ \vdots & \vdots & \ddots & \vdots \\ \bbm u_n^T \bbm u_1 & \bbm u_n^T \bbm u_2 & \cdots & \bbm u_n^T \bbm u_n \end{bmatrix} = \begin{bmatrix} \bbm u_1 \cdot \bbm u_1 & \bbm u_1\cdot \bbm u_2 & \cdots & \bbm u_1\cdot \bbm u_n \\ \bbm u_2 \cdot \bbm u_1 & \bbm u_2\cdot \bbm u_2 & \cdots & \bbm u_2\cdot \bbm u_n \\ \vdots & \vdots & \ddots & \vdots \\ \bbm u_n \cdot \bbm u_1 & \bbm u_n\cdot \bbm u_2 & \cdots & \bbm u_n\cdot \bbm u_n \end{bmatrix} \]

From this, we see that \( U \) has orthonormal columns if and only if \( U^T U = I_n \). \( \Box \)

We saw in Lecture 38 that we have a convenient formula for coordinates in an orthogonal basis. The formula is even more convenient when the basis is orthonormal.

Theorem (Coordinates in an Orthonormal Basis). Let \( H \) be a subspace of \( \mathbb R^n \) and suppose \( {\cal B} = \{ \bbm u_1, \ldots, \bbm u_p \} \) is an orthonormal basis for \( H \). Let \( \bbm y \in H \). Then, \( [\bbm y]_{\cal B} = \vecthree {c_1} \vdots {c_p} \), where \(c_i = \bbm y \cdot \bbm u_i \).

Proof. Since \( \cal B \) is an orthogonal set, we can apply the Coordinates in an Orthogonal Basis Theorem to write \( c_i = \frac{\bbm y \cdot \bbm u_i}{\bbm u_i \cdot \bbm u_i} \). Since \( \cal B \) is orthonormal, \( \bbm u_i \cdot \bbm u_i = 1 \), so we have \(c_i = \bbm y \cdot \bbm u_i \), as desired. \( \Box \)

Matrices with orthonormal columns also interact nicely with the dot product.

Theorem (Properties of Matrices With Orthonormal Columns). Let \( U \) be an \( m \times n\) matrix with orthonormal columns, and let \( \bbm x, \bbm y\in \mathbb R^n \).

1. \( \| U\bbm x \| = \| \bbm x \| \)
2. \( (U\bbm x)\cdot(U\bbm y) = \bbm x \cdot \bbm y \)
3. \( (U\bbm x)\cdot(U\bbm y) = 0\) if and only if \( \bbm x \cdot \bbm y = 0 \)

Proof. Both (1) and (3) follow directly from (2), so we will focus on proving (2). Write \( \bbm u_i \) for the columns of \( U\) and \(x_i\) and \(y_i\) for the entries of \(\bbm x\) and \(\bbm y\), respectively. Now \( U\bbm x = x_1 \bbm u_1 + \cdots + x_n \bbm u_n \) and \( U\bbm y = y_1 \bbm u_1 + \cdots + y_n \bbm u_n \). So, \[ (U\bbm x)\cdot (U\bbm y) = \sum_{i=1}^n \sum_{j=1}^n x_i y_j (\bbm u_i \cdot \bbm u_j) \]

Since the columns of \( U \) are orthonormal, \(\bbm u_i \cdot \bbm u_j\) equals 1 when \( i = j \) and equals 0 otherwise. Thus, \[ (U\bbm x)\cdot (U\bbm y) = x_1 y_1 + \cdots + x_n y_n = \bbm x \cdot \bbm y.\ \Box \]

Orthogonal Matrices

Definition. An \( n\times n\) matrix \( U \) is orthogonal if its columns form an orthonormal set.

From the Characterization of Matrices With Orthonormal Columns Theorem, we see that an orthogonal matrix \( U \) has \( U^T U = I_n \), and therefore \( U^T = U^{-1}\). An orthogonal matrix has orthonormal rows as well as orthonormal columns. The Properties of Matrices With Orthonormal Columns Theorem shows that orthogonal matrices in \( \mathbb R^2 \) or \( \mathbb R^3 \) represent transformations that preserve distances and angles. These types of matrices are especially important in the field of computer graphics, where we want to manipulate images and figures without stretching or distorting them.