Orthogonal Matrices

Gram–Schmidt produces orthonormal columns. Square matrices with orthonormal columns are the rigid motions of linear algebra: rotations and reflections. They are the easiest matrices to invert and they never distort lengths.

Key idea

A square matrix $Q$ is orthogonal when its columns are orthonormal, that is, $Q^{T}Q = I$. Then $Q^{-1} = Q^{T}$, and $Q$ preserves lengths and angles: $\lVert Q\mathbf{x}\rVert = \lVert\mathbf{x}\rVert$ and $(Q\mathbf{x})\cdot(Q\mathbf{y}) = \mathbf{x}\cdot\mathbf{y}$.

Why it works

Entry $(i, j)$ of $Q^{T}Q$ is (column $i$)$\cdot$(column $j$). Orthonormal columns make this $1$ when $i = j$ and $0$ otherwise: $Q^{T}Q = I$. For a square matrix, a one-sided inverse is the inverse, so $Q^{-1} = Q^{T}$. Then $(Q\mathbf{x})\cdot(Q\mathbf{y})$ $= (Q\mathbf{x})^{T}(Q\mathbf{y})$ $= \mathbf{x}^{T}Q^{T}Q\mathbf{y}$ $= \mathbf{x}^{T}\mathbf{y}$. Dot products are kept, so lengths and angles are kept.

Worked example: Checking and inverting

Show that $Q = \begin{bmatrix} \frac{1}{3} & \frac{2}{3} & \frac{2}{3} \\ \frac{2}{3} & \frac{1}{3} & -\frac{2}{3} \\ \frac{2}{3} & -\frac{2}{3} & \frac{1}{3} \end{bmatrix}$ is orthogonal and find $Q^{-1}$.

Column lengths: $\frac{1}{9}(1 + 4 + 4) = 1$, and the same for the others.
Column 1 $\cdot$ column 2 $= \frac{1}{9}(2 + 2 - 4) = 0$; similarly the other pairs. So $Q^{T}Q = I$.
$Q^{-1} = Q^{T}$. This $Q$ happens to be symmetric, so $Q^{-1} = Q$: applying it twice gives the identity (it is a reflection).

Answer: $Q^{-1} = Q^{T} = Q$

Worked example: Completing an orthogonal matrix

Find $p$ and $r$ so that $\begin{bmatrix} \frac{3}{5} & p \\ \frac{4}{5} & r \end{bmatrix}$ is orthogonal with determinant $1$.

The second column must be a unit vector perpendicular to $\left(\frac{3}{5}, \frac{4}{5}\right)$: either $\left(-\frac{4}{5}, \frac{3}{5}\right)$ or $\left(\frac{4}{5}, -\frac{3}{5}\right)$.
Determinant with the first choice: $\frac{3}{5}\cdot\frac{3}{5} - \left(-\frac{4}{5}\right)\frac{4}{5} = 1$. ✓ (The second choice gives $-1$, a reflection.)

Answer: $p = -\frac{4}{5}$, $r = \frac{3}{5}$

An orthogonal matrix moves the grid rigidly: lengths, angles and areas are unchanged ($\det Q = \pm 1$).

Every $2 \times 2$ orthogonal matrix is a rotation $\begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}$ (determinant $1$) or a reflection $\begin{bmatrix} \cos\theta & \sin\theta \\ \sin\theta & -\cos\theta \end{bmatrix}$ (determinant $-1$).
$\det Q = \pm 1$, because $(\det Q)^{2} = \det Q^{T}\det Q = \det I = 1$.
Products of orthogonal matrices are orthogonal: $(Q_1Q_2)^{T}(Q_1Q_2)$ $= Q_2^{T}Q_1^{T}Q_1Q_2$ $= I$.

Because $Q^{-1} = Q^{T}$, solving $Q\mathbf{x} = \mathbf{b}$ costs only a multiplication: $\mathbf{x} = Q^{T}\mathbf{b}$, so each coordinate is a dot product $\mathbf{q}_i\cdot\mathbf{b}$. This is the projection formula again: an orthonormal basis gives coordinates for free.

Common mistake

Calling a matrix orthogonal because its columns are perpendicular. They must also have length $1$. $\begin{bmatrix} 3 & -4 \\ 4 & 3 \end{bmatrix}$ has perpendicular columns of length $5$, so $Q^{T}Q = 25I$, not $I$; dividing by $5$ fixes it.

Rectangular matrices can have orthonormal columns too. If $U$ is $m \times n$ with orthonormal columns, then still $U^{T}U = I_n$ and $\lVert U\mathbf{x}\rVert = \lVert\mathbf{x}\rVert$, but $UU^{T}$ is not $I_m$: it is the projection onto the column space of $U$. The $Q$ in a QR factorization is usually of this kind.

Orthogonal matrices are prized in computing: multiplying by them never magnifies rounding errors, which is why QR and the SVD are built from them.

Orthogonal Matrices

Practice

Builds on

Leads to

Sources and further reading