Matrix Operations and Transposes

You multiplied matrices in precalculus with a row-times-column rule. Here we see where that rule comes from, which explains its strange features: sizes must match, and order matters.

Sums and scalar multiples work entry by entry (for matrices of the same size). The product is defined so that multiplying by $AB$ does the same as multiplying by $B$ and then by $A$: $$(AB)\mathbf{x} = A(B\mathbf{x}) \text{ for every } \mathbf{x}.$$ That forces the definition $AB$ $= [\,A\mathbf{b}_1\ A\mathbf{b}_2\ \cdots\ A\mathbf{b}_p\,]$: column $j$ of $AB$ is $A$ times column $j$ of $B$.

Why it works

Why must column $j$ of $AB$ be $A\mathbf{b}_j$? Take $\mathbf{x} = \mathbf{e}_j$ (all zeros except a $1$ in entry $j$). Then $B\mathbf{e}_j = \mathbf{b}_j$, so $(AB)\mathbf{e}_j = A\mathbf{b}_j$, and $(AB)\mathbf{e}_j$ is column $j$ of $AB$. Writing out $A\mathbf{b}_j$ with the row rule gives the familiar formula: entry $(i, j)$ of $AB$ is row $i$ of $A$ times column $j$ of $B$. And it explains the size rule: $A$ ($m \times n$) can only multiply vectors with $n$ entries, so $B$ needs $n$ rows; then $AB$ is $m \times p$.

Worked example: A product, column by column

Compute $AB$ for $A = \begin{bmatrix} 1 & 2 & 0 \\ -1 & 3 & 4 \end{bmatrix}$ and $B = \begin{bmatrix} 2 & 1 \\ 0 & -1 \\ 3 & 2 \end{bmatrix}$.

$A$ is $2 \times 3$ and $B$ is $3 \times 2$: inside sizes match (3), so $AB$ is $2 \times 2$.
Column 1: $A\begin{bmatrix} 2 \\ 0 \\ 3 \end{bmatrix}$ $= 2\begin{bmatrix} 1 \\ -1 \end{bmatrix} + 0\begin{bmatrix} 2 \\ 3 \end{bmatrix} + 3\begin{bmatrix} 0 \\ 4 \end{bmatrix}$ $= \begin{bmatrix} 2 \\ 10 \end{bmatrix}$.
Column 2: $A\begin{bmatrix} 1 \\ -1 \\ 2 \end{bmatrix}$ $= \begin{bmatrix} 1 \\ -1 \end{bmatrix} - \begin{bmatrix} 2 \\ 3 \end{bmatrix} + 2\begin{bmatrix} 0 \\ 4 \end{bmatrix}$ $= \begin{bmatrix} -1 \\ 4 \end{bmatrix}$.
Row-column check of one entry: $(AB)_{21}$ $= (-1)(2) + (3)(0) + (4)(3)$ $= 10$. ✓

Answer: $AB = \begin{bmatrix} 2 & -1 \\ 10 & 4 \end{bmatrix}$

Worked example: Order matters

Compare $CD$ and $DC$ for $C = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}$ and $D = \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix}$.

$CD = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}$.
$DC = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}$.
They differ. Even when both products exist and have the same size, $AB$ and $BA$ are usually different.

Answer: $CD \neq DC$

Rules that hold, and two that do not

Matrix algebra at a glance.
Holds	Does not hold in general
$A(BC) = (AB)C$ (associative)	$AB = BA$
$A(B + C) = AB + AC$, $(B + C)A = BA + CA$	If $AB = AC$, then $B = C$ (no cancelling)
$(A^{T})^{T} = A$, $(A + B)^{T} = A^{T} + B^{T}$	If $AB = 0$, then $A = 0$ or $B = 0$
$(AB)^{T} = B^{T}A^{T}$	$(AB)^{T} = A^{T}B^{T}$

The transpose $A^{T}$ turns the rows of $A$ into columns. The product rule reverses the order: $(AB)^{T} = B^{T}A^{T}$. Sizes explain it: if $A$ is $2 \times 3$ and $B$ is $3 \times 2$, then $A^{T}B^{T}$ is $(3 \times 2)(3 \times 2)$, which is not even defined, while $B^{T}A^{T}$ is $(2 \times 3)(3 \times 2)$, which is. Entry $(i, j)$ of $(AB)^{T}$ is entry $(j, i)$ of $AB$: row $j$ of $A$ times column $i$ of $B$, which is row $i$ of $B^{T}$ times column $j$ of $A^{T}$.

Common mistake

Assuming $AB = BA$, or writing $(AB)^{T} = A^{T}B^{T}$. Matrices are actions, and doing $B$ then $A$ is usually not the same as $A$ then $B$ (put on socks, then shoes). Quick check for the transpose rule: test the sizes with a non-square example; the wrong order is not even defined.

Each matrix is an action on the plane. Try a shear, then a rotation, and compare with the rotation first: the results differ, which is why $AB \neq BA$.

Key idea

Matrix multiplication is defined so that $(AB)\mathbf{x} = A(B\mathbf{x})$: column $j$ of $AB$ is $A\mathbf{b}_j$. It is associative and distributive but not commutative, and transposes reverse products: $(AB)^{T} = B^{T}A^{T}$.

Matrix Operations and Transposes

Rules that hold, and two that do not

Practice

Builds on

Leads to

Sources and further reading