The Matrix Equation Ax = b

We have been writing systems as combinations of columns. A matrix-vector product packs that into two symbols. It is the single most important definition in the course.

If $A$ is $m \times n$ with columns $\mathbf{a}_1, \dots, \mathbf{a}_n$, and $\mathbf{x}$ is in $\mathbb{R}^n$, then $$A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n.$$ The entries of $\mathbf{x}$ are the weights on the columns. So $\mathbf{x}$ must have exactly one entry per column of $A$, and the result lives in $\mathbb{R}^m$, like the columns.

Worked example: Two ways to compute Ax

Compute $A\mathbf{x}$ for $A = \begin{bmatrix} 1 & 2 & -1 \\ 0 & -3 & 4 \end{bmatrix}$ and $\mathbf{x} = \begin{bmatrix} 3 \\ 1 \\ 2 \end{bmatrix}$.

By columns: $3\begin{bmatrix} 1 \\ 0 \end{bmatrix} + 1\begin{bmatrix} 2 \\ -3 \end{bmatrix} + 2\begin{bmatrix} -1 \\ 4 \end{bmatrix}$ $= \begin{bmatrix} 3 \\ 0 \end{bmatrix} + \begin{bmatrix} 2 \\ -3 \end{bmatrix} + \begin{bmatrix} -2 \\ 8 \end{bmatrix}$ $= \begin{bmatrix} 3 \\ 5 \end{bmatrix}$.
By rows (faster by hand): entry $i$ is row $i$ of $A$ times $\mathbf{x}$, entry by entry, added up. Entry 1: $(1)(3) + (2)(1) + (-1)(2)$ $= 3$. Entry 2: $(0)(3) + (-3)(1) + (4)(2)$ $= 5$.
Both give $\begin{bmatrix} 3 \\ 5 \end{bmatrix}$: $A$ is $2 \times 3$, $\mathbf{x}$ has 3 entries, and $A\mathbf{x}$ has 2.

Answer: $\begin{bmatrix} 3 \\ 5 \end{bmatrix}$

So one system has three equivalent forms: a list of equations, a vector equation $x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$, and the matrix equation $A\mathbf{x} = \mathbf{b}$. All three have the same solutions, and $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ is in the span of the columns of $A$.

The columns of $A$ are $A\hat{\imath}$ and $A\hat{\jmath}$; every $A\mathbf{v}$ is a combination of them.

Why it works

$A$ respects sums and multiples: $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$ and $A(c\mathbf{u}) = cA\mathbf{u}$. Proof from the definition: $A(\mathbf{u} + \mathbf{v})$ $= (u_1 + v_1)\mathbf{a}_1 + \cdots + (u_n + v_n)\mathbf{a}_n$, and regrouping gives $(u_1\mathbf{a}_1 + \cdots) + (v_1\mathbf{a}_1 + \cdots)$ $= A\mathbf{u} + A\mathbf{v}$. Scaling is the same idea. These two facts will become the definition of a linear map.

One matrix deserves a name now. The identity matrix $I_n$ has $1$s on the diagonal and $0$s elsewhere, so its columns are $\mathbf{e}_1, \dots, \mathbf{e}_n$, and $I_n\mathbf{x} = x_1\mathbf{e}_1 + \cdots + x_n\mathbf{e}_n = \mathbf{x}$ for every $\mathbf{x}$. Likewise $A\mathbf{e}_j$ is simply column $j$ of $A$: a handy way to pull out a column.

Worked example: From a system to Ax = b

Write $3x_1 - x_2 = 4$, $x_1 + 5x_3 = 0$ as a matrix equation.

Unknowns $x_1, x_2, x_3$ give three columns; a missing unknown gets a $0$.
$\begin{bmatrix} 3 & -1 & 0 \\ 1 & 0 & 5 \end{bmatrix}\begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix}$ $= \begin{bmatrix} 4 \\ 0 \end{bmatrix}$.

Answer: $A = \begin{bmatrix} 3 & -1 & 0 \\ 1 & 0 & 5 \end{bmatrix}$, $\mathbf{b} = \begin{bmatrix} 4 \\ 0 \end{bmatrix}$

Common mistake

Multiplying entries of $\mathbf{x}$ against the columns in the wrong pattern (for example, row 1 of $A$ times row 1 of $\mathbf{x}$ only), or trying to multiply when sizes do not match. Check: $\mathbf{x}$ needs as many entries as $A$ has columns, and $A\mathbf{x}$ has as many entries as $A$ has rows. A $2 \times 3$ matrix times a vector with 2 entries is not defined.

Key idea

$A\mathbf{x}$ is the combination of the columns of $A$ with the entries of $\mathbf{x}$ as weights. So $A\mathbf{x} = \mathbf{b}$ is solvable exactly when $\mathbf{b}$ is in the span of the columns.

The Matrix Equation Ax = b

Practice

Builds on

Leads to

Sources and further reading