Linear Independence

Some sets of vectors contain a redundant member: one that the others can already build. Independence is the precise way to say there is no redundancy. It rests on homogeneous systems.

Vectors $\mathbf{v}_1, \dots, \mathbf{v}_p$ are linearly independent if the only way to make $$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$$ is with every $c_i = 0$. Otherwise they are linearly dependent, and a choice of weights, not all zero, that gives $\mathbf{0}$ is a dependence relation.

Key idea

The columns of $A$ are independent exactly when $A\mathbf{x} = \mathbf{0}$ has only the trivial solution, that is, when every column of $A$ has a pivot.

Why it works

Dependent means some vector is a combination of the others. If $c_1\mathbf{v}_1 + \cdots + c_p\mathbf{v}_p = \mathbf{0}$ with, say, $c_j \neq 0$, divide by $c_j$ and solve for $\mathbf{v}_j$: it equals a combination of the rest. Conversely, if $\mathbf{v}_j = \sum_{i \neq j} a_i\mathbf{v}_i$, then $\sum_{i \neq j} a_i\mathbf{v}_i - \mathbf{v}_j = \mathbf{0}$ is a dependence relation (the weight on $\mathbf{v}_j$ is $-1 \neq 0$). So dependent means "redundant": removing that vector does not shrink the span.

Worked example: Three vectors, none a multiple of another

Are $\mathbf{v}_1 = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}$, $\mathbf{v}_2 = \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix}$, $\mathbf{v}_3 = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ independent? If not, find a dependence relation.

Put them in the columns of $A$ and row reduce: $A$ $= \begin{bmatrix} 1 & 4 & 2 \\ 2 & 5 & 1 \\ 3 & 6 & 0 \end{bmatrix} \to \begin{bmatrix} 1 & 0 & -2 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{bmatrix}$.
Column 3 has no pivot, so $x_3$ is free: the vectors are dependent.
With $x_3 = 1$: $x_1 = 2$ and $x_2 = -1$. So $2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = \mathbf{0}$.
Check: $\begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} - \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} + \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}$ $= \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$. ✓ Equivalently, $\mathbf{v}_3 = \mathbf{v}_2 - 2\mathbf{v}_1$.

Answer: Dependent: $2\mathbf{v}_1 - \mathbf{v}_2 + \mathbf{v}_3 = \mathbf{0}$.

Common mistake

Checking only pairs. In the example, no vector is a multiple of another, yet the set is dependent: $\mathbf{v}_3$ is built from $\mathbf{v}_1$ and $\mathbf{v}_2$ together. The pair test works only for two vectors. For three or more, row reduce and look for a free variable.

Geometrically, in $\mathbb{R}^3$: two vectors are independent when they point along different lines; three are independent when they do not all lie in one plane through the origin. Independent vectors each add a new direction, so the span grows by one dimension with each of them.

Quick tests

Two vectors are dependent exactly when one is a multiple of the other.
Any set containing $\mathbf{0}$ is dependent ($1\cdot\mathbf{0} = \mathbf{0}$ is a dependence relation).
More vectors than entries ($p > n$ vectors in $\mathbb{R}^n$) is always dependent: the $n \times p$ matrix has more columns than rows, so some column has no pivot.

Worked example: No computation needed

Are $\begin{bmatrix} 1 \\ 2 \end{bmatrix}, \begin{bmatrix} 3 \\ -1 \end{bmatrix}, \begin{bmatrix} 0 \\ 5 \end{bmatrix}$ independent?

Three vectors in $\mathbb{R}^2$: the matrix is $2 \times 3$, with at most 2 pivots for 3 columns.
A free variable must exist, so the set is dependent.

Answer: Dependent.

When $\mathbf{v}$ is a multiple of $\mathbf{u}$, the two vectors are dependent and their span is only a line. Drag a tip off the line to make them independent: the span jumps to the whole plane.

Linear Independence

Quick tests

Practice

Builds on

Leads to

Sources and further reading