Stopping an infinite process, such as a Taylor series or a limit, leaves truncation error; an error of order hᵖ shrinks by a factor of about 2ᵖ when h is halved.
Many numerical methods replace an infinite process with a finite one: a series is cut off, a limit $h \to 0$ is replaced by a small $h$, an integral by a sum. The error this causes is truncation error. It belongs to the method and would be there even with exact arithmetic; round-off error, from storing a limited number of digits, is a separate thing. You met the first kind of truncation error in Taylor's Inequality.
We describe truncation error by how fast it shrinks with a step size $h$. The error $E(h)$ is of order $h^{p}$, written $E(h) = O(h^{p})$, when $|E(h)| \le Ch^{p}$ for some constant $C$ and all small enough $h$. The larger $p$, the faster the method improves as $h$ shrinks.
🔑 Key idea
If $E(h) \approx Ch^{p}$, halving $h$ divides the error by about $2^{p}$. Conversely, from the errors at $h$ and $h/2$ you can read off the order: $p \approx \log_2\frac{E(h)}{E(h/2)}$.
💡 Why it works
Taylor's theorem shows where the power comes from. For example, $$f(x + h) = f(x) + hf'(x) + \frac{h^{2}}{2}f''(\xi)$$ for some $\xi$ between $x$ and $x + h$, so $$\frac{f(x + h) - f(x)}{h} - f'(x) = \frac{h}{2}f''(\xi).$$ The truncation error is $O(h)$: as $h \to 0$, the leading term $\frac{h}{2}f''(x)$ dominates everything after it, so $E(h) \approx Ch$ with $C = \frac{1}{2}f''(x)$. A log–log plot of error against $h$ is then a straight line of slope $p$.
Error against h on log–log axes for two difference formulas. The forward difference has slope 1 (order h); the central difference has slope 2 (order h²).
✏️ Worked example: A truncated series
Approximate $e^{0.1}$ by $1 + 0.1 + \frac{0.1^{2}}{2}$ and bound the truncation error.
The sum is $1.105$.
Taylor's inequality with $n = 2$: $|R_2| \le \frac{M}{3!}(0.1)^{3}$, where $M$ bounds $e^{t}$ on $[0, 0.1]$; take $M = 1.2$.
Bound: $\frac{1.2 \times 0.001}{6} = 0.0002$. The actual error is $e^{0.1} - 1.105 = 0.000171\ldots$, inside the bound.
Answer: error at most $0.0002$
Taylor polynomials of eˣ about 0. Cutting the series off after the xⁿ term leaves a truncation error, smallest near the center.
✏️ Worked example: Reading the order from data
A method gives errors $1.64 \times 10^{-2}$ with $h = 0.1$ and $4.1 \times 10^{-3}$ with $h = 0.05$. What is its order?
A method of order 4 has error $3.2 \times 10^{-5}$ with $h = 0.2$. Estimate the error with $h = 0.1$, and the $h$ needed for $10^{-9}$.
Halving $h$ divides the error by $2^{4} = 16$: $3.2 \times 10^{-5} \div 16 = 2 \times 10^{-6}$.
For $10^{-9}$ the error must shrink by $3.2 \times 10^{4}$ from $h = 0.2$, so $h$ must shrink by $(3.2 \times 10^{4})^{1/4} \approx 13.4$: $h \approx 0.015$.
Answer: about $2 \times 10^{-6}$; $h \approx 0.015$
⚠️ Common mistake
Mixing up truncation error with round-off error. Truncation error comes from the method (a cut-off series, a finite $h$) and shrinks as you take more terms or smaller steps. Round-off comes from storing about 16 digits and does not depend on the method's formula. Test: would the error vanish with infinitely many digits? If not, it is truncation error.
Practice
Pass with 3 right answers, the last 2 in a row. Each right answer earns 2 XP.
Tobin A. Driscoll and Richard J. Braun, Fundamentals of Numerical Computation (University of Delaware, free online edition): chapter "Piecewise interpolation", section "Convergence of finite differences" (truncation error and order of accuracy)
MIT OpenCourseWare 18.330, Introduction to Numerical Analysis: lecture notes on Taylor series and orders of convergence