https://teddyrookbook.com/math/numerical-analysis/Recent content in Numerical Analysis on Teddy Rook Bookhttps://teddyrookbook.com/img/share.jpghttps://teddyrookbook.com/img/share.jpgHugoenhttps://teddyrookbook.com/math/numerical-analysis/error-measures/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/error-measures/Absolute error is |approximation − true value|; relative error divides by |true value| and tells how many significant digits are correct. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/binary-fractions/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/binary-fractions/Convert a decimal fraction to base 2 by doubling and keeping the whole-number parts: 0.1 becomes 0.000110011… and never ends. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/floating-point/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/floating-point/A float stores ±1.f × 2ᵉ with a fixed number of bits, so the gaps between floats grow with their size; machine epsilon, 2⁻⁵² ≈ 2.2 × 10⁻¹⁶ in double precision, sets the relative precision. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/cancellation/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/cancellation/Subtracting nearly equal numbers wipes out correct digits; rewrite the formula to avoid it, as in √(x + 1) − √x = 1/(√(x + 1) + √x). Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/truncation-error/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/truncation-error/Stopping an infinite process, such as a Taylor series or a limit, leaves truncation error; an error of order hᵖ shrinks by a factor of about 2ᵖ when h is halved. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/conditioning/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/conditioning/The condition number κ = |x f′(x)/f(x)| says how much a relative change in the input is magnified in the output, whatever method is used. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/stability/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/stability/A stable algorithm gives the exact answer to a slightly changed problem, so two formulas for the same quantity can behave very differently in floating point. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/horner/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/horner/Evaluate a polynomial by nesting, ((aₙx + aₙ₋₁)x + ⋯)x + a₀, with only n multiplications and n additions. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/bisection/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/bisection/Keep halving an interval where f changes sign; after n steps the midpoint is within (b − a)/2ⁿ⁺¹ of a root. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/fixed-point-iteration/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/fixed-point-iteration/Rewrite f(x) = 0 as x = g(x) and repeat xₙ₊₁ = g(xₙ); it converges near the fixed point when |g′| < 1 there. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/convergence-order/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/convergence-order/If eₙ₊₁ ≈ C eₙᵖ, the method has order p: a linear method gains about the same number of digits each step, and a quadratic one doubles them. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/newton-convergence/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/newton-convergence/Near a simple root, Newton's method converges quadratically; a poor start, a tangent that is nearly flat, or a multiple root can make it fail or slow down. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/secant-method/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/secant-method/Replace f′ with the slope through the last two points; it needs no derivative, and its order is about 1.618. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/newton-systems/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/newton-systems/Solve J(x) Δ = −F(x) with the Jacobian matrix J, then update x to x + Δ. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/operation-counts/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/operation-counts/Gaussian elimination takes about 2n³/3 arithmetic operations and each triangular solve about n², so doubling n multiplies the work by about 8. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/partial-pivoting/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/partial-pivoting/Before each elimination step, swap up the row whose entry in the pivot column is largest in size, so tiny pivots do not magnify round-off. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/lu-solve/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/lu-solve/Factor once with pivoting, then solve Ly = Pb by forward substitution and Ux = y by back substitution for each new right-hand side b. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/norms/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/norms/Measure size with ‖x‖₁, ‖x‖₂ and ‖x‖∞; the matching matrix norms ‖A‖₁ and ‖A‖∞ are the largest absolute column sum and the largest absolute row sum. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/condition-number/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/condition-number/κ(A) = ‖A‖ ‖A⁻¹‖ bounds how much a relative change in b can grow in x; a small residual does not guarantee a small error. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/jacobi/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/jacobi/Solve each equation for its own variable and update every variable at once from the previous values; strict diagonal dominance guarantees convergence. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/gauss-seidel/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/gauss-seidel/Use each new value as soon as it is computed; for strictly diagonally dominant systems it converges, usually faster than Jacobi. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/lagrange/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/lagrange/Exactly one polynomial of degree at most n passes through n + 1 points with different x-values; build it from basis polynomials that are 1 at one node and 0 at the others. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/divided-differences/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/divided-differences/Build the interpolating polynomial one point at a time from a table of divided differences, without starting over. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/interpolation-error/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/interpolation-error/f(x) − p(x) = f⁽ⁿ⁺¹⁾(ξ)/(n + 1)! · (x − x₀)(x − x₁)⋯(x − xₙ) for some ξ in the interval. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/runge-chebyshev/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/runge-chebyshev/High-degree interpolation at equally spaced points can swing wildly near the ends; Chebyshev nodes, bunched toward the ends, fix it. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/cubic-splines/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/cubic-splines/Join cubic pieces that match values, slopes and second derivatives at the knots; a natural spline has second derivative 0 at both ends. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/finite-differences/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/finite-differences/The forward difference (f(x + h) − f(x))/h has error of order h; the central difference (f(x + h) − f(x − h))/(2h) has error of order h². Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/step-size-tradeoff/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/step-size-tradeoff/Shrinking h cuts truncation error but magnifies round-off, so the total error is smallest at a moderate h, about 10⁻⁸ for a forward difference in double precision. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/richardson/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/richardson/Combine results at h and h/2 to cancel the leading error term: for a method with error of order h², use (4A(h/2) − A(h))/3. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/composite-rules/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/composite-rules/The composite trapezoid rule has error of order h² and Simpson's rule of order h⁴, so halving h cuts the error by about 4 or 16. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/romberg/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/romberg/Apply Richardson extrapolation to trapezoid estimates again and again to fill a triangle of better and better values. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/gaussian-quadrature/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/gaussian-quadrature/Choose the nodes as well as the weights: two points at ±1/√3 integrate every cubic exactly on [−1, 1]. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/adaptive-quadrature/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/adaptive-quadrature/Compare Simpson's rule on an interval with Simpson's rule on its two halves; the difference divided by 15 estimates the error, and only the intervals that fail the test are split. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/ode-error/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/ode-error/Each Euler step makes a local error of order h², and the 1/h steps together leave a global error of order h. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/ode-systems-numerical/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/ode-systems-numerical/Rewrite y″ = f(t, y, y′) as a first-order system, then take Euler or Runge–Kutta steps with vectors. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/absolute-stability/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/absolute-stability/For y′ = λy with λ < 0, Euler's method gives yₙ = (1 + hλ)ⁿ y₀, which stays bounded only when |1 + hλ| ≤ 1, that is, when h ≤ 2/|λ|. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/implicit-euler/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/implicit-euler/Backward Euler uses the slope at the new point, so each step means solving for yₙ₊₁; for y′ = λy with λ < 0 it is stable for every step size. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/multistep/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/multistep/Adams–Bashforth methods reuse earlier slopes, as in yₙ₊₁ = yₙ + (h/2)(3fₙ − fₙ₋₁), and need a one-step method to get started. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/bvp-finite-differences/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/bvp-finite-differences/Replace y″ by (yᵢ₊₁ − 2yᵢ + yᵢ₋₁)/h² at each inside grid point to get a tridiagonal linear system. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/heat-explicit/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/heat-explicit/uᵢⁿ⁺¹ = uᵢⁿ + r(uᵢ₊₁ⁿ − 2uᵢⁿ + uᵢ₋₁ⁿ) with r = kΔt/Δx² is stable only when r ≤ 1/2. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/least-squares-qr/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/least-squares-qr/Solve Rx = Qᵀb instead of the normal equations, because forming AᵀA squares the condition number. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/power-method/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/power-method/Multiply by A and rescale, again and again; the vector turns toward the dominant eigenvector at a rate set by |λ₂/λ₁|. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/inverse-iteration/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/inverse-iteration/Run the power method on (A − sI)⁻¹, solving a system each step instead of inverting, to find the eigenvalue of A closest to the shift s. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/gershgorin/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/gershgorin/Every eigenvalue lies in a disk centered at a diagonal entry aᵢᵢ, with radius equal to the sum of the other |aᵢⱼ| in that row. Numerical Analysis · Math Ladderhttps://teddyrookbook.com/math/numerical-analysis/qr-algorithm/Mon, 01 Jan 0001 00:00:00 +0000https://teddyrookbook.com/math/numerical-analysis/qr-algorithm/Factor Aₖ = QₖRₖ and set Aₖ₊₁ = RₖQₖ; each new matrix is similar to A, and its diagonal approaches the eigenvalues. Numerical Analysis · Math Ladder