In Jacobi's method, the new $x$ is already known when the new $y$ is computed, yet Jacobi ignores it. Gauss–Seidel uses each new value as soon as it exists: $$x_i^{(k+1)} = \frac{1}{a_{ii}}\left(b_i - \sum_{j < i}a_{ij}x_j^{(k+1)} - \sum_{j > i}a_{ij}x_j^{(k)}\right).$$ In matrix form, $(D + L)\mathbf{x}^{(k+1)} = \mathbf{b} - U\mathbf{x}^{(k)}$.
🔑 Key idea
Gauss–Seidel updates the variables one at a time, always with the newest values. For strictly diagonally dominant matrices it converges, usually faster than Jacobi, and it needs only one copy of $\mathbf{x}$ in memory.
💡 Why it works
Fresh values are closer to the solution, so using them at once can only help when the iteration is converging. For a $2 \times 2$ system the effect is exact: the Gauss–Seidel iteration matrix has eigenvalues $0$ and $\frac{a_{12}a_{21}}{a_{11}a_{22}}$, while Jacobi's are $\pm\sqrt{\frac{a_{12}a_{21}}{a_{11}a_{22}}}$. So $\rho_{GS} = \rho_J^{2}$: one Gauss–Seidel sweep does as much as two Jacobi sweeps. (For many larger matrices from differential equations the same relation holds.)
✏️ Worked example: Three sweeps
Apply Gauss–Seidel to $4x + y = 6$, $2x + 5y = 9$ from $(0, 0)$.
Sweep 1: $x = \frac{6 - 0}{4} = 1.5$, then $y = \frac{9 - 2(1.5)}{5} = 1.2$, using the new $x$.
Errors after three sweeps: about $0.003$ and $0.001$, versus $0.03$ and $0.05$ for Jacobi. Here $\rho_{GS} = 0.1$ and $\rho_J \approx 0.32$.
Answer: $(1.17, 1.332)$
Gauss–Seidel moves along a staircase: first across to the first line (new x), then up or down to the second line (new y).
✏️ Worked example: Spot the method
A program computes, in each sweep, x_new = (6 − y_old)/4 and y_new = (9 − 2·x_old)/5. Is it Gauss–Seidel?
The $y$ update uses x_old, the value from the previous sweep.
That is Jacobi's method. Gauss–Seidel would use x_new in the $y$ update.
Answer: No: it is Jacobi
⚠️ Common mistake
Keeping last sweep's values for variables that were already updated in this sweep. That is Jacobi's method, not Gauss–Seidel. Check each formula: every variable that comes earlier in the order must appear with its new value.
Strict diagonal dominance is one guarantee; another is that $A$ is symmetric with positive eigenvalues (symmetric positive definite), which is common in physics and engineering problems. In both cases Gauss–Seidel converges from any starting vector. A small change, multiplying each update by a factor $\omega$ between 1 and 2 (successive over-relaxation), can speed it up a great deal more.
Because Gauss–Seidel uses new values as it goes, its result depends on the order of the variables, and it is harder to run in parallel than Jacobi. Both are the starting points for faster methods such as successive over-relaxation and the Krylov methods used for very large systems.
Practice
Pass with 3 right answers, the last 2 in a row. Each right answer earns 2 XP.