Flip a coin many times and the fraction of heads drifts toward one half. Why must it? The variance of a sum shows that an average of many independent trials has small spread, and Chebyshev's inequality turns small spread into a guarantee.
🔑 Key idea
If $X_1, \ldots, X_n$ are independent with mean $\mu$ and standard deviation $\sigma$, their average $\bar X_n$ has $$E[\bar X_n] = \mu, \qquad \operatorname{Var}(\bar X_n) = \frac{\sigma^{2}}{n}, \qquad P(\lvert \bar X_n - \mu \rvert \ge \varepsilon) \le \frac{\sigma^{2}}{n\varepsilon^{2}} \to 0.$$ This is the (weak) law of large numbers.
💡 Why it works
By linearity $E[\bar X_n] = \frac{1}{n} \cdot n\mu = \mu$. The variances of independent terms add, so $\operatorname{Var}(X_1 + \cdots + X_n) = n\sigma^{2}$; dividing by $n$ divides the variance by $n^{2}$, leaving $\frac{\sigma^{2}}{n}$. Chebyshev's inequality with $c = \varepsilon$ gives the bound, and for any fixed $\varepsilon$ it shrinks to 0 as $n$ grows. The spread of the average, $\frac{\sigma}{\sqrt{n}}$, falls like one over the square root of $n$: four times as many trials halve it.
Rolling one die again and again: the running share of sixes wanders at first, then settles near one sixth.
Worked examples
✏️ Worked example: Averaging measurements
One measurement has standard deviation 5. Find the standard deviation of the average of 100 independent measurements.
The 10 extra heads are never "paid back"; they simply matter less and less as $n$ grows.
Answer: about $0.505$
Chebyshev's bound is a guarantee for every distribution, so it asks for more trials than are really needed. The central limit theorem gives the realistic picture: the fraction of heads in $n$ flips is roughly normal with standard deviation $\frac{1}{2\sqrt{n}}$, and in the coin example about 665 flips already give 99% confidence of being within $0.05$, against Chebyshev's 10,000.
⚠️ Common mistake
Expecting the next results to "make up" for a streak, so that tails are more likely after many heads. The coin has no memory: each flip is still 50–50. The average settles because the early results are swamped by many later ones, not corrected. Check: the law is about averages over many trials, never about the next single trial.
Practice
Pass with 3 right answers, the last 2 in a row. Each right answer earns 2 XP.
Grinstead and Snell, Introduction to Probability (2nd revised edition, American Mathematical Society; free from Dartmouth College), chapter 8 (Law of Large Numbers), section 8.1
MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2014), readings on the law of large numbers