ProbabilityInequalities and Limit Theorems

The Central Limit Theorem for Sums

A sum of n independent copies of X is close to normal, with mean nμ and standard deviation σ√n.

Not started Jump to practice

The law of large numbers says an average settles near its mean. The central limit theorem says more: the shape of a sum of many independent pieces is close to a normal curve, whatever the pieces look like. In the sampling distribution of a mean you used this for averages; here it is stated for sums.

Key idea

If $X_1, \ldots, X_n$ are independent with mean $\mu$ and standard deviation $\sigma$, then for large $n$ the sum $S_n = X_1 + \cdots + X_n$ is approximately normal with $$E[S_n] = n\mu, \qquad \operatorname{SD}(S_n) = \sigma\sqrt{n}, \qquad P(S_n \le s) \approx \Phi\left(\frac{s - n\mu}{\sigma\sqrt{n}}\right).$$

Why it works

The mean and standard deviation of $S_n$ come from linearity and from adding the variances of independent terms ($n\sigma^{2}$). The normal shape is the deep part. A sketch with moment generating functions: standardize, $Z_n = \frac{S_n - n\mu}{\sigma\sqrt{n}}$. Its MGF is the product of $n$ identical factors, each about $1 + \frac{t^{2}}{2n}$ for large $n$, so $M_{Z_n}(t) \approx \left(1 + \frac{t^{2}}{2n}\right)^{n} \to e^{t^{2}/2}$, the MGF of the standard normal.
The sum of several independent uniform numbers. Add copies: the shape quickly becomes a bell curve with mean n times one half and standard deviation growing like the square root of n. Switch to the average to see it narrow instead.
Even from a skewed population, averages of samples pile up in a nearly normal shape.

Worked examples

Worked example: Boxes on a pallet

Boxes weigh 500 g on average with standard deviation 10 g, independently. Find the approximate probability that 36 boxes weigh more than 18.1 kg in total. Use $\Phi(1.67) = 0.9525$.
  1. $E[S] = 36 \times 500 = 18{,}000$ g and $\operatorname{SD}(S) = 10\sqrt{36} = 60$ g.
  2. $z = \frac{18{,}100 - 18{,}000}{60} \approx 1.67$.
  3. $$P(S > 18{,}100) \approx 1 - 0.9525 = 0.0475$$

Answer: about $0.048$

Worked example: Total call time

Calls last 5 minutes on average with standard deviation 3 minutes, independently. Find the approximate probability that 49 calls take more than 4 hours in total. Use $\Phi(0.24) = 0.5948$.
  1. $E[S] = 245$ minutes and $\operatorname{SD}(S) = 3\sqrt{49} = 21$ minutes.
  2. $z = \frac{240 - 245}{21} \approx -0.24$.
  3. $$P(S > 240) \approx 1 - \Phi(-0.24) = \Phi(0.24) = 0.5948$$

Answer: about $0.595$

Worked example: A total that is rarely exceeded

With the boxes above, find the weight that 36 boxes stay below about 95% of the time. Use $\Phi(1.645) \approx 0.95$.
  1. $$s = n\mu + 1.645\,\sigma\sqrt{n} = 18{,}000 + 1.645 \times 60 \approx 18{,}099$$ g.
  2. Two sums of the same pieces differ only in $n$: doubling the number of boxes doubles the mean but multiplies the spread by only $\sqrt{2}$.

Answer: about 18.1 kg

Common mistake

Using $\sigma n$ or $\frac{\sigma}{\sqrt{n}}$ for the spread of a sum. For a sum it is $\sigma\sqrt{n}$; $\frac{\sigma}{\sqrt{n}}$ belongs to the average, $\frac{S_n}{n}$. Check: a sum of more pieces is more spread out, but not in proportion to $n$, because independent errors partly cancel.

Practice

Pass with 3 right answers, the last 2 in a row. Each right answer earns 3 XP.

Loading practice…

Builds on

Leads to

Sources and further reading

  • Grinstead and Snell, Introduction to Probability (2nd revised edition, American Mathematical Society; free from Dartmouth College), chapter 9 (Central Limit Theorem), sections 9.2 and 9.3
  • MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2014), readings on the central limit theorem
@teddyrookbook GitHub