The law of total probability finds a probability by splitting into cases. The same idea works for averages: find the average within each case, then average those, weighting each case by its probability. Within a case the average comes from the conditional distribution.
Key idea
$$E[X \mid Y = y] = \sum_x x\,P(X = x \mid Y = y)$$ is the mean of $X$ within the slice $Y = y$. The tower rule: $$E[X] = \sum_y E[X \mid Y = y]\,P(Y = y),$$ written compactly as $E[X] = E\left[E[X \mid Y]\right]$.
Why it works
Start from $E[X] = \sum_{x, y} x\,p(x, y)$ and write each cell as $$p(x, y) = P(X = x \mid Y = y)P(Y = y)$$ Grouping by $y$: $$E[X] = \sum_y P(Y = y)\sum_x x\,P(X = x \mid Y = y) = \sum_y P(Y = y)\,E[X \mid Y = y].$$ So $E[X]$ is a weighted average of the slice means, with the slice probabilities as weights.
| $X = 0$ | $X = 1$ | $X = 2$ | $P(Y = y)$ | |
|---|---|---|---|---|
| $Y = 0$ | $\frac{1}{6}$ | $\frac{1}{6}$ | $0$ | $\frac{1}{3}$ |
| $Y = 1$ | $\frac{1}{12}$ | $\frac{1}{4}$ | $\frac{1}{3}$ | $\frac{2}{3}$ |
| $P(X = x)$ | $\frac{1}{4}$ | $\frac{5}{12}$ | $\frac{1}{3}$ | $1$ |
Worked examples
Worked example: Slice means from a table
From the table, find $E[X \mid Y = 0]$, $E[X \mid Y = 1]$ and $E[X]$.
- Given $Y = 0$: probabilities $\frac{1}{2}, \frac{1}{2}, 0$, so $E[X \mid Y = 0] = \frac{1}{2}$.
- Given $Y = 1$: probabilities $\frac{1}{8}, \frac{3}{8}, \frac{1}{2}$, so $E[X \mid Y = 1] = \frac{3}{8} + 1 = \frac{11}{8}$.
- $$E[X] = \frac{1}{3} \cdot \frac{1}{2} + \frac{2}{3} \cdot \frac{11}{8} = \frac{1}{6} + \frac{11}{12} = \frac{13}{12}$$ matching the marginal calculation.
Answer: $\frac{1}{2}$, $\frac{11}{8}$ and $\frac{13}{12}$
Worked example: A two-stage experiment
Roll a fair die to get $N$, then flip a fair coin $N$ times. Find the expected number of heads.
- Given $N = n$, the heads are binomial with mean $\frac{n}{2}$: $E[X \mid N] = \frac{N}{2}$.
- $E[X] = E\left[\frac{N}{2}\right] = \frac{1}{2} \cdot \frac{7}{2} = \frac{7}{4}$.
Answer: $\frac{7}{4}$
Worked example: Group averages
A school's students: 50% in group 1 with mean commute 30 minutes, 30% in group 2 with mean 40, 20% in group 3 with mean 60. Find the mean commute overall.
- $$E[X] = 0.5(30) + 0.3(40) + 0.2(60) = 15 + 12 + 12 = 39$$ minutes.
- The plain average of the three means, about 43.3, is wrong: group 3 is small.
Answer: 39 minutes
Common mistake
Averaging the conditional means without weighting them by the probabilities of their slices. Check: the overall mean must lie between the smallest and largest slice means, closer to the big slices; and it must agree with $E[X]$ computed from the marginal distribution.