The expected value $E[X] = \sum x\,P(X = x)$ is the long-run average of $X$. Often we need the average of something computed from $X$: a payoff, a square, a cost. There is no need to find the distribution of that new quantity first.
Key idea
For a function $g$, $$E[g(X)] = \sum_x g(x)\,P(X = x).$$ Apply $g$ to each value, then weight by the probabilities. In general $E[g(X)] \ne g(E[X])$; they agree when $g(x) = ax + b$.
Why it works
Let $Y = g(X)$. By definition $E[Y] = \sum_y y\,P(Y = y)$, and $P(Y = y)$ adds $P(X = x)$ over all $x$ with $g(x) = y$. So $$E[Y] = \sum_y \sum_{x:\,g(x) = y} g(x)\,P(X = x) = \sum_x g(x)\,P(X = x),$$ since every $x$ appears exactly once in the double sum. For a straight line, $$E[aX + b] = a\sum x\,p(x) + b\sum p(x) = aE[X] + b$$ For a curve there is no such shortcut.
| $x$ | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| $P(X = x)$ | $\frac{1}{6}$ | $\frac{1}{6}$ | $\frac{1}{6}$ | $\frac{1}{6}$ | $\frac{1}{6}$ | $\frac{1}{6}$ |
| $x^{2}$ | 1 | 4 | 9 | 16 | 25 | 36 |
| $\max(x - 4, 0)$ | 0 | 0 | 0 | 0 | 1 | 2 |
Worked examples
Worked example: The square of a die roll
Roll a fair die. Find $E[X^{2}]$ and compare it with $(E[X])^{2}$.
- $$E[X^{2}] = \frac{1 + 4 + 9 + 16 + 25 + 36}{6} = \frac{91}{6} \approx 15.17$$
- $E[X] = \frac{7}{2}$, so $(E[X])^{2} = \frac{49}{4} = 12.25$.
- They differ by $\frac{91}{6} - \frac{49}{4} = \frac{35}{12}$, which will turn out to be the variance.
Answer: $E[X^{2}] = \frac{91}{6}$, larger than $(E[X])^{2} = \frac{49}{4}$.
Worked example: A payoff
A game pays $\max(X - 4, 0)$ dollars on a fair die roll $X$: \$1 for a 5, \$2 for a 6, nothing otherwise. Find the expected payoff.
- $$E[\max(X - 4, 0)] = 0 \cdot \frac{4}{6} + 1 \cdot \frac{1}{6} + 2 \cdot \frac{1}{6} = \frac{1}{2}$$
- Plugging in the mean gives $\max(3.5 - 4, 0) = 0$, which would call the game worthless.
Answer: \$0.50
Worked example: A reciprocal
For the die, find $E\left[\frac{1}{X}\right]$.
- $$E\left[\frac{1}{X}\right] = \frac{1}{6}\left(1 + \frac{1}{2} + \frac{1}{3} + \frac{1}{4} + \frac{1}{5} + \frac{1}{6}\right) = \frac{1}{6} \cdot \frac{49}{20} = \frac{49}{120} \approx 0.41$$
- Compare $\frac{1}{E[X]} = \frac{2}{7} \approx 0.29$.
Answer: $\frac{49}{120}$
The same rule handles any function the problems might use: $\lvert X - c \rvert$ (an average distance), $X(X - 1)$ (useful for counting pairs), $2^{X}$ (a doubling payoff) or $(X - c)^{2}$. In each case make a row of $g$-values, multiply by the probabilities, and add.
Common mistake
Finding $E[X^{2}]$ by squaring $E[X]$. Averaging and squaring do not commute: big values get squared into much bigger ones, so the average of the squares is larger. Check with a variable that is $-1$ or $1$ with equal chances: $E[X] = 0$, but $X^{2} = 1$ always, so $E[X^{2}] = 1$.
For a curve that bends upward (convex), such as $x^{2}$, $e^{x}$ or $\frac{1}{x}$ for $x > 0$, the inequality always goes one way: $E[g(X)] \ge g(E[X])$. This is Jensen's inequality.