ProbabilityDiscrete Random Variables

Linearity of Expectation and Indicators

E[X + Y] = E[X] + E[Y] always, so a count has mean equal to the sum of its indicators' probabilities: a shuffled deck has on average 1 card left in its starting place.

Not started Jump to practice

In sums of random variables you added means. The rule is stronger than it first looks: it needs no independence at all. Together with indicator variables it computes expected counts that would be hard to find any other way, building on expected values of functions.

Key idea

$E[X + Y] = E[X] + E[Y]$ for any random variables, dependent or not. An indicator $I_A$ is 1 when $A$ happens and 0 otherwise, and $E[I_A] = P(A)$. So if a count is $X = I_{A_1} + \cdots + I_{A_n}$, then $$E[X] = P(A_1) + \cdots + P(A_n).$$

Why it works

For discrete $X$ and $Y$ with joint probabilities $p(x, y)$: $$E[X + Y] = \sum_{x,y} (x + y)\,p(x, y) = \sum_x x \sum_y p(x, y) + \sum_y y \sum_x p(x, y) = E[X] + E[Y],$$ because summing the joint probabilities over one variable gives the other's distribution. Nothing in this used independence. And $$E[I_A] = 1 \cdot P(A) + 0 \cdot P(A^{c}) = P(A)$$

Worked examples

Worked example: Cards in their own place

A deck of 52 cards, numbered by position, is shuffled. Find the expected number of cards that end up in their original position.
  1. Let $I_k = 1$ when card $k$ stays in position $k$. In a random order, card $k$ is equally likely to be in any of the 52 spots, so $P(I_k = 1) = \frac{1}{52}$.
  2. The $I_k$ are dependent (if 51 cards stay, so does the last), but linearity does not care.
  3. $E[X] = 52 \cdot \frac{1}{52} = 1$, for a deck of any size.

Answer: 1

Worked example: Shared birthdays

30 people have independent, uniformly random birthdays (365 days). Find the expected number of pairs of people who share a birthday.
  1. One indicator per pair: there are $\binom{30}{2} = 435$ pairs.
  2. Each pair matches with probability $\frac{1}{365}$.
  3. $E[X] = \frac{435}{365} = \frac{87}{73} \approx 1.19$. Compare the birthday problem: a match is likely, and on average there is more than one.

Answer: $\frac{87}{73}$

Worked example: Different faces

Roll a die 6 times. Find the expected number of different faces that appear.
  1. Let $I_f = 1$ if face $f$ appears at least once. Face $f$ is missed by all 6 rolls with probability $\left(\frac{5}{6}\right)^{6}$.
  2. $E[I_f] = 1 - \left(\frac{5}{6}\right)^{6}$.
  3. $$E[X] = 6\left(1 - \left(\frac{5}{6}\right)^{6}\right) = \frac{31031}{7776} \approx 3.99$$ about 4 faces, not 6.

Answer: $\frac{31031}{7776} \approx 3.99$

The same recipe handles many counts. Name the small events, find the chance of one of them, and multiply by how many there are:

  • Patterns in coin flips. In $n$ flips there are $n - 1$ neighboring pairs, each HH with chance $\frac{1}{4}$, so the expected number of HH is $\frac{n - 1}{4}$. A new run starts at each switch, and each neighboring pair is a switch with chance $\frac{1}{2}$: expected runs $= 1 + \frac{n - 1}{2}$.
  • Empty boxes. Toss $b$ balls into $m$ boxes: box $j$ stays empty with chance $\left(\frac{m - 1}{m}\right)^{b}$, so the expected number of empty boxes is $m\left(\frac{m - 1}{m}\right)^{b}$.
  • Mixed neighbors. Line up $b$ boys and $g$ girls at random ($n = b + g$). Each of the $n - 1$ neighboring pairs is mixed with chance $\frac{2bg}{n(n - 1)}$, so the expected number of mixed pairs is $\frac{2bg}{n}$.

Common mistake

Believing that $E[X + Y] = E[X] + E[Y]$ needs independence, and so giving up on counts made of dependent pieces. Expected values always add. (Variances are different: there the dependence enters through covariance terms, as in the variance of a sum.) Check small cases by listing outcomes: for a 2-card deck, both cards stay with probability $\frac{1}{2}$, so the expected number is $2 \cdot \frac{1}{2} = 1$.

Practice

Pass with 3 right answers, the last 2 in a row. Each right answer earns 3 XP.

Loading practice…

Builds on

Leads to

Sources and further reading

  • Grinstead and Snell, Introduction to Probability (2nd revised edition, American Mathematical Society; free from Dartmouth College), chapter 6 (Expected Value and Variance), section 6.1, sums of random variables
  • MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2014), readings on expectation and indicator variables
@teddyrookbook GitHub