Maximum and Minimum of Independent Variables

A system of parts in series fails when its first part fails; a job finishes when its slowest worker does. These are the minimum and the maximum of several random variables. With independence and CDFs both have neat formulas, and the minimum of exponential times is exponential again.

Key idea

For independent $X_1, \ldots, X_n$ with the same CDF $F$: $$P(\max \le x) = F(x)^{n}, \qquad P(\min > x) = (1 - F(x))^{n}.$$ If the $X_i$ are exponential with rates $\lambda_1, \ldots, \lambda_n$, the minimum is exponential with rate $\lambda_1 + \cdots + \lambda_n$, and $X_i$ is the first with probability $\frac{\lambda_i}{\lambda_1 + \cdots + \lambda_n}$.

Why it works

The largest is at most $x$ exactly when every $X_i$ is at most $x$; by independence, the probabilities multiply. Likewise the smallest exceeds $x$ exactly when every $X_i$ does. For exponentials, $$P(\min > t) = \prod e^{-\lambda_i t} = e^{-(\lambda_1 + \cdots + \lambda_n)t}$$ Working through these "all of them" statements avoids the hard "at least one" cases.

Two independent uniform numbers: the larger is below 0.6 exactly when the point lies in the small square, whose area is 0.6 squared.

Worked examples

Worked example: Dice

Roll 3 dice. Find the probability that the largest is at most 4, and that it is exactly 4.

$P(\max \le 4) = \left(\frac{4}{6}\right)^{3} = \frac{8}{27}$.
$$P(\max = 4) = P(\max \le 4) - P(\max \le 3) = \frac{8}{27} - \frac{1}{8} = \frac{37}{216}$$

Answer: $\frac{8}{27}$ and $\frac{37}{216}$

Worked example: The largest of n uniforms

$n$ numbers are uniform on $[0, 1]$, independent. Find the expected largest.

$F_{\max}(x) = x^{n}$, so the density is $nx^{n - 1}$ on $[0, 1]$.
$E[\max] = \int_{0}^{1} x \cdot nx^{n - 1}\,dx = \frac{n}{n + 1}$. Similarly $E[\min] = \frac{1}{n + 1}$.
The $n$ points split $[0, 1]$ into $n + 1$ gaps of equal average length.

Answer: $\frac{n}{n + 1}$

Worked example: Parts in series

A device has two parts with independent exponential lifetimes, means 4 years and 12 years, and fails when either part fails. Find its mean lifetime and the chance that the first part fails first.

Rates: $\frac{1}{4}$ and $\frac{1}{12}$ per year; total $\frac{1}{3}$.
The device's lifetime is exponential with rate $\frac{1}{3}$: mean 3 years.
The first part fails first with probability $\frac{1/4}{1/3} = \frac{3}{4}$.

Answer: 3 years; $\frac{3}{4}$

The exponential case has a memorable form. Think of independent alarm clocks ringing at exponential times. The first ring comes at rate $\lambda_1 + \cdots + \lambda_n$, so the chance that all of them are still silent at time $t$ is $e^{-(\lambda_1 + \cdots + \lambda_n)t}$, and clock $i$ rings first with probability equal to its share of the total rate. After the first ring the others start fresh, because exponentials have no memory.

Common mistake

Writing $P(\max \le x) = n \cdot F(x)$ (which can exceed 1), or attacking $P(\min \le x)$ directly, which needs inclusion–exclusion. Check: rewrite the event about the max or min as "all of them" and multiply; for the min, use the complement: $P(\min \le x) = 1 - (1 - F(x))^{n}$.

Sources and further reading

MIT OpenCourseWare 18.440, Probability and Random Variables (Spring 2014), lecture notes on maxima, minima and competing exponential clocks

Grinstead and Snell, Introduction to Probability (2nd revised edition, American Mathematical Society; free from Dartmouth College), chapter 5 (Distributions and Densities), section 5.2, the exponential density

Maximum and Minimum of Independent Variables

Worked examples

Practice

Builds on

Leads to

Sources and further reading