In the addition rule you met rules such as $$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$ and $P(A^{c}) = 1 - P(A)$. They feel like separate facts, but each one follows from three short statements. Probability as a branch of mathematics starts from these three axioms, set down by Andrey Kolmogorov in 1933, and every rule in this course is built on them.
A sample space $S$ is the set of all outcomes of an experiment, and an event is a set of outcomes (a subset of $S$). We write $A^{c}$ for the complement of $A$ (some books write $A'$), and call events disjoint (or mutually exclusive) when they share no outcome.
Key idea
1. $P(A) \ge 0$ for every event $A$;
2. $P(S) = 1$;
3. if $A_1, A_2, \ldots$ are pairwise disjoint, then $P(A_1 \cup A_2 \cup \cdots) = \sum_{i} P(A_i)$.
Why it works
Complement: $A$ and $A^{c}$ are disjoint and together make $S$, so axioms 3 and 2 give $P(A) + P(A^{c}) = 1$. With $A = S$ this also shows $P(\varnothing) = 0$.
Bigger events are more likely: if $A \subseteq B$, then $B$ is the disjoint union of $A$ and $B \cap A^{c}$, so $P(B) = P(A) + P(B \cap A^{c}) \ge P(A)$. In particular $P(A) \le P(S) = 1$.
Addition rule: $A \cup B$ is the disjoint union of $A$ and $B \cap A^{c}$, while $B$ is the disjoint union of $A \cap B$ and $B \cap A^{c}$. Subtracting, $$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$
The square is a good mental model: area is never negative (axiom 1), the whole square has area 1 (axiom 2), and areas of regions that do not overlap add (axiom 3). It also makes two bounds easy to see. An overlap fits inside each event, so $P(A \cap B) \le \min(P(A), P(B))$. And a union is never bigger than the whole square or than the two areas together, so $P(A \cup B) \le \min(1,\ P(A) + P(B))$.
Worked examples
Worked example: Finding a missing probability
- The four outcomes are disjoint and together they make up $S$.
- By axioms 3 and 2, their probabilities add to 1.
- $$P(\text{yellow}) = 1 - (0.15 + 0.3 + 0.25) = 0.3$$
Answer: $0.3$
Worked example: How much must two events overlap?
- By the addition rule, $$P(A \cap B) = P(A) + P(B) - P(A \cup B)$$
- $P(A \cup B)$ is at most 1, so $P(A \cap B) \ge 0.7 + 0.6 - 1 = 0.3$.
- The value $0.3$ is reached when $A \cup B$ is the whole sample space.
Answer: $0.3$: the events must overlap at least that much.
Worked example: Spotting an impossible claim
- $A \cap B \subseteq A$, so $P(A \cap B) \le P(A) = 0.4$.
- $0.45 > 0.4$ breaks that rule.
Answer: No: an overlap can never be more likely than either event.
Common mistake