ProbabilityJoint Distributions

Conditional Distributions and Independent Variables

P(X = x ∣ Y = y) divides a row of the joint table by the row's total; X and Y are independent when every cell equals the product of its marginals.

Not started Jump to practice

Once a row of a joint table is known to have happened, the other variable has a new distribution: the conditional probability idea, applied to every cell of that row. Comparing those conditional distributions is also how independence of two random variables is tested.

Key idea

$$P(X = x \mid Y = y) = \frac{p(x, y)}{p_Y(y)}:$$ divide the row $Y = y$ by its total. $X$ and $Y$ are independent when $p(x, y) = p_X(x)\,p_Y(y)$ for every cell; equivalently, every row's conditional distribution is the same.

Why it works

Given $Y = y$, the sample space shrinks to that row, and dividing by the row total makes the new probabilities add to 1. If every cell factors as $p_X(x)p_Y(y)$, then dividing row $y$ by $p_Y(y)$ leaves $p_X(x)$: knowing $Y$ changes nothing about $X$. If even one cell fails to factor, the conditional distribution in that row differs from the marginal, so $Y$ carries information about $X$.
A joint pmf with its marginals in the margins.
$X = 0$$X = 1$$X = 2$$P(Y = y)$
$Y = 0$$\frac{1}{6}$$\frac{1}{6}$$0$$\frac{1}{3}$
$Y = 1$$\frac{1}{12}$$\frac{1}{4}$$\frac{1}{3}$$\frac{2}{3}$
$P(X = x)$$\frac{1}{4}$$\frac{5}{12}$$\frac{1}{3}$$1$
Click a row label to condition on it: the row is divided by its total and drawn as bars that add to 1. Switch views to test every cell for independence.

Worked examples

Worked example: A conditional distribution

From the table, find the distribution of $X$ given $Y = 1$.
  1. Row $Y = 1$: $\frac{1}{12}, \frac{1}{4}, \frac{1}{3}$, with total $\frac{2}{3}$.
  2. Divide: $P(X = 0 \mid Y = 1) = \frac{1}{8}$, $P(X = 1 \mid Y = 1) = \frac{3}{8}$, $P(X = 2 \mid Y = 1) = \frac{1}{2}$.
  3. They add to 1. Given $Y = 0$ instead, the distribution is $\frac{1}{2}, \frac{1}{2}, 0$: very different, so $X$ and $Y$ are dependent.

Answer: $\left(\frac{1}{8}, \frac{3}{8}, \frac{1}{2}\right)$

Worked example: One matching cell is not enough

A joint table has rows $Y = 0$: $\frac{3}{16}, \frac{3}{16}, \frac{1}{8}$ and $Y = 1$: $\frac{1}{16}, \frac{5}{16}, \frac{1}{8}$ (for $X = 0, 1, 2$). Are $X$ and $Y$ independent?
  1. Marginals: $p_X = \left(\frac{1}{4}, \frac{1}{2}, \frac{1}{4}\right)$ and $p_Y = \left(\frac{1}{2}, \frac{1}{2}\right)$.
  2. Column $X = 2$ factors: $\frac{1}{8} = \frac{1}{4} \cdot \frac{1}{2}$ in both rows.
  3. But $p(0, 0) = \frac{3}{16}$ while $p_X(0)p_Y(0) = \frac{1}{8}$.

Answer: No. Two cells factor, but independence needs all six.

Conditional distributions can also be the starting data. If $P(Y = 0) = \frac{1}{4}$ and you know the distribution of $X$ in each row, then each cell is $$p(x, y) = P(X = x \mid Y = y)\,P(Y = y)$$ adding a column gives $P(X = x)$, and dividing a cell by its column total reverses the conditioning, as in Bayes' theorem.

Common mistake

Checking one cell, finding that it factors, and declaring $X$ and $Y$ independent. Independence is a statement about every cell. Check: compare whole conditional distributions, or test each cell against the product of its two marginals.

Practice

Pass with 3 right answers, the last 2 in a row. Each right answer earns 2 XP.

Loading practice…

Builds on

Leads to

Sources and further reading

  • MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2014), readings on joint distributions and independence
  • Grinstead and Snell, Introduction to Probability (2nd revised edition, American Mathematical Society; free from Dartmouth College), chapter 4 (Conditional Probability), section 4.1
@teddyrookbook GitHub