Means of sums simply add, by linearity. Variances add only for uncorrelated variables; in general the covariance of every pair joins in. This matters most for counts made of dependent indicators.
Key idea
$$\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y),$$ and in general $$\operatorname{Var}(aX + bY) = a^{2}\operatorname{Var}(X) + b^{2}\operatorname{Var}(Y) + 2ab\operatorname{Cov}(X, Y)$$ For a longer sum, $$\operatorname{Var}\left(\sum X_i\right) = \sum \operatorname{Var}(X_i) + 2\sum_{i < j}\operatorname{Cov}(X_i, X_j).$$
Why it works
Write $\tilde X = X - \mu_X$ and $\tilde Y = Y - \mu_Y$. Then $$\operatorname{Var}(X + Y) = E[(\tilde X + \tilde Y)^{2}] = E[\tilde X^{2}] + E[\tilde Y^{2}] + 2E[\tilde X\tilde Y]$$ which is the formula. For $n$ terms the square of a sum has $n$ squares and $n(n - 1)$ cross terms, which pair up into $2\sum_{i < j}$. Independent (or merely uncorrelated) variables have all covariances 0, which is the rule from sums of random variables.
Worked examples
Worked example: Two correlated variables
$\operatorname{Var}(X) = 9$, $\operatorname{Var}(Y) = 4$, $\operatorname{Cov}(X, Y) = -2$. Find $\operatorname{Var}(X + Y)$ and $\operatorname{Var}(X - Y)$.
- $\operatorname{Var}(X + Y) = 9 + 4 + 2(-2) = 9$.
- $$\operatorname{Var}(X - Y) = 9 + 4 - 2(-2) = 17$$
- Negative covariance: the two partly cancel in the sum and reinforce in the difference.
Answer: 9 and 17
Worked example: Hearts in a hand
Deal 5 cards from a deck. $X$ is the number of hearts. Find $\operatorname{Var}(X)$ with indicators.
- $X = I_1 + \cdots + I_5$, $I_j = 1$ if card $j$ is a heart. $P(I_j = 1) = \frac{1}{4}$, so $\operatorname{Var}(I_j) = \frac{1}{4} \cdot \frac{3}{4} = \frac{3}{16}$.
- Two cards both hearts: $\frac{13 \cdot 12}{52 \cdot 51} = \frac{1}{17}$, so $\operatorname{Cov}(I_j, I_k) = \frac{1}{17} - \frac{1}{16} = -\frac{1}{272}$.
- $$\operatorname{Var}(X) = 5 \cdot \frac{3}{16} + 5 \cdot 4 \cdot \left(-\frac{1}{272}\right) = \frac{235}{272} \approx 0.86$$ a bit less than the binomial's $\frac{15}{16}$.
Answer: $\frac{235}{272}$
Worked example: Cards in their own place
For a shuffled deck of $n \ge 2$ cards, the number of cards in their original position has mean 1. Find its variance.
- Each indicator has variance $\frac{1}{n}\left(1 - \frac{1}{n}\right)$.
- Two given cards both stay with probability $\frac{1}{n(n - 1)}$, so each covariance is $\frac{1}{n(n - 1)} - \frac{1}{n^{2}} = \frac{1}{n^{2}(n - 1)}$.
- $$\operatorname{Var} = n \cdot \frac{n - 1}{n^{2}} + n(n - 1) \cdot \frac{1}{n^{2}(n - 1)} = \frac{n - 1}{n} + \frac{1}{n} = 1$$
Answer: 1, for every deck size.
With a correlation instead of a covariance, use $\operatorname{Cov}(X, Y) = \rho\,\sigma_X\sigma_Y$. This is why diversification works: two investments with standard deviation 10 each and correlation $-\frac{1}{2}$ have $$\operatorname{Var}(X + Y) = 100 + 100 - 100 = 100$$ so the total is no more variable than one investment alone.
Common mistake
Dropping the covariance terms when the pieces are dependent, as with cards dealt without replacement or a shuffled deck. Check: ask whether one piece tells you anything about another. If so, compute the pairwise covariances; there are $\binom{n}{2}$ pairs, each counted twice.