Econometrics · Chapter 12

Difference-in-Differences and Event Studies

A state raises its minimum wage. A year later, restaurants there employ a few more people per store. Did the law do that, or was everyone hiring anyway? You can’t rerun history without the law, but you can borrow a neighbor’s history. Here you’ll build that comparison with a dashed line, break the one assumption it rests on, read an event-study plot the way referees do, and watch the most popular regression in applied economics give the wrong sign when policies roll out at different times.

Scenario

Comparison

True effect of the policy Break parallel trends Stores per group Noise Show the true path without the policy Show each store’s path

Live numbers

Missions

What you’re seeing

The story. Two groups of fast-food stores are surveyed twice. Between the surveys, a policy hits the gold stores (the treated group) but not the blue ones (the control group). Each small dot is one store; the big dots are the four group averages. Everything is simulated, so you set the truth: the policy’s real effect, and whether the two groups would have moved in parallel without it.

The dashed line. We never see what the treated stores would have done without the policy. Difference-in-differences fills the gap by assuming they would have changed by the same amount as the control stores. That assumed path is the dashed gold line, and the estimate is the green bracket between where the treated group ended up and where the dashed line says it would have been.

The dotted line. In a simulation you can peek at the truth: the pink dotted line is where the treated stores really would have gone. When the trends are parallel, dashed and dotted lines end in about the same place. Slide “Break parallel trends” and they split; the red gap is the bias, and no amount of data removes it.

The right side. The regression card shows the same numbers as a regression with an interaction term, with standard errors clustered by store. “Run 1,000 studies” draws 1,000 fresh samples and piles up their estimates, so you can see whether the method is right on average and how often its 95% intervals catch the truth.

Two differences

The question is causal: what did the policy do to the stores that got it? In the language of potential outcomes, each treated store has an outcome with the policy, y(1), which we see, and one without it, y(0), which we don’t. The target is the average treatment effect on the treated, ATT = E[y(1) − y(0) | treated], in the period after the policy.

Two easy comparisons each fail. Before vs after for the treated stores mixes the policy with everything else that changed over the year (a recession, a new menu, the season). Treated vs control after the policy mixes the policy with any old difference between the groups (bigger stores, a different state). Difference-in-differences subtracts one from the other, and both problems cancel, as long as they are the same size in both groups:

δ̂₁ = (ȳ_{T, after} − ȳ_{T, before}) − (ȳ_{C, after} − ȳ_{C, before})the change for the treated group minus the change for the control group

The assumption that makes this work is called parallel trends: without the policy, the average outcome of the treated group would have changed by the same amount as the control group’s.

E[y_after(0) − y_before(0) | treated] = E[y_after(0) − y_before(0) | control]

Notice what it does not require. The groups can start at different levels (in the simulation the treated stores start about 3 jobs lower), and the world can change for everyone (here every store loses about 2 jobs over the year). Only the changes must match. If the treated group would have gained an extra d anyway, the estimate is off by exactly d. That is the slider you’ve been moving.

The regression form

Put the two periods in one dataset with a row for each store in each period, and let d2 = 1 in the second period and dT = 1 for the treated group. Following Wooldridge (chapter 13), estimate

y = β₀ + δ₀·d2 + β₁·dT + δ₁·d2·dT + u

With two groups and two periods this regression is saturated: it has one coefficient for each of the four cells, so it reproduces the four averages exactly. β₀ is the control group before, β₀ + β₁ the treated group before, β₀ + δ₀ the control group after, and β₀ + δ₀ + β₁ + δ₁ the treated group after. Subtract and you find that the interaction coefficient δ̂₁ is the difference-in-differences. The regression adds two things the table can’t: standard errors, and room for control variables (a store’s size, a city’s unemployment rate) that soak up noise.

Worked example: New Jersey’s minimum wage, 1992

On April 1, 1992, New Jersey raised its minimum wage from $4.25 to $5.05 an hour. Pennsylvania next door did not. David Card and Alan Krueger surveyed about 400 fast-food restaurants (Burger King, KFC, Wendy’s and Roy Rogers) in both states in February–March 1992 and again in November–December. Their Table 3 reports average full-time-equivalent (FTE) employment per store:

	Before	After	Change
New Jersey (treated)	20.44	21.03	+0.59
Pennsylvania (control)	23.33	21.17	−2.16
Difference	−2.89	−0.14	+2.76 (SE 1.36)

Before vs after in New Jersey: +0.59 jobs per store. That ignores whatever pulled Pennsylvania down by 2.16.
New Jersey vs Pennsylvania after: −0.14. That ignores that New Jersey stores were already 2.89 jobs smaller.
Difference-in-differences: 0.59 − (−2.16) = 2.75, which the paper reports as 2.76 from unrounded numbers.

The estimate was positive, the opposite of what the simplest supply-and-demand model predicts, and it started a long debate. David Neumark and William Wascher re-examined it with payroll records and found employment fell; Card and Krueger replied with Bureau of Labor Statistics data that supported their original result (both papers appeared in the American Economic Review in 2000). The method itself became a staple, and in 2021 Card shared the Nobel memorial prize in economics, in part for this kind of “natural experiment”.

Worked example: reading the regression

A coffee chain tests a new scheduling app in 30 of its 80 stores for one quarter. Average weekly sales, in thousands of dollars: test stores 52.0 before and 55.5 after; the other 50 stores 50.0 before and 51.0 after. Estimating the regression on the 160 store-quarters gives

ŷ = 50.0 + 1.0·d2 + 2.0·dT + 2.5·d2·dT

β̂₀ = 50.0: the other stores before. β̂₁ = 2.0: test stores were already selling $2,000 more a week. Not the app’s doing.
δ̂₀ = 1.0: sales rose $1,000 a week everywhere (the season, prices).
δ̂₁ = 2.5: the app’s estimated effect, $2,500 a week per store, if the two kinds of stores would otherwise have grown alike.

If the chain picked its 30 busiest, fastest-growing stores for the test, the parallel-trends assumption is in trouble, and the honest answer is “less than $2,500, by an unknown amount”. That’s why a company that can choose should pick test stores at random (see Chapter 1).

Try this: press “Diverging trends”. The dashed and dotted lines split, and the red bracket is the bias: 2 jobs, larger than the true effect of 1.5. Now raise “Stores per group” to 200 and run 1,000 studies. The histogram gets narrower, but it stays centered on the wrong number. More data makes you more precise, not more right.

Try this: set the trend break to 0, switch the comparison to “Before vs after”, and watch the estimate miss by about −2: that’s the common change. “Treated vs control” misses by about −3: that’s the starting gap. Then switch back to DiD.

Common mistake: “The two groups have to look alike”

Difference-in-differences does not need the treated and control groups to have the same level of the outcome. New Jersey’s stores were smaller than Pennsylvania’s, and that is fine: the fixed difference cancels. What it needs is that the gap would have stayed the same without the policy. Two groups that look identical in levels but are on different paths will mislead you; two groups at different levels moving in step won’t. (Groups that differ a lot in levels do make parallel trends less believable, and it also matters whether the outcome is in levels or logs: trends can be parallel in one and not the other.)

What has to be true

Parallel trends

Without the policy, both groups’ averages would have changed by the same amount. Untestable after the policy starts, because one of the two paths is never seen.

No anticipation

The treated group doesn’t react before the policy starts. If restaurants cut hiring the moment a law is announced, the “before” period is already contaminated.

No spillovers

The policy doesn’t change the control group. If New Jersey workers commuted to Pennsylvania jobs, both groups would move.

Stable groups

The same kind of units are in each group in both periods. If the policy closed the weakest stores, the treated group’s average rises with no one better off.

Clustered standard errors

Outcomes of the same store in two periods are related: a big store is big both times. The usual OLS standard errors assume every row is independent, so they get the uncertainty wrong. The fix is to cluster: let errors be correlated in any way within a store (or a state), and treat only different clusters as independent. In the simulation, compare the regression card with the classic formula: with these settings, clustering gives an SE of about 0.44 for δ̂₁ while the classic formula says about 1.19. Here the classic formula is too pessimistic, because each store’s own level cancels in its before-after change.

With many periods, the danger runs the other way. Marianne Bertrand, Esther Duflo and Sendhil Mullainathan (2004) took 21 years of U.S. wage data from the Current Population Survey, made up fake state “laws” at random dates, and ran standard difference-in-differences regressions. With ordinary standard errors, up to 45% of these placebo laws came out “significant” at the 5% level, because outcomes within a state are correlated from year to year. Allowing for correlation within each state over time (clustering by state) fixed most of it when there were enough states. The rule of thumb: cluster at the level where the policy is assigned, and worry when there are only a handful of clusters (then use a method built for few clusters, such as the wild cluster bootstrap).

Event studies: leads and lags

With more than two periods, don’t squeeze everything into one number. Estimate a separate treated-minus-control difference for every period, relative to the last period before the policy:

y_it = α_i + λ_t + Σ_{k ≠ −1} β_k·dT_i·1[t − E = k] + u_itα_i: a fixed effect for each unit; λ_t: one for each period; E: the period the policy starts; k: periods since it started

The coefficients after the policy (k ≥ 0, the lags) trace out how the effect builds or fades. The coefficients before it (k ≤ −2, the leads) are a check: if the trends really were parallel before the policy, the leads should hover around zero. A clear slope in the leads is a warning that the groups were already drifting apart.

Shape of the effect

Effect size Pre-trend of the treated group

Serial correlation of errors Units per group

The joint test asks whether all four leads are zero at once, using the clustered covariance matrix (a Wald test, reported as an F statistic). It is the formal version of eyeballing the left side of the plot.

Try this: choose “Anticipation”. Units start responding two periods early, so period −1 is already partly treated. Every later coefficient is measured against a contaminated baseline and comes out too small, and the lead at −2 dips below zero. Then set the pre-trend to +0.2 with a jump: the leads climb in a straight line toward zero and the lags keep climbing, so a naive reading would credit the policy with the drift.

Common mistake: “The pre-trend test passed, so parallel trends holds”

A test that fails to reject is not proof. With few units or noisy data, the leads have wide intervals and a real pre-trend can hide inside them; turn “Units per group” down to 10 with a pre-trend of 0.1 and see. Jonathan Roth (2022) showed that studies that pass a pre-test can still be badly biased, and that selecting studies on passing the pre-test can make the bias worse. Modern practice reports how big a violation of parallel trends would have to be to overturn the conclusion (Rambachan and Roth, 2023) instead of treating the pre-test as a yes-or-no gate.

The leads have a famous real-world cousin: the Ashenfelter dip. Orley Ashenfelter (1978) noticed that people who enrolled in government job-training programs had unusually low earnings in the year just before enrolling, often because they had just lost a job. Their earnings would have bounced back anyway, so a before-after comparison flatters the program. An event study shows the dip as negative leads.

Staggered adoption: when two-way fixed effects goes wrong

Real policies rarely start everywhere on the same day. States adopt laws in different years; a company rolls out a new system store by store. For decades the standard tool was the two-way fixed effects (TWFE) regression

y_it = α_i + λ_t + β·D_it + u_it, D_it = 1 once unit i is treated

and β̂ was read as “the effect”. Around 2018–2021, several papers showed what β̂ actually is when the effect differs across groups or changes over time. Andrew Goodman-Bacon (2021) proved that it is a weighted average of every possible two-group, two-period DiD in the data, including comparisons that use already-treated units as the control group. If the effect grows over time, those already-treated “controls” are still rising because of the policy, so subtracting their change pulls the estimate down, sometimes below zero. Clément de Chaisemartin and Xavier D’Haultfœuille (2020) showed the same thing another way: β̂ is a weighted sum of the effects in each treated group and period, and some of the weights can be negative.

The true effects

Effect size Include a never-treated group

Early group adopts in Late group adopts in

The dots in the top-right panel are the 2×2 comparisons. Green ones compare a treated group with the never-treated group: always clean. Violet compares the early group with the late group before the late group is treated: also clean. Red compares the late group with the early group after the early group is already treated, and is the “forbidden comparison”. When effects are the same everywhere and constant over time, every dot sits on the truth and TWFE is fine. When effects grow, the red dot falls, and with enough weight it drags TWFE with it.

Try this: pick “Grows each period”, switch off the never-treated group, and set the early group to adopt in period 2 and the late group in period 9. TWFE comes out negative although every true effect is positive. Now pick “Same everywhere”: all dots agree. Pick “Bigger for early group”: TWFE lands between the two groups’ effects, but at a weighted average that answers no particular question.

The fix: clean comparisons only. Brantly Callaway and Pedro Sant’Anna (2021) estimate a separate effect for each adoption group g in each period t, comparing it only with units that are not yet or never treated, each time against the period just before g adopted:

ATT(g, t) = (ȳ_{g, t} − ȳ_{g, g−1}) − (ȳ_{C, t} − ȳ_{C, g−1})

Then they average these building blocks with weights the researcher chooses openly (by group size, by time since adoption, and so on). Liyang Sun and Sarah Abraham (2021) fixed event studies in the same spirit, and Kirill Borusyak, Xavier Jaravel and Jann Spiess (2024) proposed estimating the untreated outcomes from untreated observations only and then imputing. The common lesson: decide which comparisons you trust, and make sure the estimator uses only those.

A short history

A water company moves upstream. John Snow’s 1855 study of cholera in London is often described as an early ancestor of the method. One water company moved its intake to a cleaner stretch of the Thames while another kept drawing sewage-laden water, and Snow compared death rates among their customers before and after.

Training programs. Orley Ashenfelter (1978) and Ashenfelter and David Card (1985) used before-and-after comparisons of trainees and non-trainees to evaluate U.S. job-training programs, and found the dip in pre-program earnings that now carries Ashenfelter’s name.

Natural experiments. Card (1990) compared Miami’s labor market with similar cities after the 1980 Mariel boatlift brought about 125,000 Cuban immigrants to the United States, many of whom settled in Miami. Card and Krueger (1994) studied the New Jersey minimum wage. These papers made “find a good comparison group” a central habit of empirical economics. Card received half of the 2021 Nobel memorial prize in economics for his empirical contributions to labor economics; Joshua Angrist and Guido Imbens shared the other half for methods of causal analysis.

The staggered-timing revolution. Between 2018 and 2021, papers by Goodman-Bacon, de Chaisemartin and D’Haultfœuille, Callaway and Sant’Anna, Sun and Abraham, and Borusyak, Jaravel and Spiess changed how economists estimate difference-in-differences with many periods. Many older TWFE studies have been re-run with the new estimators; some conclusions held up and some did not.

Where it shows up

Business. A chain pilots a new price, layout or app in some regions first. Comparing pilot regions’ change with the rest’s change is DiD; staggered rollouts across regions are exactly the case where TWFE can mislead. Henry Ford’s 1914 wage jump (Henry Ford’s Five-Dollar Day) is the kind of one-firm change you would study by comparing Ford’s worker turnover before and after with other employers’.
Policy. Minimum wages, Medicaid expansions that states adopted in different years, smoking bans, and school reforms are routinely studied with DiD and event studies.
Sports. When one league changes a rule and a similar league doesn’t, the change in the first league minus the change in the second isolates the rule’s effect on scoring or injuries.
Science and health. Environmental economists compare counties near a newly regulated plant with similar counties farther away, before and after.
Daily life. If you start a new study routine and your grade rises, ask how much your classmates’ grades rose over the same weeks. That’s a one-person difference-in-differences.

Practice

A city bans plastic bags. Average weekly bag purchases per household: in the city, 9.0 before and 4.0 after; in a similar city with no ban, 8.0 before and 7.0 after. Estimate the effect of the ban by difference-in-differences.
Show the answer
Change in the treated city: 4.0 − 9.0 = −5.0. Change in the control city: 7.0 − 8.0 = −1.0. DiD = −5.0 − (−1.0) = −4.0 bags per household per week. The before-after comparison alone (−5.0) would credit the ban with the −1.0 that happened everywhere.
In the regression y = β₀ + δ₀·d2 + β₁·dT + δ₁·d2·dT + u, the estimates are β̂₀ = 12, δ̂₀ = 3, β̂₁ = −2, δ̂₁ = 1.5. What are the four cell averages?
Show the answer
Control before: 12. Control after: 12 + 3 = 15. Treated before: 12 − 2 = 10. Treated after: 12 + 3 − 2 + 1.5 = 14.5. Check: (14.5 − 10) − (15 − 12) = 4.5 − 3 = 1.5 = δ̂₁.
Using the Card and Krueger table above, by how much would “New Jersey after vs Pennsylvania after” have missed the DiD estimate, and why?
Show the answer
The after-only comparison is −0.14; the DiD is 2.76 (2.75 from the rounded table). It misses by about −2.9, which is the difference that already existed before the law (−2.89): New Jersey’s stores were smaller to begin with.
Before a training program, the earnings of future trainees grew by $500 a year faster than non-trainees’ for three years. The DiD estimate over one year is $1,200. If the faster growth would have continued, what is a better estimate, and what assumption did you make?
Show the answer
Subtract the extra trend: $1,200 − $500 = $700. The assumption is that the pre-period gap in growth would have continued unchanged, which is itself untestable. (If the faster growth was a bounce back from an Ashenfelter dip, the right correction could be different.) Reporting a range of answers under different trend assumptions is the honest approach.
A policy is assigned by state, and you have 40,000 workers from 8 states over 10 years. Why is clustering by worker not enough, and what goes wrong with only 8 clusters?
Show the answer
Workers in the same state share state-level shocks that persist over years, and the policy varies only at the state level, so the information is really about 8 states, not 40,000 workers. Clustering by worker treats different workers in the same state as independent and makes the standard errors far too small. Clustering by state is right in principle, but the cluster-robust formula relies on many clusters; with 8 it is unreliable, so use a method designed for few clusters, such as the wild cluster bootstrap, and be modest about precision.
Two groups, ten periods. Group E adopts in period 3, group L in period 8, no never-treated group. The effect is 1 in a unit’s first treated period and grows by 1 each period after. Compute the “forbidden” 2×2 comparison: L vs E, comparing periods 8–10 with periods 3–7 (ignore everything but the effects).
Show the answer
L’s effects in periods 8, 9, 10 are 1, 2, 3, average 2; before period 8 it had none, so L’s change is +2. E’s effects in periods 3–7 are 1 to 5 (average 3) and in periods 8–10 are 6, 7, 8 (average 7), so E’s change is +4. The comparison is 2 − 4 = −2: negative although every effect is positive. In the simulation with these settings, this comparison gets 60% of the weight, and TWFE comes out at 0.00 while the true average effect is 3.82.
In an event study with the policy at k = 0 and reference period k = −1, the leads are −0.6, −0.4, −0.2 at k = −4, −3, −2 and the lags are 0.2, 0.4, 0.6 at k = 0, 1, 2. What would you conclude?
Show the answer
The coefficients form one straight line rising by 0.2 per period, with no break at the policy. That is what a pre-existing difference in trends looks like, not a policy effect. Extrapolating the pre-trend predicts exactly the lags you see, so the evidence for any effect is weak.

How the interactive works

The 2×2. Store i has a level a_i drawn from a normal distribution (mean 23 jobs for control stores, 20 for treated, spread 3). Before: y = a_i + noise. After, without the policy: y(0) = a_i − 2 + (the trend break, for treated stores only) + noise; with it, y(1) = y(0) + the true effect. The regression uses the shared course library with standard errors clustered by store (CR1, the same small-sample factor Stata uses). Moving a slider keeps the same random draws, so you see the change, not new noise.

The event study. Each unit has its own level, every period has a common shock, and each unit’s errors follow an AR(1) process with the serial correlation you choose. The treated group starts at period 0. The regression removes unit and period means (the two-way “within” transform, exact for a balanced panel) and clusters by unit. The dashed truth line is what each coefficient estimates: the effect in that period minus the effect in period −1, plus any pre-trend relative to period −1.

Staggered adoption. Ten periods, groups of equal size, no noise, so every number on the screen is exact. TWFE is computed by the within transform; the Goodman-Bacon weights use the formulas in his Theorem 1; the cell weights are each treated cell’s residualized treatment, divided by their total.

How it was checked (Node.js, before the page was built):

The interaction coefficient equals the DiD of the four means (2.0037 both ways), and the fast Monte Carlo formula gives the same clustered SE as the full regression (0.4387).
4,000 simulated studies with parallel trends and a true effect of 2: mean estimate 2.011, clustered 95% intervals covered the truth 95.2% of the time (the classic intervals covered 100%: too wide). With the trend broken by 1.5: mean 3.511, coverage 11.6%.
Event study: the within-transform coefficients match a regression with all unit and period dummies to 10⁻¹³. In 400 samples without a pre-trend, the joint pre-test rejected 6.5% of the time at the 5% level, and the intervals covered the truth 94.6% of the time; with a pre-trend of 0.25, it rejected 40% of the time with 25 units per group.
Staggered adoption: the Goodman-Bacon weights add to 1.0000 and the weighted 2×2 estimates reproduce TWFE exactly, also with three adoption groups, unequal group sizes and noise, where TWFE matched a unit-level dummy regression to six decimals (0.874893). The cell weights reproduce TWFE too. With early adoption in period 3, late in 8, growing effects and no never-treated group, TWFE is 0.00 against a true ATT of 3.82; adopting in periods 2 and 9 gives −0.67 against 4.36. Callaway–Sant’Anna recovers the truth on every cell it can estimate.

Check yourself

What does the parallel-trends assumption require?
Show the answer
Levels can differ and everyone can trend; only the untreated changes must match. That is what lets the control group’s change stand in for the treated group’s missing one.
In the regression y = β₀ + δ₀·d2 + β₁·dT + δ₁·d2·dT + u, which coefficient is the difference-in-differences estimate?
Show the answer
The interaction δ₁ is the extra change for the treated group in the second period, which is exactly the DiD of the four cell averages.
You double the number of stores in a study where parallel trends fails by 2 jobs. What happens to the bias?
Show the answer
Bias comes from the design, not from sampling noise. More data narrows the histogram, so you become more confident in the wrong number.
In an event study, what are the coefficients before the policy (the leads) mainly used for?
Show the answer
Only the reference period is set to zero. If trends were parallel and nobody anticipated the policy, the other leads should be near zero; a pattern in them is a warning.
Why can two-way fixed effects be badly biased when a policy is adopted at different times and its effect grows over time?
Show the answer
Goodman-Bacon’s decomposition shows that TWFE averages all 2×2 comparisons, including “late vs already-treated early”. When the early group’s effect keeps growing, that comparison is biased downward and can even flip the overall sign.

Sources

J. M. Wooldridge, Introductory Econometrics: A Modern Approach, 7th ed. (Cengage, 2020), chapter 13.2 (policy analysis with pooled cross sections, the DiD regression and its notation) and 14 (panel methods).
J. D. Angrist and J.-S. Pischke, Mostly Harmless Econometrics (Princeton, 2009), chapter 5.2 (differences-in-differences, the Card and Krueger table); and Mastering ’Metrics (Princeton, 2015), chapter 5.
D. Card and A. B. Krueger, Minimum wages and employment: a case study of the fast-food industry in New Jersey and Pennsylvania, American Economic Review 84(4), 772–793 (1994). D. Neumark and W. Wascher, and D. Card and A. B. Krueger, comment and reply, American Economic Review 90(5) (2000).
M. Bertrand, E. Duflo and S. Mullainathan, How much should we trust differences-in-differences estimates?, Quarterly Journal of Economics 119(1), 249–275 (2004).
O. Ashenfelter, Estimating the effect of training programs on earnings, Review of Economics and Statistics 60(1), 47–57 (1978).
A. Goodman-Bacon, Difference-in-differences with variation in treatment timing, Journal of Econometrics 225(2), 254–277 (2021).
B. Callaway and P. H. C. Sant’Anna, Difference-in-differences with multiple time periods, Journal of Econometrics 225(2), 200–230 (2021).
C. de Chaisemartin and X. D’Haultfœuille, Two-way fixed effects estimators with heterogeneous treatment effects, American Economic Review 110(9), 2964–2996 (2020).
L. Sun and S. Abraham, Estimating dynamic treatment effects in event studies with heterogeneous treatment effects, Journal of Econometrics 225(2), 175–199 (2021). K. Borusyak, X. Jaravel and J. Spiess, Revisiting event-study designs: robust and efficient estimation, Review of Economic Studies (2024).
J. Roth, Pretest with caution: event-study estimates after testing for parallel trends, American Economic Review: Insights 4(3), 305–322 (2022). A. Rambachan and J. Roth, A more credible approach to parallel trends, Review of Economic Studies 90(5), 2555–2591 (2023).
Wikipedia (secondary): Difference in differences, David Card.

I finished this chapterSaved on this device only.