Econometrics · Chapter 15

Time Series: Trends, Autocorrelation and Forecasting

Sales this month, unemployment this quarter, a price today: economic data come one period after another, and each value remembers the last. That memory is what lets us forecast, and it is also what fools careless regressions. Here you’ll turn one dial from white noise to a random walk and watch the memory appear in the autocorrelations, regress two series that have nothing to do with each other and get “significant” results most of the time, draw forecast fans that widen with the horizon, and find two wandering series held together by an invisible rubber band.

Mode

Live numbers

Missions

    What you’re seeing

    The stream. Each new value is yt = φyt−1 + et + θet−1, where et is a fresh random shock. With φ = 0 and θ = 0 every value is new noise. Raise φ and each value keeps a share of the last one, so the line drifts in slow waves. At φ = 1 nothing is ever forgotten: that’s a random walk, and it can wander anywhere.

    The bars. The autocorrelation at lag k is the correlation between the series and itself k periods earlier. The dashed bands at ±1.96/√T show how big a correlation pure chance would produce in a window this long. The partial autocorrelation at lag k is what lag k adds after accounting for lags 1 to k − 1. The gold dots are the true values for the model you set.

    Two random walks. Switch the mode and the screen shows two series built completely separately. Regress one on the other and the slope often looks “significant”, because both wander and any two wanderers seem to move together for a while. The histogram collects the t statistics of many such regressions; a valid 5% test would put only 5% of them in the red tails.

    Stationarity and memory

    A time series is (covariance) stationary if its mean and variance don’t change over time and the covariance between yt and yt−k depends only on the gap k, not on t. Stationary series keep coming back to the same average, and their memory fades: values far apart are nearly independent (Wooldridge calls this weak dependence). Most of the tools from earlier chapters work for stationary, weakly dependent series. Trouble starts when a series has a trend or never forgets.

    Trends and seasons

    Many series grow: GDP, prices, population. A deterministic trend model writes yt = α + βt + et; including t (or working with growth rates) keeps the trend from masquerading as a relationship with any other trending variable. Many series also repeat within the year: retail sales jump every December, ice cream sales every July. Seasonal dummy variables, one for each month or quarter, absorb that pattern, which is also what government agencies remove when they publish “seasonally adjusted” numbers.

    AR, MA and ARMA

    AR(p): autoregressive

    yt = c + φ₁yt−1 + … + φpyt−p + et. Today depends on the last p values. The ACF fades out gradually; the PACF cuts off after lag p.

    MA(q): moving average

    yt = μ + et + θ₁et−1 + … + θqet−q. A shock echoes for exactly q periods. The ACF cuts off after lag q; the PACF fades.

    ARMA(p, q)

    Both parts together: a compact way to describe a long memory with few numbers. George Box and Gwilym Jenkins (1970) turned choosing, fitting and checking these models into a method.

    AR(1): Var(y) = σ²/(1 − φ²),   Corr(yt, yt−k) = φkstationary only when |φ| < 1; the closer φ is to 1, the bigger the swings and the longer the memory
    Worked example: how long a shock lasts

    An AR(1) with φ = 0.8 and shock variance σ² = 1:

    1. Variance: 1/(1 − 0.64) = 2.78, so the series swings 2.78 times as much as one shock.
    2. Autocorrelations: 0.8, 0.64, 0.51, 0.41, … at lags 1, 2, 3, 4.
    3. Half-life of a shock: solve 0.8k = 0.5, so k = ln 0.5 / ln 0.8 ≈ 3.1 periods.

    With φ = 0.95 the half-life is 13.5 periods, and with φ = 0.99 it is 69. At φ = 1 it is infinite: the shock never fades.

    Try this: set φ = 0 and θ = 0.7. The autocorrelation is about 0.47 at lag 1 and then drops to nothing: an MA(1) remembers one shock for exactly one period. Now set θ = 0 and φ = 0.7: the ACF fades gradually (0.7, 0.49, 0.34, …) while the PACF has one bar at lag 1 and nothing after. Reading these two pictures is how Box and Jenkins picked a model.

    Unit roots and the Dickey–Fuller test

    At φ = 1, the AR(1) becomes a random walk: yt = yt−1 + et. Every shock is permanent, the variance grows with time (Var(yt) = tσ² from a fixed start), and the series has no average to return to. Such a series is said to have a unit root, or to be integrated of order one, I(1); its first difference Δyt = et is stationary, I(0). Many asset prices and some macroeconomic series behave much like this.

    To test for a unit root, subtract yt−1 from both sides and regress:

    Δyt = α + θyt−1 + et,   H₀: θ = 0 (unit root) vs H₁: θ < 0the Dickey–Fuller test; reject at 5% if the t statistic is below −2.86 (with a constant, large samples)

    The catch, found by David Dickey and Wayne Fuller (1979): under the null, the t statistic does not follow a t or normal distribution, because yt−1 is itself wandering. Its critical values are further left: −3.43 at 1%, −2.86 at 5%, −2.57 at 10% (compared with −2.33, −1.65 and −1.28 for a one-sided normal test). The augmented Dickey–Fuller test adds lagged differences Δyt−j to soak up extra short-run dynamics; with a time trend in the regression, the critical values shift further still.

    Worked example: can we reject a unit root?

    For 200 quarters of a series, the Dickey–Fuller regression gives Δŷt = 0.21 − 0.034yt−1, with a standard error of 0.016 on the slope.

    1. t = −0.034/0.016 = −2.13.
    2. A normal table would call that significant at 5% (one-sided: below −1.65). The correct comparison is with −2.86.
    3. −2.13 > −2.86, so we can’t reject a unit root. The estimated φ = 1 − 0.034 = 0.966 is close to 1, and the test can’t tell it from 1 in 200 quarters.

    Unit-root tests have little power against φ just below 1: in our simulations with 200 periods, the test rejected only 35% of the time when φ = 0.95, 87% when φ = 0.9, and every time when φ = 0.8.

    Spurious regression

    In 1926 the statistician G. Udny Yule asked why we sometimes get nonsense correlations between time series. His example: in England and Wales from 1866 to 1911, the share of marriages performed in the Church of England and the mortality rate were correlated at about 0.95. Neither caused the other; both happened to decline over the same decades. Clive Granger and Paul Newbold (1974) showed by simulation that regressing one independent random walk on another produces “significant” slopes most of the time, and Peter Phillips (1986) proved why: with two I(1) series, the t statistic grows with the sample size instead of settling down, so more data makes it worse.

    Common mistake: “The trends match, so the variables must be related”

    Two series that both wander, or both trend, will line up for long stretches by coincidence. A high R² and a large t statistic from regressing one on the other mean little. In our simulations of independent random walks, the slope was “significant” at 5% in 76% of regressions with 100 periods, 82% with 200 and 89% with 500. Granger and Newbold’s warning sign: an R² larger than the Durbin–Watson statistic. The fixes: difference the series (4.6% false alarms in our tests, as a 5% test should give), include trends, or test whether the series are cointegrated before trusting a levels regression.

    Serial correlation and Newey–West standard errors

    Even with stationary data, regression errors in time series are usually correlated over time. OLS is still unbiased if the regressors are strictly exogenous, but the usual standard errors assume independent errors and are too small when errors are positively correlated. Whitney Newey and Kenneth West (1987) proposed standard errors that allow both heteroskedasticity and autocorrelation up to some lag g (HAC standard errors):

    Var(β̂) = (X′X)−1 [ Σt ût² xtxt′ + Σh=1g (1 − h/(g+1)) Σt ûtût−h(xtxt−h′ + xt−hxt′) ] (X′X)−1

    The declining weights keep the estimate positive. Wooldridge suggests g = the integer part of 4(n/100)2/9, which is 4 for 100 to 200 observations. In the simulation, regressing one independent AR(1) series with φ = 0.9 on another (200 periods), the OLS test rejects 50% of the time; Newey–West cuts that to 27%. Better, though not perfect in a sample this size, and no help at all for two random walks (67%), where the regression itself is meaningless.

    Forecasting

    For a stationary AR model, the best forecast (in mean squared error) of a future value is its conditional expectation given today’s information. For an AR(1), ŷT+h|T = μ + φh(yT − μ): the forecast starts from today and decays toward the long-run mean μ = c/(1 − φ). The uncertainty grows with the horizon, because more unknown shocks pile up:

    Var(forecast error at h) = σ² (1 + ψ₁² + … + ψh−1²)the ψ’s say how much a shock still matters j periods later; for an AR(1), ψj = φj; for a random walk, all ψ = 1, so the variance is hσ²

    What generated the data

    Model you fit

    Worked example: forecasting unemployment

    Suppose a monthly unemployment rate follows ut = 0.25 + 0.95ut−1 + et with σ = 0.5 percentage points, and this month it is 7%.

    1. Long-run mean: 0.25/(1 − 0.95) = 5%.
    2. Forecasts: next month 0.25 + 0.95 × 7 = 6.90%; in general 5 + 2 × 0.95h, so in a year 5 + 2 × 0.540 = 6.08%.
    3. Standard errors: 0.50 one month ahead, 0.5 × √(1 + 0.95²) = 0.69 two months ahead, 1.35 twelve months ahead. They level off at 0.5/√(1 − 0.95²) = 1.60, the spread of the series itself.
    4. 95% interval for a year from now: 6.08 ± 1.96 × 1.35, about 3.4% to 8.7%.

    For a random walk, the twelve-month standard error would be 0.5 × √12 = 1.73 and would keep growing, without limit.

    Try this: choose “Unemployment” and fit AR(1), then run 500 forecast tests. One month ahead, the band catches about 95%; twelve ahead, fewer (86% in our tests). The formula treats the estimated φ as if it were the true one, and estimates of a φ near 1 are biased down in samples of 100, so long-horizon bands come out too narrow. Real forecasters widen them, or simulate the uncertainty in the estimates.

    Cointegration and error correction

    Two series can each wander like random walks and still be tied together. The prices of the same commodity in two cities, short- and long-term interest rates, or a company’s two classes of shares each drift, but if they drift too far apart, buyers and sellers push them back. Robert Engle and Clive Granger (1987) called this cointegration: xt and yt are I(1), but some combination yt − βxt is stationary. The regression of y on x is then not spurious: β̂ converges to the true β very fast (“super-consistency”).

    The Engle–Granger test has two steps. First, regress yt on xt by OLS and keep the residuals ût, the estimated spread. Second, run a Dickey–Fuller test on ût. If the spread has a unit root, the two series are not cointegrated. Because β̂ was chosen to make the spread look as stationary as possible, the critical values are stricter than Dickey–Fuller’s: −3.90 at 1%, −3.34 at 5% and −3.04 at 10% for two variables with a constant.

    The error-correction model then describes how the series move back together. With ût−1 the gap from the long-run relation last period:

    Δyt = α₀ + γΔxt + δût−1 + et,   δ < 0

    γ is the short-run effect of a change in x; δ is the speed of adjustment. If y is above its long-run level, δ < 0 pulls it down next period. In the simulation, δ estimates ρ − 1.

    Worked example: closing a gap

    An error-correction model gives Δŷt = 0.02 + 0.9Δxt − 0.25ût−1, and y is currently 2 units above its long-run relation with x.

    1. If x doesn’t change, y is expected to fall by 0.25 × 2 = 0.5 next period (plus the small drift of 0.02).
    2. Each period, a quarter of the remaining gap closes, so the gap shrinks by a factor of 0.75: half of it is gone in ln 0.5/ln 0.75 ≈ 2.4 periods.
    3. If x rises by 1, y rises by 0.9 at once, and the error-correction term handles the rest over time.
    Try this: set ρ = 1. The spread is now a random walk itself, the two series are not cointegrated, and the regression of y on x is spurious. Run 500 tests: the Engle–Granger test rejects about 5% of the time, as it should. Then set ρ = 0.8 and run them again (about 100% rejections with 200 periods). At ρ = 0.9 the test rejects only about two-thirds of the time: like all unit-root tests, it has trouble with spreads that revert slowly.
    Markets and money. “Pairs trading” bets that a cointegrated spread will close. It can fail exactly when it matters most: relations estimated in calm times break in a crisis, and a spread can widen for longer than a trader can afford to wait. The fund in Long-Term Capital Management: When Geniuses Borrow Too Much made bets that similar prices would move back together, borrowed heavily, and nearly brought down Wall Street when they moved apart instead. Nothing on this page is financial or investment advice.

    A short history

    The first models. Yule fitted an autoregressive model to sunspot numbers in 1927, and in the same year the Russian economist Eugen Slutsky showed that summing random shocks (a moving average) can produce cycles that look like business cycles.

    Box and Jenkins. Their 1970 book Time Series Analysis: Forecasting and Control made ARMA models a practical forecasting method: identify a model from the ACF and PACF, estimate it, check the residuals, repeat.

    Unit roots. Dickey and Fuller’s test (1979), Granger and Newbold’s warning about spurious regressions (1974) and Phillips’s theory (1986) taught economists to check whether their series wander before regressing them on each other. Charles Nelson and Charles Plosser (1982) argued that many U.S. macroeconomic series look like they have unit roots, which started a long debate.

    Cointegration. Engle and Granger’s 1987 paper showed how to model wandering series that move together. In 2003, Granger received half of the Nobel memorial prize in economics for methods of analyzing time series with common trends (cointegration), and Engle the other half for methods of analyzing time series with time-varying volatility (ARCH).

    Where it shows up

    • Business. Demand forecasting for inventory, staffing and budgets starts with trend, seasonality and AR terms. A retailer that forgets December’s seasonal jump will either run out of stock or overstaff January.
    • Policy. Central banks and the Bureau of Labor Statistics publish seasonally adjusted series; forecasters model inflation and unemployment with autoregressions as a baseline that more elaborate models must beat.
    • Finance. Whether stock prices are close to random walks is one of the oldest questions in finance; interest rates of different maturities tend to be cointegrated.
    • Science. Climate scientists face spurious-regression issues when relating trending series, and use the same tools: differencing, trend terms and cointegration tests.
    • Daily life. “Stock prices and ice cream sales both rose this decade” is a spurious regression waiting to happen. Before believing that two trending things are linked, ask whether they would line up just by both trending.

    Practice

    1. An AR(1) has φ = 0.6 and shock variance 4. What are its variance and its autocorrelation at lag 3?

      Show the answer

      Variance = 4/(1 − 0.36) = 6.25. Lag-3 autocorrelation = 0.6³ = 0.216.

    2. A sample ACF has spikes at lags 1 and 2 and nothing after; the PACF fades out slowly. Which model fits?

      Show the answer

      An MA(2): the ACF of an MA(q) cuts off after lag q while its PACF tails off.

    3. A Dickey–Fuller regression with a constant gives t = −3.1 for the coefficient on yt−1. Do you reject a unit root at 5%? At 1%?

      Show the answer

      At 5%: yes, −3.1 < −2.86. At 1%: no, −3.1 > −3.43. The evidence against a unit root is moderate.

    4. Regressing a country’s GDP on the number of televisions sold, both in levels over 40 years, gives R² = 0.93 and a Durbin–Watson statistic of 0.21. What do you suspect, and what would you do?

      Show the answer

      Spurious regression: both series trend, the residuals are highly autocorrelated (DW far below 2) and R² > DW, Granger and Newbold’s warning sign. Test both for unit roots; if they are I(1), either difference them (regress growth on growth) or test for cointegration before interpreting a levels regression.

    5. A random walk has shock standard deviation 2. What is the standard error of the forecast 9 periods ahead? What if it were an AR(1) with φ = 0.5 instead?

      Show the answer

      Random walk: 2 × √9 = 6. AR(1) with φ = 0.5: 2 × √(1 + 0.25 + 0.0625 + …) for 9 terms ≈ 2 × √(1.3333) ≈ 2.31, because the forecast variance levels off at σ²/(1 − φ²) = 4/0.75.

    6. With 150 observations, how many lags does Wooldridge’s rule of thumb suggest for Newey–West standard errors?

      Show the answer

      4 × (150/100)2/9 = 4 × 1.094 = 4.38, so g = 4.

    7. Engle–Granger step 2 gives t = −3.0 on the lagged residual (two variables, constant). An analyst compares it with −2.86 and declares the series cointegrated. What went wrong?

      Show the answer

      The residuals come from a regression chosen to make them look stationary, so the Dickey–Fuller critical values are too lenient. The Engle–Granger 5% critical value is −3.34; −3.0 does not reject “no cointegration” at 5% (only at about 10%, where the critical value is −3.04, it is borderline and still doesn’t reject).

    How the interactive works

    The stream. Shocks are standard normal; the series follows yt = φyt−1 + et + θet−1, 24 new values a second. The ACF uses the course library over the last T values; the PACF comes from the ACF by the Durbin–Levinson recursion; theoretical values use the ARMA(1,1) formulas. The Dickey–Fuller test is the regression with a constant and no extra lags, with large-sample critical values.

    Spurious regressions. Two series drawn with different seeds, each AR(φ) (a random walk at φ = 1) after a burn-in. The slope’s t statistic uses OLS or Newey–West standard errors (Bartlett weights, Wooldridge’s lag rule).

    Forecasts and cointegration. AR(p) models are fitted by OLS with a constant; intervals use the estimated coefficients as if they were true. In the cointegration lab, x is a random walk and y = 2 + βx + u with ut = ρut−1 + noise; the Engle–Granger test is a Dickey–Fuller regression on the residuals without a constant.

    How it was checked (Node.js and NumPy, before the page was built):

    • Theory: the PACF of an AR(1) with φ = 0.7 is 0.700, 0, 0, …; 200,000 draws of an ARMA(0.6, 0.4) gave sample autocorrelations 0.756, 0.453, 0.271 against theory 0.756, 0.454, 0.272, and variance 2.560 against 2.563.
    • Dickey–Fuller: 4,000 random walks of 500 periods gave 1%, 5% and 10% quantiles of −3.51, −2.85 and −2.54 (tables: −3.43, −2.86, −2.57); with 200 periods the test rejected 5.1% at 5%. Engle–Granger: 3,000 pairs gave −3.92, −3.36, −3.07 (tables: −3.90, −3.34, −3.04).
    • Spurious regression of independent random walks: 76.3% / 82.0% / 89.3% “significant” at T = 100 / 200 / 500; differenced, 4.6%. The Newey–West standard errors matched a NumPy implementation to ten digits, and equal HC0 with zero lags.
    • Forecast intervals: with the true model order and 100 periods of history, 95% bands covered 94.8% / 92.0% / 91.1% (AR(2)) and 94.6% / 91.1% / 86.1% (AR(1), φ = 0.95) at horizons 1 / 4 / 12; for a random walk, 95.0% / 95.3% / 95.9%.
    • Error correction: with ρ = 0.8 and 200 periods, the mean δ̂ was −0.226 (truth −0.20) and γ̂ 1.502 (truth 1.5).

    Check yourself

    1. In an AR(1) with φ = 0.9, what is the autocorrelation at lag 2?

      Show the answer

      φk = 0.9² = 0.81.

    2. Why can’t you use the usual −1.65 critical value for a Dickey–Fuller test?

      Show the answer

      With a unit root, the regressor yt−1 wanders, and the t statistic has the Dickey–Fuller distribution, with a 5% critical value of about −2.86 (with a constant).

    3. What happens to a spurious regression of two independent random walks as the sample gets longer?

      Show the answer

      The t statistic grows with T (Phillips, 1986): in our simulations, 76% false alarms at 100 periods and 89% at 500.

    4. For a random walk with shock standard deviation σ, the standard error of the h-step forecast is…

      Show the answer

      h independent shocks add up, so the variance is hσ² and the standard error is σ√h.

    5. Two I(1) series are cointegrated. What does that mean?

      Show the answer

      Cointegration means yt − βxt is stationary: the series share a common stochastic trend, and the error-correction model describes how gaps close.

    Sources

    • J. M. Wooldridge, Introductory Econometrics: A Modern Approach, 7th ed. (Cengage, 2020), chapters 10 (basic time series regression, trends and seasonality), 11 (stationarity, weak dependence, random walks), 12 (serial correlation and Newey–West standard errors) and 18 (unit roots, spurious regression, cointegration, error correction and forecasting).
    • J. H. Stock and M. W. Watson, Introduction to Econometrics, 4th ed. (Pearson, 2019), chapters 15 and 16 (forecasting, autoregressions, nonstationarity and cointegration).
    • J. D. Hamilton, Time Series Analysis (Princeton, 1994), chapters 3 (ARMA processes), 4 (forecasting), 17 (unit roots) and 19 (cointegration).
    • Penn State Eberly College of Science, STAT 510: Applied Time Series Analysis (ACF and PACF, ARIMA models, forecasting).
    • G. U. Yule, Why do we sometimes get nonsense-correlations between time-series?, Journal of the Royal Statistical Society 89(1), 1–63 (1926).
    • C. W. J. Granger and P. Newbold, Spurious regressions in econometrics, Journal of Econometrics 2(2), 111–120 (1974). P. C. B. Phillips, Understanding spurious regressions in econometrics, Journal of Econometrics 33(3), 311–340 (1986).
    • D. A. Dickey and W. A. Fuller, Distribution of the estimators for autoregressive time series with a unit root, Journal of the American Statistical Association 74(366), 427–431 (1979).
    • R. F. Engle and C. W. J. Granger, Co-integration and error correction: representation, estimation, and testing, Econometrica 55(2), 251–276 (1987).
    • W. K. Newey and K. D. West, A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix, Econometrica 55(3), 703–708 (1987).
    • G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control (Holden-Day, 1970).
    • Wikipedia (secondary): Spurious relationship, Dickey–Fuller test, Cointegration, Clive Granger.
    Saved on this device only.
    @teddyrookbook GitHub