Remedial Measures for Autocorrelation: GLS and Differencing

Imagine trying to predict the daily sales at a popular coffee shop. You build a model that includes the price of coffee, the local weather, and maybe a “day of the week” variable. You run your regression and get what looks like a great result. But there’s a hidden problem: today’s sales aren’t just influenced by today’s weather; they’re also incredibly similar to *yesterday’s* sales. This “memory” in your data, where the error in one period is correlated with the error in the previous period, is called autocorrelation.

When this happens, the trusty method of Ordinary Least Squares (OLS) starts to get things wrong. While your estimated coefficients (the ‘betas’) are still unbiased, the standard errors associated with them are not. They are typically understated, making your t-statistics look larger than they should. The result? You might confidently declare that “weather” is a highly significant predictor of coffee sales when, in reality, its effect is much less certain. Your model’s R-squared will likely be inflated, and your Durbin-Watson statistic will flash a warning sign. This is a serious problem for anyone who wants to make reliable inferences. Fortunately, econometrics provides a toolkit of “remedial measures” to fix this. Let’s explore the most important ones, from the theoretically perfect to the pragmatically powerful.

The ‘gold standard’ solution: Generalized Least Squares (GLS) 🔗

When we find autocorrelation, our OLS model has violated one of the key Gauss-Markov assumptions: that the error terms are uncorrelated. Because of this, OLS is no longer the BLUE-the Best Linear Unbiased Estimator. It’s still “linear” and “unbiased,” but it’s no longer “best,” meaning another estimator exists that is also unbiased but has a smaller variance. That “best” estimator is Generalized Least Squares (GLS).

In simple terms, GLS is a method that accounts for the known structure of the errors to produce estimates that are, once again, BLUE. It’s like giving OLS a new set of instructions that says, “Hey, pay attention to the fact that the errors are linked, and adjust your calculations accordingly.”

But how do we actually *do* GLS? We don’t just click a “GLS” button. Instead, we perform a clever transformation on our data so that running *OLS* on the *transformed data* is equivalent to running GLS on the original data. This brilliant workaround is most commonly known as quasi-first differencing.

Bringing GLS to life: The quasi-first difference transformation 🔗

Let’s assume our errors follow the simplest and most common pattern: a first-order autoregressive scheme, or AR(1). This means an error today is a fraction of yesterday’s error, plus a new, random shock.

Our original model is:
Yt = β0 + β1Xt + ut

And the error structure is:
ut = ρut-1 + εt

Here, ρ (rho) is the autocorrelation coefficient (a value between -1 and 1), and ε_t is a new, “well-behaved” error term that meets all the classical assumptions. The whole problem lies with ut, not εt.

The GLS procedure, in this case, is to “filter out” the problematic part of the error. We do this by using quasi-first differencing. The ‘quasi’ part is because we subtract rho times the previous value, not the full previous value.

Here is the logic:

1. Take the original equation at time t:
   Yt = β0 + β1Xt + ut
2. Take the same equation at time t-1 and multiply it by ρ:
   ρYt-1 = ρβ0 + ρβ1Xt-1 + ρut-1
3. Subtract the second equation from the first:
   (Yt - ρYt-1) = (β0 - ρβ0) + (β1Xt - ρβ1Xt-1) + (ut - ρut-1)

This looks messy, but let’s simplify. Let’s define our new, transformed variables:

• Yt* = (Yt - ρYt-1)
• Xt* = (Xt - ρβ1Xt-1)
• β0* = (β0 - ρβ0), or β0(1-ρ)

And what about the error term? Based on our AR(1) definition, (ut - ρut-1) is just εt.

So, our new, transformed model is:
Yt* = β0* + β1Xt* + εt

This new equation is now free of autocorrelation! Its error term, εt, is well-behaved. If we now run OLS on this transformed model, our estimates for β0* and β1 will be efficient and our standard errors will be correct. This transformation is the heart of the GLS procedure, and it produces BLUE estimators. Of course, in practice, we don’t know the true `ρ`, so we have to estimate it first (often by running the original OLS regression and calculating `ρ` from its residuals), a process known as Feasible Generalized Least Squares (FGLS).

Don’t lose that first observation: The Prais-Winsten transformation 🔗

The quasi-difference method is brilliant, but it has one annoying side effect. Look at the transformed variable: Yt* = (Yt - ρYt-1). To calculate this, you need the *previous* period’s value. But what about the very first observation in your dataset (at t=1)? It has no t=0 to subtract from.

The simplest solution, known as the Cochrane-Orcutt procedure, is to just drop the first observation and run the regression from t=2 to your final data point, T. If you have a massive dataset (say, 10,000 observations), losing one point is no big deal.

But what if you only have 30 observations? Or 20? Losing one data point means losing a significant chunk of your statistical power. This is a real problem in macroeconomics, where data might only be available annually.

This is where the Prais-Winsten transformation comes in. Conceived by Sigbert Prais and Christopher Winsten in 1954, it’s a modification that cleverly “rescues” that first observation. It recognizes that the first observation isn’t part of the `t-1` differencing scheme and needs its own special transformation to be comparable to the others.

While the other observations (from t=2 to T) are transformed using the quasi-difference, the first observation (at t=1) is transformed by multiplying it by √(1-ρ²):

• Y1* = Y1 * √(1-ρ²)
• X1* = X1 * √(1-ρ²)
• β0,1* = β0 * √(1-ρ²)

The statistical reasoning is that the variance of the first error u1 (assuming a stationary process) is σ²/(1-ρ²), while the variance of the subsequent transformed errors εt is just σ². This transformation scales the first observation so that its error variance matches all the others. By applying this specific fix to the first data point and the quasi-difference to the rest, the Prais-Winsten method allows the GLS procedure to use the full sample, preserving precious data and increasing efficiency, especially in smaller datasets.

When the past is everything: First-differencing for unit roots 🔗

We’ve been assuming ρ is a fraction, like 0.7 or 0.9. But what happens if the autocorrelation is *so* strong that ρ is essentially equal to +1? This is a very special and common case in economics, known as a “unit root.”

If ρ = 1, our error structure ut = ρut-1 + εt becomes ut = ut-1 + εt. This is a “random walk.” It means the error today is just yesterday’s error *plus* a new random shock. The memory isn’t just strong; it’s permanent. The influence of a shock never fades away.

Many economic time series, like stock prices or GDP, behave this way. They are “non-stationary”-they don’t revert to a mean and their variance isn’t constant. Running OLS in this situation is disastrous and can lead to “spurious regression,” where you find a statistically significant relationship between two variables that are, in reality, totally unrelated.

So what’s the remedy? Let’s look back at our quasi-difference transformation:
Yt* = (Yt - ρYt-1)

If we set ρ = 1, the transformation becomes:
Yt* = (Yt - Yt-1)

This is simply first-differencing. We are no longer looking at the *level* of Y, but the *change* in Y from one period to the next (often written as ΔYt). When we suspect a unit root, the most effective remedial measure is to transform all our variables by taking their first difference and running the regression on these new, differenced variables:

ΔYt = β1ΔXt + εt

This transformation is often enough to make the variables stationary and remove the autocorrelation, making our OLS estimates valid again. A key clue that you might need to do this is a Durbin-Watson statistic that is very small (e.g., under 1.0), which indicates strong positive autocorrelation, a hallmark of a unit root.

The modern, pragmatic fix: Newey-West HAC standard errors 🔗

The solutions we’ve discussed-GLS, Prais-Winsten, and First-Differencing-are powerful, but they all share a common assumption: that we, the researchers, correctly identified the *exact* form of the problem. GLS, for example, works perfectly *if* the problem is a pure AR(1) process. But what if it’s a more complex AR(2) process? Or a moving average (MA) process? What if we have both autocorrelation *and* heteroscedasticity (non-constant error variance) at the same time?

In the real world, data is rarely so clean. This is where the Newey-West method provides an incredibly useful and robust solution. This approach has a fundamentally different philosophy.

Instead of trying to “fix” the model by transforming the data, the Newey-West method says: “Let’s just run our original OLS regression. We know that in large samples, the OLS coefficients are still consistent (they get arbitrarily close to the true values as sample size increases). The *only* thing that’s wrong is our standard errors. So, let’s just invent a new formula to calculate the standard errors that is ‘robust’ to the problem.”

That’s exactly what it does. The Newey-West estimator provides Heteroscedasticity-and-Autocorrelation-Consistent (HAC) standard errors. It’s a complex formula that adjusts the standard errors to be reliable *even if* both autocorrelation and heteroscedasticity are present, and even if we don’t know their exact form.

This method doesn’t change your β coefficients at all. It just corrects the “margin of error” around them. This has become the standard approach in much of modern applied econometrics, especially when working with large time-series datasets where “messy” errors are the norm, not the exception. The key caveat is that this is a large-sample solution; its statistical properties are only guaranteed when you have plenty of data. In a small sample, a correctly-specified GLS model is still theoretically more efficient.

Ultimately, dealing with autocorrelation is a core skill in econometrics. The key is to first diagnose the problem (using tools like the Durbin-Watson test or correlograms) and then choose the remedy that best fits the situation: the elegant GLS/Prais-Winsten for pure cases, first-differencing for unit roots, or the robust Newey-West HAC errors for the complex, real-world problems that large datasets often present.

What do you think? In your own field or studies, which do you think is a more common problem: “pure” autocorrelation (where GLS is perfect) or a “messy” combination of autocorrelation and heteroscedasticity (where Newey-West is needed)? And why do you think it’s so important for economists to get standard errors right, even if the coefficient estimates themselves are unbiased?

References

1. https://www.statlect.com/fundamentals-of-statistics/generalized-least-squares
2. https://en.wikipedia.org/wiki/Prais%E2%80%93Winsten_estimation
3. https://www.investopedia.com/terms/d/durbin-watson-statistic.asp
4. https://en.wikipedia.org/wiki/Newey%E2%80%93West_estimator