The Consequences of Measurement Errors in Regression Models

In a perfect world, all our data would be perfectly accurate. Every income reported, every expense logged, every hour worked-all captured with pinpoint precision. But as any economist or data scientist will tell you, we don’t live in that world. We live in a world of surveys with fuzzy memories, data entry typos, and variables that are just plain hard to measure. This is the world of measurement error, or “errors in variables,” and it’s not just a minor nuisance. Depending on *where* this error lives, it can fundamentally mislead us, causing our statistical models to tell us stories that are systematically wrong.

When we build a regression model, we’re trying to find the true relationship between variables. But what happens when the data we feed that model is flawed? Let’s explore the consequences, because they are drastically different depending on whether the flaw is in the variable we’re predicting or the variable we’re using to predict.

What happens when our target is a bit fuzzy? (Error in the dependent variable) 🔗

Let’s start with the more “forgiving” scenario. Imagine you’re a real estate analyst trying to model home prices. Your model is:

$$Price = \beta_0 + \beta_1 \times SquareFootage + \epsilon$$

You are trying to predict the dependent variable (Y), which is `Price`. Now, let’s say your `SquareFootage` data (the independent variable, X) is perfect, but your `Price` data is a bit messy. Maybe it includes closing costs for some homes but not others, or there are typos in the records.

Your observed price, $Y$, isn’t the true price, $Y^*$. Instead, $Y = Y^* + u$, where $u$ is the measurement error. Your regression model effectively becomes:

$$Y^* + u = \beta_0 + \beta_1 \times X + \epsilon$$

If we rearrange this, we get $Y^* = \beta_0 + \beta_1 \times X + (\epsilon – u)$. The new, combined error term is $(\epsilon – u)$.

The good news and the bad news 🔗

Here’s the good news: as long as this measurement error $u$ is random and not correlated with your $X$ variable (`SquareFootage`), your OLS (Ordinary Least Squares) estimators for $\beta_0$ and $\beta_1$ are still unbiased. On average, your model will still find the correct relationship. Your estimate for the price per square foot ($\beta_1$) isn’t systematically skewed high or low.

So, what’s the catch? The bad news is all about precision. The variance of this new, combined error term is larger than the variance of the original error $\epsilon$. This inflated error variance has major consequences:

• Less precise estimates: The variance of your $\beta$ estimators increases. Think of it like trying to shoot at a target in a gusty wind. You’re still centered on the bullseye (unbiased), but your shots are more spread out.
• Lower t-statistics: Because the standard errors of your coefficients are larger, your t-statistics will be smaller.
• Wider confidence intervals: You’ll be less certain about the true range of your $\beta$ parameters.
• Higher p-values: You might incorrectly conclude that `SquareFootage` has no significant effect on `Price` (a Type II error), simply because the “noise” from the measurement error drowned out the “signal” from the true relationship.

In short, measurement error in the dependent variable is like trying to listen to a clear radio station with a lot of static. The music is still the right music, but it’s much harder to hear clearly.

The real troublemaker: When our measuring stick is wrong (Error in the independent variable) 🔗

Now, let’s flip the script. Imagine your `Price` data (Y) is perfect, but your `SquareFootage` data (X) is flawed. The true, architectural-grade measurement is $X^*$, but what you have, $X$, is based on hurried estimates or old, inaccurate city records.

So, $X = X^* + v$, where $v$ is the measurement error in our independent variable.

Our true model is $Y = \beta_0 + \beta_1 \times X^* + \epsilon$. But we can’t run this model. We have to run this one:

$$Y = \beta_0 + \beta_1 \times (X – v) + \epsilon$$

If we rearrange this to look like a standard regression, we get:

$$Y = \beta_0 + \beta_1 \times X + (\epsilon – \beta_1 v)$$

This looks harmless enough, but it hides a devastating problem. The new, combined error term is $(\epsilon – \beta_1 v)$. The independent variable we are using is $X$, which is equal to $X^* + v$.

Do you see the problem? The error $v$ is now in both the new error term *and* the independent variable $X$. This means our independent variable $X$ is now correlated with the error term. This isn’t just a minor issue; it violates one of the most fundamental assumptions of OLS regression.

This violation is catastrophic. The consequences are far more severe than in the first case:

1. The OLS estimators are biased. They are systematically wrong, even on average.
2. The OLS estimators are inconsistent. This is the truly fatal flaw.

Why “inconsistent” is so much worse (The Asymptotic Bias Problem) 🔗

With many statistical problems, we can just “get more data.” If an estimator is biased but *consistent*, it means the bias will shrink and eventually disappear as our sample size grows to infinity. We can solve the problem with a bigger sample.

Not here. Inconsistency, also known as asymptotic bias, means that our estimator is wrong, and it *stays* wrong, no matter how much data we collect. Even with a billion data points, our $\hat{\beta}_1$ will not converge to the true $\beta_1$. It converges to the wrong number.

In the simple case of one independent variable, this bias has a specific name: attenuation bias, or “bias to zero.” The OLS estimator will systematically underestimate the true magnitude of the effect.

The formula for what our estimator converges to (its “probability limit” or plim) is:

$$plim(\hat{\beta}_1) = \beta_1 \left( \frac{\sigma_{X^*}^2}{\sigma_{X^*}^2 + \sigma_v^2} \right)$$

Let’s break down that fraction. It’s the variance of the *true* variable $X^*$ divided by (the variance of the *true* variable $X^*$ + the variance of the *error* $v$). Since variances can’t be negative, this fraction is always less than 1.

This means our estimate is a “watered-down” version of the real effect. If the true effect of education on income is $\beta_1 = 0.10$ (one more year of school adds 10% to income), but there is measurement error in our “years of education” data, our model might give us an estimate of $\hat{\beta}_1 = 0.06$. We would be making policy decisions based on a severe underestimation of the returns to education.

This is like having a crooked bow in archery. You can practice all day (collect more data), but you will never hit the bullseye because you are fundamentally aiming at the wrong spot.

What if everything is measured wrong? (Errors in Both X and Y) 🔗

This is the most realistic scenario. What happens if our `Price` data (Y) has typos *and* our `SquareFootage` data (X) has estimation errors?

It’s essentially the worst of both worlds. The measurement error in $Y$ adds “noise,” increases the variance of our estimators, and makes our results less precise (as in our first case). The measurement error in $X$ introduces the severe bias and inconsistency (as in our second case).

The OLS estimator remains biased and inconsistent. In most typical scenarios, the attenuation bias (underestimation) from the $X$ error is the dominant problem. The model will still give us a conservative, but fundamentally incorrect, estimate of the true relationship.

A classic example: Why is saving so hard to predict? 🔗

One of the most famous applications of the errors-in-variables model is Milton Friedman’s Permanent Income Hypothesis (PIH). Friedman’s theory states that people don’t base their consumption choices on their income *this week* or *this month*. They base it on their expected, long-term average income, which he called permanent income.

Here’s how it breaks down into a measurement error problem:

• Measured Income ($X$): This is what we observe in a survey. It’s what a person earned in a given year.
• Permanent Income ($X^*$): This is their true, long-term average earning potential. This is the variable we *wish* we could see, but we can’t.
• Transitory Income ($v$): This is the “measurement error.” It’s the difference between measured and permanent income. It could be a one-time bonus, a temporary layoff, or a lucky break. So, $X = X^* + v$.

The same logic applies to consumption:

• Measured Consumption ($Y$): What a person spent in a given year.

* Permanent Consumption ($Y^*$): Their normal, long-term sustainable spending level.

* Transitory Consumption ($u$): The “error,” like an unexpected car repair or an impulse vacation. So, $Y = Y^* + u$.

Friedman’s theory is that $Y^*$ is a function of $X^*$. But as economists, we can only run a regression on the *measured* data ($Y$ on $X$). This is a classic errors-in-variables model with errors in both variables.

Because of the measurement error in $X$ (the “transitory income”), OLS will suffer from attenuation bias. It will systematically underestimate the true marginal propensity to consume (MPC) out of permanent income. Early models that ignored this problem found a low MPC, suggesting people saved a lot. Friedman showed that this was a statistical illusion-a direct consequence of measurement error. The true, long-term MPC was actually much higher.

Ultimately, data is rarely, if ever, perfect. Ignoring measurement error is easy, but it’s perilous. While random errors in our dependent variable just add noise, errors in our independent variables are a far more sinister problem, biasing our results and leading us to incorrect conclusions, no matter how much data we have.

What do you think?

Thinking about your own field or interests, where do you suspect measurement error might be leading to biased results? How much trust should we put in a model if we know the data isn’t perfect?

References

1. https://online.stat.psu.edu/stat501/lesson/11/11.2
2. https://www.nber.org/system/files/chapters/c4441/c4441.pdf