Solving Measurement Error with Instrumental Variables

In a perfect world, measuring the impact of one thing on another would be simple. How much does one extra year of education increase your income? How much does a new fertilizer increase crop yield? We’d just collect the data, run a standard regression (like an Ordinary Least Squares, or OLS), and get our answer. But as you know, we don’t live in a perfect world. Our data is often messy, incomplete, or “noisy.”

One of the most common and frustrating problems in economics is measurement error. What if the education data we have isn’t perfect? What if some people rounded up their years of schooling, or what if the survey just asked for a “degree level” instead of exact years? This is called an “errors-in-variables” problem. When the variable we’re using to make a prediction (our independent variable, or ‘X’) is measured incorrectly, it creates a massive headache. Our standard OLS method, which is the workhorse of statistics, starts to give us wrong answers. It becomes biased and, even worse, inconsistent. This is where a clever and powerful technique called the Instrumental Variables (IV) method comes to the rescue. It’s a way to find a clear signal through all the statistical noise.

The big problem: When our numbers lie 🔗

Let’s first understand *why* this measurement error is so disastrous. Imagine you’re trying to find the relationship between the *true* amount of fertilizer used (let’s call it $X^*$) and the crop yield (Y). But you don’t have $X^*$. All you have is the farmer’s “recalled amount” of fertilizer (X), which is $X^*$ plus some random error ($v$).

So, $X = X^* + v$.

The model we *want* to estimate is: $Y = \beta_0 + \beta_1 X^* + \epsilon$

But the model we *have* to estimate is: $Y = \beta_0 + \beta_1 (X – v) + \epsilon$

When our standard OLS method looks at this, it gets confused. It sees the variable X, but that X is contaminated by the measurement error $v$. This contamination makes our X variable correlated with the new, combined error term. This violates a core assumption of OLS, and as a result, the estimate it gives us for $\beta_1$ (the effect of fertilizer) will be wrong. Specifically, it will be biased, and in this case, it will likely be biased toward zero. We’ll underestimate the true effect of the fertilizer.

This general problem is called endogeneity-when our independent variable is correlated with the error term. Measurement error is just one major cause. This is where we need to call for backup.

What is an instrumental variable? 🔗

An instrumental variable, often just called an “instrument” and labeled ‘Z’, is a *third* variable that we bring in to solve our problem. It’s a bit like a special translator or a clean “middle-man.”

Think of it this way: Our variable X is “tainted.” We can’t trust it directly. We need to find another variable Z that can “vouch for” X. To be a valid instrument, Z must follow two strict, golden rules:

Rule 1: The instrument must be relevant 🔗

This first rule is simple: the instrument (Z) must be highly correlated with our problematic variable (X). If our instrument has no connection to X, it’s useless.

For example, if we’re trying to fix the measurement error in our “fertilizer” variable (X), a terrible instrument would be “the farmer’s favorite color.” It has zero correlation with how much fertilizer they use.

A *good* instrument might be “the price of fertilizer in that farmer’s district” (Z). Why? Because the local price (Z) almost certainly affects how much fertilizer the farmer buys (X). When the price is high, they’ll probably buy less, and when it’s low, they’ll buy more. This is a strong, testable correlation. This is the relevance condition.

Rule 2: The instrument must be exogenous 🔗

This is the harder rule, and the one that requires the most careful thought. The instrument (Z) must be uncorrelated with the model’s error term ($\epsilon$) *and* the measurement error ($v$).

This means the *only* way the instrument (Z) can affect our outcome (Y) is *through* its effect on our problematic variable (X). It can’t have its own separate, secret path to Y.

Let’s check our fertilizer price example. Does the “local price of fertilizer” (Z) affect “crop yield” (Y)?

• Path 1 (The good path): Price (Z) $\rightarrow$ Amount Used (X) $\rightarrow$ Crop Yield (Y). This is the path we *want* to use.
• Path 2 (A potential bad path): Price (Z) $\rightarrow$ Crop Yield (Y)? Is this possible? Well, what if the local price of fertilizer is high only in very remote, mountainous regions? And what if those same regions *also* have poor soil quality? In that case, our instrument (Z) is correlated with “soil quality,” which is part of our error term ($\epsilon$). This would make it a *bad* instrument.

This is called the exclusion restriction. We must be able to convincingly argue that our instrument (Z) is “exogenous”-meaning it’s determined by forces outside of our main equation and doesn’t suffer from its own set of problems. Finding a variable that is both relevant and truly exogenous is the art and science of IV estimation.

How the instrumental variables estimator actually works 🔗

So how does this magic work? The OLS method works by looking at how X and Y move together (their covariance). But we know X is “tainted.”

The IV method performs a clever substitution. Instead of using all the variation in X (which includes the “bad” variation from the measurement error), it *only* uses the part of the variation in X that is “sparked” or “caused by” our clean instrument, Z.

It’s like filtering out the noise. We’re telling the model, “Don’t look at all the random reasons why farmers used different amounts of fertilizer. I only want you to look at the differences in fertilizer use *that were caused by differences in local price*.” Since we believe the local price (Z) is “clean” (exogenous), the part of X that it influences must also be “clean.”

The math behind the method 🔗

In a simple two-variable model, the OLS estimator for the slope is $\hat{\beta}_{OLS} = \frac{Cov(X, Y)}{Var(X)}$.

The IV estimator cleverly swaps out the “tainted” parts. The formula for the IV estimator, as given in many textbooks, is:

$$ \hat{\beta}_{IV} = \frac{\sum z_i y_i}{\sum z_i x_i} $$

(This is the form for variables expressed as deviations from their means).

Let’s break this down. The numerator, $\sum z_i y_i$, is essentially the sample covariance between our *clean instrument* (Z) and our *outcome* (Y). The denominator, $\sum z_i x_i$, is the sample covariance between our *clean instrument* (Z) and our *problematic variable* (X).

The logic is that $\frac{Cov(Z, Y)}{Cov(Z, X)}$ gives us a clean estimate of the effect of X on Y. We are isolating *only* the part of the X-Y relationship that is “instrumented” by Z. By doing this, we bypass the measurement error that was contaminating the $Var(X)$ term in the OLS formula.

A crucial trade-off: Consistency vs. unbiasedness 🔗

Now, this powerful method comes with a very important trade-off. We must be clear on two key statistical properties: unbiasedness and consistency.

• Unbiasedness: An estimator is unbiased if, on average, it hits the true population parameter. If you took 100 different samples and calculated your estimate 100 times, the *average* of your 100 estimates would be the true value. OLS, when its assumptions hold, is unbiased.
• Consistency: An estimator is consistent if, as your sample size gets larger and larger (approaching infinity), the estimate gets closer and closer (converges) to the true population parameter.

Here’s the catch: The IV estimator is not unbiased. In any given, finite sample (especially small ones), the IV estimate is likely to be biased. It won’t, on average, hit the true value.

So why on earth would we use it? Because it is consistent.

When we have measurement error, our OLS estimator is *both* biased *and* inconsistent. It’s wrong in small samples, and it’s *still* wrong in massive samples. It never finds the true answer. The IV estimator, on the other hand, sacrifices unbiasedness for the far more important property of consistency. We are accepting a small-sample bias in exchange for the guarantee that if we get enough data, our estimate will converge to the correct answer. The OLS estimate will *never* converge to the correct answer. It’s a fantastic bargain to make.

It’s helpful to remember that OLS is just a special case of IV. OLS is what you get if you use X to instrument for itself (Z = X). This only works if X is perfectly “clean” (exogenous) to begin with. When it’s not, we must find an instrument that is.

A final note on the challenge of instrument selection 🔗

This entire method hinges on finding a valid instrument. As we discussed, a valid instrument must be both relevant and exogenous.

1. We can test for relevance. We can run a regression of X on Z (called the “first-stage” regression) and check if Z is a statistically significant predictor of X. If it’s not (a “weak instrument”), our IV estimates can be very unreliable.
2. We cannot test for exogeneity. The exclusion restriction-the assumption that Z is uncorrelated with the error term-is an *assumption*. It cannot be proven with data.

This is why the most famous and respected papers using IV are often celebrated for their cleverness in finding a believable instrument. For example, to study the effect of education on income, researchers used a person’s “quarter of birth” as an instrument. The logic? Compulsory schooling laws forced kids born in different quarters to start school at different ages, which slightly affected their total years of education (X). But (the argument goes) the quarter you were born in (Z) has no direct effect on your future income (Y) *except* through that education channel.

In policy, researchers wanting to study the effect of smoking (X) on health (Y) have used state-level cigarette taxes (Z) as an instrument. The argument is that taxes (Z) affect smoking behavior (X), but the tax rate itself doesn’t directly impact your personal health (Y), except by influencing your smoking.

Ultimately, IV is a powerful tool for fixing a “broken” variable, but it places all the burden on finding an external variable that is both strong and, most importantly, believable.

What do you think? Can you think of another real-world situation where a variable you want to measure (like “employee motivation” or “firm innovation”) would be very difficult to measure perfectly? What kind of problems would that cause?

References

1. https://www.investopedia.com/terms/i/instrumental-variable.asp
2. https://economics.yale.edu/~fmap/course/551/iv_lecture.pdf
3. https://www.nber.org/papers/w3352