Using the Instrumental Variable Estimator for Stochastic Regressors

Imagine you’re trying to figure out if coffee *really* makes people more productive. You collect data on coffee sales and office output. But there’s a problem: what if the most motivated (and already productive) people are the ones who drink the most coffee? Your simple regression might just be picking up on that motivation, not the true effect of the coffee itself. This “contamination” from a hidden variable is one of the biggest headaches in econometrics. It’s a case where our explanatory variable (coffee) is correlated with the error term (which contains motivation). This is often called endogeneity, or the problem of stochastic regressors.

When our standard tool, Ordinary Least Squares (OLS), faces this problem, it gives us biased and inconsistent estimates. We can’t trust the results. We need a way to isolate the *clean* part of the relationship. This is where a powerful and clever technique comes in: the Instrumental Variable (IV) Estimator.

What is endogeneity and why does it break our models? 🔗

Let’s quickly refresh our memory on our workhorse model, OLS. For OLS to give us a true, unbiased estimate of a variable’s effect, it relies on a critical assumption: our explanatory variables (the ‘X’s) must be uncorrelated with the error term (the ‘u’). The error term is a catch-all bucket for everything *else* that affects our outcome (the ‘Y’) but isn’t in our model. This includes unobserved factors like innate ability, motivation, luck, or measurement error.

Endogeneity occurs when this assumption is violated: $\text{Cov}(X, u) \neq 0$. The explanatory variable we’re interested in is, for some reason, tangled up with those unobserved factors in the error term.

This happens for a few common reasons:

• Omitted Variables: This was our coffee example. We omitted ‘motivation,’ which affects both coffee drinking (X) and productivity (Y).
• Simultaneity: What if X causes Y, but Y *also* causes X? For example, crime rates might affect the number of police officers, but the number of police officers also affects crime rates.
• Measurement Error: If we measure our X variable poorly, that measurement error can end up in the error term, creating a correlation.

When we have endogeneity, OLS can’t tell what’s causing what. It sees the correlation between X and Y and mistakenly attributes *all* of it to X’s causal effect, even the part that’s really coming from the hidden ‘u’. The job of the Instrumental Variable (IV) Estimator is to fix this.

How the instrumental variable estimator saves the day 🔗

Think of your endogenous variable (X) as “contaminated.” It has two parts: a “good” part that is independent of the error term, and a “bad” part that is correlated with it. OLS uses the *whole* variable, “good” and “bad” parts combined, which is why it gets a biased answer.

The IV technique is a clever workaround. It basically says, “Okay, since X is contaminated, I’m not going to use it directly. Instead, I’ll find a *new* variable, let’s call it ‘Z’.” This new variable, ‘Z’, is our instrument.

This instrument ‘Z’ acts like a clean data filter. It’s job is to “talk” to the endogenous variable ‘X’ and pull out *only* the “good” variation-the part that is *not* correlated with the error term ‘u’. It then uses this clean, “instrumented” variation from X to estimate its true effect on Y.

But for this to work, our chosen instrument ‘Z’ must be very special. It can’t just be any variable. It must follow two strict rules.

The two golden conditions for a valid instrument 🔗

Finding a variable that satisfies both of these conditions is the hardest part of IV estimation. It requires deep thinking, creativity, and strong justification.

1. The relevance condition: The instrument must be related to the endogenous variable 🔗

This first condition is simple: our instrument (Z) must have a meaningful correlation with our problematic endogenous variable (X). The technical way to say this is $\text{Cov}(Z, X) \neq 0$.

Think about it: if our instrument has *no* relationship with our endogenous variable, how can it possibly “filter” it or pull out its “good” variation? It can’t. If Z and X are total strangers, Z is useless to us. We need our instrument to have a strong, clear voice in predicting X.

In a model, we can (and must) test this. We run a regression (called the “first-stage” regression) where we try to predict X using Z (and any other exogenous variables in our model). We then check if the coefficient on Z is statistically significant. If it’s not, or if it’s very weak, then our instrument is not relevant.

2. The exogeneity condition: The instrument must be uncorrelated with the error term 🔗

This is the “golden” rule, the one that’s much harder and more important. The instrument (Z) must be exogenous. This means it cannot be correlated with the error term ‘u’ from our main equation. The technical expression is $\text{Cov}(Z, u) = 0$.

This also implies a crucial restriction: the instrument can *only* affect our outcome (Y) *through* its effect on the endogenous variable (X). It cannot have its own, separate path to Y. It can’t be related to any of those unobserved factors (like motivation or ability) that are hiding in ‘u’.

Why is this so critical? Because the entire logic of IV is that Z is our “clean” variable. We are *assuming* that any relationship Z has with Y *must* be coming via X. If Z is also contaminated (correlated with u), then we haven’t solved anything. We’ve just replaced one contaminated variable with another.

The scariest part? Unlike the relevance condition, we can’t *statistically test* the exogeneity condition. It is an assumption that we, the researchers, must defend based on economic theory, logic, and common sense. This is where the “art” of econometrics truly lies.

A classic example: Using quarter of birth for education 🔗

Let’s make this concrete. A famous question in economics is: “What is the causal effect of an extra year of schooling (X) on future wages (Y)?”

This is a classic endogeneity problem. Why? Because a person’s “innate ability” (which is unobserved and stuck in the error term ‘u’) likely affects *both* how much schooling they get (X) and their future wages (Y). Smarter people might get more schooling *and* earn more, regardless of the schooling itself. OLS will be biased.

So, we need an instrument (Z). We need something that affects *how much schooling people get* (relevance) but has *no other effect on their future wages* (exogeneity).

Economists Joshua Angrist and Alan Krueger had a brilliant idea: a person’s quarter of birth. Their logic was based on compulsory schooling laws. In many places, kids start school in the calendar year they turn six, and they can drop out when they turn 16.

• Relevance: If you’re born in the first quarter of the year (Jan-Mar), you’ll be one of the oldest in your grade. You’ll turn 16 while still in, say, 10th grade. But if you’re born in the last quarter (Oct-Dec), you’ll be one of the youngest. You might not turn 16 until you’re already in 11th grade. Because of the dropout law, people born later in the year are “forced” to get slightly more schooling, on average. So, quarter of birth (Z) *is* correlated with years of schooling (X).
• Exogeneity: Is your birth quarter (Z) correlated with your innate ability (u) or anything else that affects your wages? It’s highly unlikely. We generally assume that which quarter of the year you’re born in is random. It doesn’t make you smarter, more motivated, or have better connections. Its *only* path to your future wages is through that little “push” it gives you in schooling.

It’s a valid instrument! By using quarter of birth as an instrument, Angrist and Krueger were able to isolate the *clean* variation in schooling and find a credible estimate for its true effect on wages.

How the math untangles the estimate 🔗

So how does the calculator actually *use* Z? It all comes down to “moment conditions,” which are assumptions about the data’s population.

In OLS, we make the moment condition $E(X’u) = 0$ (we assume X is exogenous). We solve for the $\beta$ (beta) that makes this true. But in our endogeneity case, we know this assumption is false.

In IV, we *swap* that assumption. We use our instrument Z and assume the exogeneity condition: $E(Z’u) = 0$. We are now solving for the $\beta$ that makes *this* new assumption hold true. We’re using the fact that Z is “clean” to pin down our estimate.

By substituting the equation $y = X\beta + u$ into our moment condition, we get $E(Z'(y – X\beta)) = 0$. With a bit of algebra, we can solve for our vector of parameters, $\beta$.

The famous solution for $\beta$ in the population is: $$ \beta = [E(Z’X)]^{-1} E(Z’Y) $$ This formula shows exactly how the instrument is used. It uses the relationship between the instrument and X ($E(Z’X)$), and the relationship between the instrument and Y ($E(Z’Y)$), to bridge the gap and find the true $\beta$.

In a real-world sample, we just replace the population expectations (E) with our sample data matrices (Z, X, and Y), giving us the IV estimator we can actually compute: $$ \hat{\beta}_{IV} = (Z’X)^{-1} Z’Y $$ For this to work, the matrix $(Z’X)$ must be invertible (have full rank). This is the rank condition, and it’s mathematically related to the relevance condition-if our instruments are not relevant, this matrix can’t be inverted, and the whole thing breaks down.

The big limitation: What if your instrument is weak? 🔗

The IV estimator is a fantastic tool, but it’s not a magic wand. Its biggest practical limitation is the one we’ve been discussing: finding a *good* instrument. It’s incredibly difficult.

But what happens if we use a “bad” instrument? If we use an instrument that violates the exogeneity condition (it’s correlated with u), our IV estimates will be just as biased and inconsistent as OLS. We’ve gained nothing.

A more subtle problem is the weak instrument problem. This happens when our instrument *is* valid (it’s exogenous), but it fails the relevance condition *badly*. It’s only *very weakly* correlated with our endogenous variable X. Our “first-stage” F-statistic is very low.

When this happens, the IV estimator becomes very imprecise. Our standard errors can become enormous, making our estimate practically useless (e.g., “the effect is between -50 and +50”). Even worse, in small samples, a weak instrument can make the IV estimator *more* biased than the original OLS estimator we were trying to fix!

This creates a painful trade-off. Sometimes, a small, known bias from OLS is better than the massive variance and potential bias from a very weak instrument. Econometrics is full of these judgment calls.

Ultimately, the Instrumental Variable estimator is one of the most important methods for getting at *causality*. It allows us to step beyond simple correlation and find credible answers to ‘what if’ questions, as long as we can find a valid, and ideally strong, instrument.

What do you think? Can you think of another real-world example where a simple regression would give a misleading answer due to endogeneity? And what might you (creatively) try to use as an instrument in that case?

References

1. https://www.investopedia.com/terms/i/instrumental-variable.asp
2. https://blogs.worldbank.org/impactevaluations/introduction-instrumental-variables
3. https://ocw.mit.edu/courses/14-387-applied-econometrics-fall-2017/resources/mit14_387f17_lec10/