Properties of OLS Estimators: BLUE and Gauss-Markov Theorem

When you first encounter econometrics, you’re often given a cloud of data points and a simple goal: draw the best possible straight line through it. That line is your model, and the method everyone tells you to use is Ordinary Least Squares (OLS). But why OLS? Is it just a random convention, or is there something special about it? It turns out, OLS isn’t just *a* method; it’s a method with some truly powerful and unique “superpowers.”

Think of it like choosing a tool to build a house. You don’t just want *a* hammer; you want the one that’s perfectly balanced, won’t break, and drives the nail straight every time. OLS is that precision tool for regression analysis. Its properties are so reliable that they’ve become the foundation of econometric modeling. We can break these “superpowers” down into two main categories: its mechanical, algebraic properties (what it does by construction) and its deeper, statistical properties (how it behaves when we think about the real world). Let’s dive in and see why OLS earned its top-tier status.

The mechanics: Algebraic properties of OLS 🔗

First, let’s look at the “algebraic” or “numerical” properties. These are characteristics that are always true by the very definition of the OLS method. They are mathematical facts that come directly from the formulas used to calculate the OLS intercept (β̂₀) and slope (β̂₁). They hold true for any dataset, regardless of the underlying theory.

Property 1: The regression line passes through the sample means 🔗

This is the most intuitive property. The OLS regression line will always, without exception, pass directly through the “center of gravity” of your data cloud. This center point is defined by the sample mean of your X variable (X̄) and the sample mean of your Y variable (Ȳ).

If you have your estimated regression equation, Ŷ = β̂₀ + β̂₁X, and you plug in the average value of X (X̄), the predicted value of Y you get will be exactly the average value of Y (Ȳ). This shows that the regression line is a true “central” estimator. It balances the data perfectly, with the pull of the data points above the line at the mean being perfectly offset by the pull of the data points below it.

Property 2: The mean of the fitted values equals the mean of the actual values 🔗

This is a direct consequence of the first property. The “fitted values” (often called Ŷ or “Y-hat”) are the predictions your model makes for each observation. This property states that if you take the average of all your model’s predictions (the mean of Ŷ), it will be exactly equal to the average of the actual Y values you were trying to predict (the mean of Y).

In simple terms: Mean(Ŷ) = Mean(Y). This is a reassuring sanity check. It tells us that our model, on average, isn’t systematically over-predicting or under-predicting. The average of its guesses is “anchored” to the average of the reality it’s trying to model.

Property 3: The residuals and regressors are orthogonal (uncorrelated) 🔗

This one sounds complex, but it’s the most important algebraic property. “Orthogonal” is a geometric term that, in this context, simply means uncorrelated. The residuals (e) are the errors of your model-the difference between the actual value (Y) and your fitted value (Ŷ) for each point. This property says that the set of all your residuals (e) is completely uncorrelated with your explanatory variable (X).

Why is this critical? The whole point of regression is to have X explain as much of Y as possible. The residual (e) is, by definition, the part of Y that X cannot explain. The OLS formulas are explicitly designed to “squeeze” every last drop of explanatory power out of X. If the residuals *were* correlated with X, it would mean there’s still some pattern in the errors that X could have explained, and our line wouldn’t actually be the “best fit.” OLS guarantees that the errors are just random noise, with no lingering information related to X left behind.

From sample to population: Statistical properties of OLS 🔗

The algebraic properties are neat, but they only tell us about our specific sample. As economists and data scientists, we’re rarely just interested in our sample; we want to use our sample to make inferences about the entire population. We want to know if our calculated β̂₁ (the slope from our sample) is a good estimate of the *true* β₁ (the unknown slope for the whole population).

This is where statistical properties come in. These properties are not automatically true; they only hold if a set of assumptions-known as the Classical Linear Regression Model (CLRM) assumptions-are met. These assumptions include things like the relationship being linear, the errors having a mean of zero, and no perfect collinearity.

Property 1: Unbiasedness 🔗

An estimator is “unbiased” if its expected value is equal to the true population parameter. In plain English: if you could draw 1,000 different random samples from the population and calculate 1,000 different β̂₁ estimates, the *average* of all those 1,000 estimates would be spot-on the true, unknown population value β₁.

Think of it like throwing darts. Unbiasedness doesn’t mean *your* single dart (your one β̂₁) hits the bullseye. It means you’re not systematically aiming high or low. On average, your collection of attempts is centered perfectly on the target. This gives us confidence that our method isn’t inherently flawed. Provided the CLRM assumptions hold (especially the one that says the error term’s expected value is zero), OLS is an unbiased estimator.

Property 2: Consistency 🔗

Consistency is about what happens as your sample size (n) grows. An estimator is “consistent” if it converges to the true population value as the sample size gets infinitely large.

This is an incredibly important property. It’s the mathematical justification for “big data.” It means that while your estimate from a small sample might be a bit off (due to random chance), your estimate will get more and more precise as you collect more data. With a large enough sample, you can be almost certain that your β̂₁ is very, very close to the true β₁. OLS is a consistent estimator, which gives us faith that gathering more data is a worthwhile effort.

The crown jewel: The Gauss-Markov Theorem and BLUE 🔗

We’ve established that OLS is unbiased (it’s “on target” on average) and consistent (it gets better with more data). But are there *other* estimators that are also unbiased? Yes, infinitely many! So how do we choose? This is where the most famous theorem in introductory econometrics comes in: the Gauss-Markov Theorem.

The theorem states that, under the CLRM assumptions, the OLS estimator is BLUE. This is an acronym that packs a huge punch.

B: Best 🔗

This is the most important letter. “Best” in this context means minimum variance. Let’s go back to the dart analogy. We already know OLS is unbiased, meaning the *average* of 1,000 dart throws is the bullseye. But what about the *spread* of those darts? A “bad” estimator might be unbiased, but its throws land all over the dartboard. A “good” estimator will have a tight cluster of darts right around the center.

“Best” means that the OLS estimator produces the tightest possible cluster. Its variance-the spread of its estimates from sample to sample-is smaller than that of *any other* linear unbiased estimator. OLS is the most precise, most efficient, and most reliable estimator in its class.

L: Linear 🔗

This simply means that the OLS estimator (the formula for β̂₁) is a linear function of the dependent variable, Y. It’s a weighted sum of the Y values. This is a practical property. It means the estimator is simple to calculate, easy to understand, and mathematically tractable, unlike some highly complex non-linear methods.

U: Unbiased 🔗

We’ve already covered this. It’s the “unbiased” part of BLUE. Its expected value is the true population value. It’s correct on average.

E: Estimator 🔗

This is self-explanatory. OLS is an estimator-a formula or rule we use with our sample data to estimate an unknown population parameter.

Putting it all together, BLUE means OLS is the Best Linear Unbiased Estimator. It’s the “best in class.” If your data meets the CLRM assumptions, you cannot find another estimator that is both linear and unbiased and has a smaller variance than OLS. This is why OLS is the default and the gold standard. It’s not just a guess; it’s provably the most precise linear and unbiased tool we have. These are the properties that justify its use in everything from academic research to economic policy analysis at central banks.

So, the next time you run a regression, you’re not just finding *a* line-you’re using a method that is mathematically guaranteed to pass through your data’s center of gravity, is “on target” on average, and is the most precise tool of its kind. That’s the power of OLS.

What do you think? In your own words, how would you explain the difference between an estimator being “unbiased” and being “best” to a friend? And what do you think might happen to our trust in OLS if one of the Gauss-Markov assumptions (like “no heteroskedasticity,” meaning the error variance is constant) is violated?

References

1. https://stats.libretexts.org/Bookshelves/Introductory_Statistics/Book%3A_Introductory_Statistics_(OpenStax)/12%3A_Linear_Regression_and_Correlation/12.05%3A_Properties_of_Linear_Regression
2. https://www.econometrics-with-r.org/4-2-properties-of-the-ols-estimator.html
3. https://ocw.mit.edu/courses/14-382-econometrics-spring-2017/resources/mit14_382s17_lec3/
4. https://www.rbi.org.in/Scripts/PublicationsView.aspx?id=19999