Understanding the Linear Probability Model (LPM)

Imagine you’re a bank’s loan officer. Your entire job boils down to a single, critical decision for each applicant: “yes” or “no.” Approve the loan, or deny it. This is a binary choice. Now, imagine you’re an economist trying to build a model to help that officer. You have data on applicants: their income, their credit history, their age. How can you use this data to predict that “yes/no” outcome? Your first instinct might be to reach for the most familiar tool in the box: the standard linear regression. What happens when you do? You get the Linear Probability Model (LPM), one of the simplest, most intuitive, and most fundamentally flawed models in econometrics.

The Linear Probability Model is a classic case of using a simple tool for a complicated job. It’s appealing because it’s easy to understand, but it comes with serious side effects. Let’s walk through what it is, where it goes wrong, and how to make sense of it.

Specifying the Linear Probability Model 🔗

At its heart, the LPM is just a regular Ordinary Least Squares (OLS) regression. The only “trick” is that the dependent variable-the thing you’re trying to predict (our ‘Y’)-is a binary or “dummy” variable. It can only take on two values, typically 1 or 0.

• Y = 1 if an event happens (e.g., the person owns a house, the loan is approved, the patient recovers).
• Y = 0 if the event does not happen (e.g., the person rents, the loan is denied, the patient does not recover).

An example using house ownership 🔗

Let’s stick with the topic summary’s example: house ownership. We want to know how a person’s annual income (we’ll call it ‘X’) affects the likelihood that they own a house. Our binary dependent variable, ‘Y’, is:

Y = 1 if the person owns a house
Y = 0 if the person does not own a house

The Linear Probability Model for this relationship is a simple regression equation:

Y = β₀ + β₁X + ε

Here, β₀ is the intercept, β₁ is the slope coefficient for income, and ε is the error term. If we run this regression using OLS, we get our estimated line:

Ŷ = β̂₀ + β̂₁X

Where Ŷ (Y-hat) is the predicted value of Y. But what does this predicted value actually *mean* when Y can only be 0 or 1?

How the conditional expectation is interpreted as a probability 🔗

This is the central, clever insight of the LPM. In statistics, the “expected value” (or mean) of a 0/1 variable is simply the probability that the variable equals 1. Think about it: if you flip a coin 10 times (where 1=Heads, 0=Tails) and get 7 heads, the mean of your sample is (7*1 + 3*0) / 10 = 0.7. This mean, 0.7, is the observed probability of getting heads.

Formally, the expected value of Y, given a certain level of income (X), is:

E(Y|X) = [ 1 * P(Y=1 | X) ] + [ 0 * P(Y=0 | X) ]

This simplifies to:

E(Y|X) = P(Y=1 | X)

This is the magic trick. The conditional mean of our binary variable is the conditional probability that the event occurs. Since our OLS regression line (Ŷ = β̂₀ + β̂₁X) is our best estimate of the conditional mean (E(Y|X)), our predicted value, Ŷ, is actually an estimate of the probability that Y=1.

So, our OLS regression equation:

Ŷ = β̂₀ + β̂₁X

…is re-interpreted as:

Predicted P(Y=1 | X) = β̂₀ + β̂₁X

This is why it’s called the Linear Probability Model. We are assuming that the probability of owning a house is a linear function of income.

The key limitations of LPM 🔗

This is where our simple, intuitive model starts to fall apart. By forcing a linear model onto a binary outcome, we violate several key assumptions of OLS regression, leading to three major problems.

Problem 1: Predicted probabilities can be outside the 0-1 range 🔗

This is the most obvious and nonsensical flaw. A probability, by definition, must lie between 0 and 1 (or 0% and 100%). You can’t have a -20% chance of rain or a 130% chance of owning a home. But the LPM, being just a straight line, has no boundaries.

Imagine our estimated house ownership model is:

P(OwnsHouse=1) = -0.10 + 0.00002 * (Income)

What happens if we plug in a very low income, say $3,000 a year?

P(OwnsHouse=1) = -0.10 + 0.00002 * (3000) = -0.10 + 0.06 = -0.04

The model predicts a -4% probability of owning a house. This is meaningless.

Now, what about a very high income, say $6,000,000 a year?

P(OwnsHouse=1) = -0.10 + 0.00002 * (6000000) = -0.10 + 1.20 = 1.10

The model predicts a 110% probability of owning a house. This is equally meaningless. This failure isn’t just a small quirk; it’s a fundamental sign that the model is misspecified. The relationship between income and home ownership probably isn’t a straight line. It likely flattens out, approaching 0% at the low end and 100% at the high end, forming an ‘S’ shape (a sigmoid curve), which models like Logit and Probit are designed to capture.

[Image: Graph showing X-Y axes, with a 0 and 1 line on the Y-axis. A straight regression line cuts through the 0 line for low X-values and goes above the 1 line for high X-values.]

Problem 2: The non-normality of the error term 🔗

One of the core assumptions of OLS (required for valid hypothesis testing in small samples) is that the error term, ε, is normally distributed (a “bell curve”). In the LPM, this assumption is impossible to meet.

The error term is the difference between the actual outcome (Y) and the predicted probability (Ŷ, which is E(Y|X)). Since the actual outcome Y can only be 0 or 1, the error term can only take on two possible values for any given X:

1. If the person does own a house (Y=1):
   ε = Y - E(Y|X) = 1 - P(Y=1|X)
2. If the person does not own a house (Y=0):
   ε = Y - E(Y|X) = 0 - P(Y=1|X) = -P(Y=1|X)

The error term is not “bell-shaped”; it has a binary distribution. This means that for any specific income level, the errors aren’t normally distributed. While this is less of a problem in very large samples (thanks to the Central Limit Theorem), in smaller samples, it means our t-statistics and F-statistic (which we use to see if our variables are “significant”) don’t follow their assumed distributions. This makes our p-values and confidence intervals unreliable.

Problem 3: Inherent heteroscedasticity 🔗

This is the most serious statistical problem with the LPM. OLS assumes homoscedasticity, a fancy word meaning the variance (or “scatter”) of the error terms is constant for all levels of X. In the LPM, the variance of the error term is anything but constant.

As shown by many econometrics tutorials, the variance of the error term in an LPM is given by:

Var(ε|X) = P(Y=1|X) * [1 – P(Y=1|X)]

Let’s break this down. The variance of our error *depends* on the predicted probability, which itself *depends on X*. This is the definition of heteroscedasticity. The variance is not constant.

Look at the formula: the variance is maximized when the predicted probability is 0.5 (0.5 * (1-0.5) = 0.25) and gets smaller and smaller as the probability approaches 0 or 1. For example, if P(Y=1|X) = 0.1, the variance is 0.1 * 0.9 = 0.09. If P(Y=1|X) = 0.01, the variance is 0.01 * 0.99 = 0.0099.

This means our model is very uncertain (has high variance) about predictions near 50/50 but very certain (has low variance) about predictions near 0% or 100%. OLS doesn’t know this; it assumes the variance is the same everywhere. This causes two issues:

1. Inefficiency: Our coefficient estimates (β̂) are still unbiased, but they are no longer the “Best” (most efficient) estimators. We could get more precise estimates using a different method (like Weighted Least Squares) that accounts for this changing variance.
2. Incorrect Standard Errors: This is the big one. OLS calculates standard errors assuming homoscedasticity. Since that assumption is false, our standard errors are biased. This, again, makes all our hypothesis tests (t-tests, F-tests) and p-values completely untrustworthy. We can’t tell if our income variable *actually* has a statistically significant effect. (Note: This specific problem can be “patched” by using “heteroscedasticity-robust standard errors,” but this is a fix for a broken model, not a true solution).

Interpreting LPM results 🔗

After all those problems, why would anyone even *talk* about the LPM? The answer is its one, beautiful, redeeming quality: its coefficients are incredibly easy to interpret.

What the slope coefficient represents 🔗

In our model, P(OwnsHouse=1) = β̂₀ + β̂₁Income, the slope coefficient β̂₁ has a direct and intuitive meaning. It is the absolute change in the probability that Y=1 for a one-unit change in X.

Let’s use our number from before: β̂₁ = 0.00002.

The interpretation is: “A one-dollar increase in annual income is associated with a 0.00002 increase in the probability of owning a house.”

That’s a bit small, so we can scale it up: “A $10,000 increase in annual income (a 10,000-unit change in X) is associated with a 10,000 * 0.00002 = 0.20 increase in the probability of owning a house.”

More simply: “A $10,000 raise is associated with a 20 percentage point increase in the probability of owning a home.”

This direct, simple interpretation of the coefficient as a change in probability is the LPM’s main advantage. In more complex models like logit or probit, the coefficient represents the change in the “log-odds” or a “z-score,” which is not directly intuitive and must be converted into a marginal effect (a probability change) that often varies depending on the value of X.

Handling negative predicted probabilities 🔗

So what do you do when your model predicts a -4% chance of home ownership? The practical, if unsatisfying, answer is that researchers simply truncate the predictions. Any predicted probability less than 0 is treated as 0. Any predicted probability greater than 1 is treated as 1.

If the model predicts -4%, you would report that person’s probability of ownership as 0%. If it predicts 110%, you report it as 100%. This is a common-sense “fix,” but it’s important to remember that it’s a workaround. The fact that you have to “fix” your predictions at all is a clear signal that the model itself is a poor fit for the data.

While the LPM is deeply flawed and has been largely replaced by logit and probit models, it’s not entirely useless. Some researchers, as this brief from the U.S. Department of Health and Human Services notes, still find it useful for estimating average effects in specific contexts (like randomized trials), especially when the independent variables are also binary. Its simplicity makes it a valuable first step and a powerful teaching tool for understanding *why* more complex models are necessary.

What do you think? Given the serious flaws, do you think the LPM’s simple interpretation ever justifies its use in a final analysis? Have you ever encountered a “yes/no” question in your own life where you realized a simple linear trend just didn’t make sense?

References

1. https://stats.stackexchange.com/questions/91816/interpreting-coefficient-marginal-effect-from-linear-probability-model
2. https://opa.hhs.gov/sites/default/files/2020-07/lpm-tabrief.pdf