Testing for Normality of Errors in the Classical Normal Linear Regression Model

Imagine you’re building a house. You’ve got the blueprints (your theory), you’ve laid the foundation (collected your data), and you’ve put up the walls (estimated your regression coefficients). Everything looks good. But before you move in, you need to check the plumbing and electricity. In econometrics, one of these crucial “utility” checks is testing the assumptions of your model. Today, we’re looking at one of the most famous assumptions of the Classical Normal Linear Regression Model (CNLRM): the idea that the “errors” or “residuals” follow a normal distribution.

But why do we even care? Does it mean our model is “wrong” if they aren’t normal? And how do we even check? We’ll explore this by looking at a simple visual tool, the P-P plot, and a formal statistical detective, the Jarque-Bera test.

The “normality assumption”: why do we care? 🔗

When we move from the Classical Linear Regression Model (CLRM) to the Classical Normal Linear Regression Model (CNLRM), we add one powerful assumption: that the unobserved error term ($u_i$) is normally distributed. This means we assume all those little misses-the differences between our model’s predictions and the actual data points-are random, centered around zero, and follow a perfect bell curve.

A common misconception is that this assumption is needed to make our OLS estimators “good.” In truth, the Gauss-Markov theorem already proved that our estimators are BLUE (Best Linear Unbiased Estimators) even without normality. Our coefficients are still unbiased and efficient.

So, why add the assumption? The answer has everything to do with hypothesis testing.

The crucial role of normality in hypothesis testing 🔗

The entire point of building a model is often to test hypotheses. Is this coefficient statistically significant? Does this variable *actually* have an impact on our outcome? To answer these questions, we use t-tests (for individual coefficients) and F-tests (for the overall significance of the model).

Here’s the catch: for those t-statistics and F-statistics to follow their respective t and F distributions, we need to assume the errors are normal. This is especially true in small samples. If we have a small sample and our errors are not normal, the p-values and confidence intervals our software spits out might be completely unreliable. We might conclude a variable is significant when it isn’t, or vice-versa.

Think of it like this: the t-test is a specific tool designed to work on a specific type of material (a normally distributed error). If you use it on a different material (a non-normal error), the tool might break or give you a faulty result.

What about large samples? The Central Limit Theorem to the rescue 🔗

This is where things get interesting. The strict requirement for normal errors becomes less critical as our sample size (n) gets larger. Thanks to the Central Limit Theorem (CLT), as ‘n’ increases, the distribution of our OLS estimators will *tend* toward normality, even if the underlying errors ($u_i$) are not normal.

In modern research, where datasets can have thousands or even millions of observations, many practitioners are less concerned about normality. However, testing for it is still a vital diagnostic step. It can reveal other problems in your model, like hidden outliers or an incorrect functional form. While the Jarque-Bera test is designed for large samples, it’s a good habit to always check your residuals. Other tests, like the Shapiro-Wilk test, are often preferred for smaller samples.

Our first diagnostic: a visual check with P-P plots 🔗

Before we jump to complex formulas, the best first step is always to just *look* at the residuals. You can plot them in a histogram, but a much more powerful tool is the Normal Probability-Probability (P-P) plot. (A very similar cousin is the Q-Q plot, or Quantile-Quantile plot, which plots quantiles instead of probabilities. Both are used for the same purpose.)

A P-P plot compares the empirical cumulative distribution function (what your residuals *actually* look like) against the theoretical cumulative distribution function (what a perfect normal distribution looks like).

How to read a P-P plot 🔗

Interpreting a P-P plot is beautifully simple. The plot includes a 45-degree diagonal line. This line represents perfect normality-it’s where the actual probabilities would lie if they perfectly matched the theoretical probabilities.

• If the points hug the 45-degree line: This is good news! Your residuals are behaving as expected and appear to be normally distributed.
• If the points deviate from the line in a pattern: This signals non-normality.

As statistical software guides like Minitab explain, specific patterns can even tell you *how* your distribution is non-normal. An “S” shape, for instance, suggests your distribution has “heavy tails” (more prone to extreme outliers) than a normal distribution. A bow- or banana-shaped curve suggests your data is skewed.

The P-P plot is a fantastic starting point, but it’s subjective. What one person sees as a “slight” deviation, another might see as a major problem. That’s why we need a formal test.

The formal detective: the Jarque-Bera (JB) test 🔗

The Jarque-Bera (JB) test is a “goodness-of-fit” test that gives us a formal, objective number to assess normality. It was developed by economists Carlos Jarque and Anil K. Bera in 1980. The entire test is built on two key concepts that describe the *shape* of a distribution: skewness and kurtosis.

The building blocks: skewness and kurtosis 🔗

A normal distribution (our “ideal” bell curve) has two defining shape characteristics:

1. Skewness (S): This measures the asymmetry of the distribution. A normal distribution is perfectly symmetric, so its skewness is 0.
   • Positive Skew (S > 0): The distribution has a long tail to the right.
   • Negative Skew (S < 0): The distribution has a long tail to the left.
2. Kurtosis (K): This measures the “peakedness” and “tailedness” of the distribution. A normal distribution, by definition, has a kurtosis of 3.
   • Leptokurtic (K > 3): The distribution is more peaked in the center and has “fatter tails” (more outliers) than a normal distribution.
   • Platykurtic (K < 3): The distribution is flatter and has “thinner tails” (fewer outliers) than a normal distribution.

The Jarque-Bera test is essentially a joint test that asks: “Are the skewness of our residuals close to 0 *and* is the kurtosis close to 3?”

Calculating the JB statistic 🔗

The test starts with a clear set of hypotheses:

• H₀ (Null Hypothesis): The residuals are normally distributed (i.e., S = 0 and K = 3).
• H₁ (Alternative Hypothesis): The residuals are not normally distributed.

The test then combines skewness (S) and kurtosis (K) from our sample data ($n$ observations) into a single test statistic, using the following formula:

$$ JB = \frac{n}{6} \left( S^2 + \frac{(K-3)^2}{4} \right) $$

Let’s break this down. The `(K-3)` term is often called “excess kurtosis”-it’s just a measure of how much our kurtosis differs from the normal 3. Notice that if our residuals are perfectly normal (S=0 and K=3), the entire expression in the brackets becomes zero, and the JB statistic is 0. The *further* our skewness is from 0 or our kurtosis is from 3, the larger and larger the JB statistic becomes.

Interpreting the results: what does the number mean? 🔗

You’ll never have to calculate this by hand. Your statistical software (like EViews, Stata, R, or Python) will give you the JB statistic and, most importantly, a p-value.

The chi-squared distribution 🔗

How does the software get that p-value? Under the null hypothesis (that the errors *are* normal), the JB statistic asymptotically follows a chi-squared ($\chi²$) distribution with 2 degrees of freedom. The “2” comes from the two parameters we are testing: skewness and kurtosis.

Your software calculates your JB statistic (e.g., 25.4) and then looks at the $\chi²$ distribution to see how likely it would be to get a number that large or larger purely by chance, *if* the null hypothesis were true. This probability is your p-value.

The simple rule: just look at the p-value 🔗

This is the only part you truly need to remember for practical purposes. Let’s use a standard significance level of 5% (or $\alpha = 0.05$).

• If p-value > 0.05: This is a “high” p-value. It means a large JB statistic like the one we got is reasonably likely to occur by chance. Therefore, we fail to reject the null hypothesis. We don’t have strong evidence to say our errors are non-normal. We can cautiously proceed.
• If p-value < 0.05: This is a “low” p-value. It’s telling us it’s *highly unlikely* we’d get a JB statistic this large if the errors were truly normal. Therefore, we reject the null hypothesis. We have significant statistical evidence to conclude that our residuals are *not* normally distributed.

What to do if you reject normality? 🔗

If your p-value is low, don’t panic! It doesn’t mean your entire model is useless. It’s a diagnostic, a call to investigate further.

1. Check your P-P plot: Is the non-normality caused by one or two extreme outliers? If so, are those outliers data entry errors, or are they legitimate (but influential) data points?
2. Check for omitted variables: Sometimes, non-normality is a symptom of a deeper problem, like a missing variable or an incorrect functional form (e.g., you modeled a relationship as linear when it’s really quadratic).
3. Consider transformations: A common fix for a skewed variable is to use a log transformation (e.g., using `log(Y)` instead of `Y` as your dependent variable).
4. Cite the Central Limit Theorem: If your sample is very large, you can note the non-normality, report the JB test, and argue that thanks to the CLT, your t-tests and F-tests remain asymptotically valid.

Checking for normality is a fundamental part of being a good econometrician. It’s about understanding the limits of your tools and ensuring that when you make a claim (like “this variable is significant”), you’re standing on a solid statistical foundation.

What do you think? Have you ever encountered a model where the Jarque-Bera test failed dramatically? What did the P-P plot look like, and what steps did you take to address it?

References

1. https://pubmed.ncbi.nlm.nih.gov/29258908/
2. https://www.geeksforgeeks.org/r-machine-learning/how-to-conduct-a-jarque-bera-test-in-r/
3. https://support.minitab.com/en-us/minitab/help-and-how-to/statistical-modeling/regression/how-to/fit-regression-model/interpret-the-results/all-statistics-and-graphs/residual-plots/
4. https://www.mystatisticsmentor.com/understanding-the-jarque-bera-test-history-theory-and-real-life-applications/
5. https://en.wikipedia.org/wiki/Jarque%E2%80%93_Bera_test