From ARMA to ARIMA: Modelling Non-Stationary Time Series

We all have a desire to predict the future, especially when it comes to money. Will inflation go up? Where is the stock market headed? What will our business sales be next quarter? Economists and data scientists build models to answer these questions, but they often run into a huge problem: most real-world data is wild, trend-following, and unpredictable. It doesn’t like to stay still. This unruliness is what we call “non-stationarity,” and it’s the single biggest challenge in time series forecasting. Fortunately, we have a powerful tool to tame this wild data: the ARIMA model.

This model is a cornerstone of modern forecasting, and its magic lies in one simple idea: if the data is too unruly to model directly, let’s first make it calm, and then build a model on its calmer, more predictable version. It’s all about transforming chaos into order.

What does it mean for data to be ‘stationary’? 🔗

Before we can understand the ‘Integrated’ part of ARIMA, we must first understand its opposite: stationarity. A time series is stationary if its statistical properties-specifically its mean, variance, and autocorrelation-are all constant over time. In simple terms, the data behaves consistently. If you plot it, it will look like it’s hovering around a single average line, and the “width” or “spread” of its wiggles will be roughly the same throughout. Think of a calm, steady river flowing flat. No matter which section of the river you look at, its average level and choppiness are the same.

Most of our simple statistical models *require* data to be stationary. They are built on the assumption that the data’s “behavior” doesn’t change, making it possible to predict its future behavior. Trying to model non-stationary data with these tools is like trying to build a house on quicksand.

Why most economic data isn’t stationary 🔗

Now, think about most real-world economic data. Does it look like a calm, flat river? Absolutely not. Consider India’s GDP over the last 30 years; it has a clear, strong, upward trend. Its mean (average level) is constantly increasing. This is a classic example of non-stationarity.

Or, think about a company’s stock price. It might not have a clear trend, but it wanders around in what’s called a “random walk.” Its mean isn’t constant; where it will be tomorrow depends heavily on where it is today. This is also non-stationary. Most financial and economic data-like inflation, exchange rates, or sales figures-have trends, cycles, or other properties that change over time. They are non-stationary and, in their raw form, un-modellable.

The ‘I’ in ARIMA: Taming wild data with differencing 🔗

So, what do we do? We can’t just give up. This is where the ‘I’-which stands for Integrated-comes in. The technique we use is called differencing. It’s a surprisingly simple and brilliant transformation.

Instead of looking at the *level* of the data, we look at the *change* from one period to the next. The most common form is first-order differencing. We create a new time series by subtracting yesterday’s value from today’s value.

The formula is: $ \Delta Y_t = Y_t – Y_{t-1} $

Let’s use an analogy. Imagine you’re tracking your total savings over time. The plot of your *total savings* will (hopefully) always trend upwards. This is non-stationary. What if, instead, you plotted the *amount you saved each month*? This new series-the *change* in your savings-would likely be much more stable. It might hover around an average of, say, ₹5,000, sometimes a bit more, sometimes a bit less. This new series of “monthly savings” is very likely to be stationary!

By differencing the data, we’ve removed the trend and created a new, stationary series that we *can* model.

Understanding ‘integrated processes’ or I(d) 🔗

A time series that becomes stationary after being differenced ‘d’ times is called an “integrated process of order d,” denoted as I(d).

• I(1): This means the original data was non-stationary, but its first difference ($ \Delta Y_t $) is stationary. This is extremely common for data with a trend, like GDP or price levels.
• I(2): This is less common, but it means the data has an “accelerating” trend. The first difference is still non-stationary, but the *second* difference (the *change* in the *change*) is stationary.
• I(0): This simply means the data was already stationary in the first place, and no differencing was needed (d=0).

The ‘d’ in the ARIMA model is exactly this: it’s the order of integration, or simply, the number of times we need to difference the data to make it stationary.

Building the full model: Introducing ARIMA(p,d,q) 🔗

Now that we have the ‘I(d)’ part, we can bring in the ‘AR’ and ‘MA’ components. These are the parts that actually do the modelling and forecasting, but they are applied *after* the data has been made stationary by differencing.

The full model is written as ARIMA(p, d, q).

• p: The AutoRegressive (AR) part.
• d: The Integrated (I) part.
• q: The Moving Average (MA) part.

Let’s briefly look at ‘p’ and ‘q’.

The ‘AR’ (p): The autoregressive part 🔗

This component assumes that the current value of the (differenced) series can be explained as a function of its past values. The ‘p’ tells us *how many* past values (or “lags”) to use. For example, an AR(1) model, or p=1, means that the value of the series today is influenced by its value yesterday. It’s like saying, “Today’s ‘change in inflation’ is likely to be similar to yesterday’s ‘change in inflation’.”

The ‘MA’ (q): The moving average part 🔗

This component is a bit more abstract. It assumes that the current value is related to the “forecast errors” from past periods. A forecast error is the “shock” or “surprise”-the difference between our last forecast and what actually happened. An MA(1) model, or q=1, means the value today is influenced by the forecast error we made yesterday. It’s a way of modelling unexpected shocks that have a lingering effect.

Putting it all together: p, d, and q 🔗

The ARIMA(p,d,q) model combines all three pieces into one powerful forecasting machine. The process works like this:

1. Check for stationarity. If the data (like India’s Consumer Price Index) is trending, it’s non-stationary.
2. Find ‘d’. We difference the data until the new, differenced series *is* stationary. If we difference it once, then `d=1`.
3. Find ‘p’ and ‘q’. Now, we look at this new stationary series (e.g., the *monthly change* in the CPI) and use statistical tools to find the best ‘p’ and ‘q’ to model its behavior.

An ARIMA(1,1,1) model, for example, is a very common model for economic data. It means we take the first difference (`d=1`), and then model that differenced series using one autoregressive term (`p=1`) and one moving average term (`q=1`).

Why ARIMA is the ‘Swiss Army knife’ of time series 🔗

The final part of the puzzle is understanding that ARIMA isn’t just one model; it’s an entire *family* of models. This is what the prompt means by a “unifying model.” The (p,d,q) parameters allow it to take the shape of many simpler models, making it incredibly flexible.

This flexibility is why it’s a cornerstone of the famous Box-Jenkins methodology, a systematic process for identifying, estimating, and checking time series models.

`[Image: A simple flowchart of the Box-Jenkins model: 1. Identification (Check stationarity, find p,d,q using ACF/PACF plots) -> 2. Estimation (Fit the model to the data) -> 3. Diagnostic Checking (Check if model residuals are random and white noise) -> 4. Forecast (If model is good).]`

The special cases hidden inside ARIMA 🔗

By setting the parameters to zero, we can see how ARIMA “contains” these other models:

• ARIMA(p, 0, 0) = AR(p) model: If the data is already stationary (`d=0`) and has no moving average component (`q=0`), the model is just a pure autoregressive model.
• ARIMA(0, 0, q) = MA(q) model: If the data is stationary (`d=0`) and has no autoregressive component (`p=0`), it’s a pure moving average model.
• ARIMA(p, 0, q) = ARMA(p,q) model: This is the model for stationary data that has *both* AR and MA components. The ARIMA framework is a direct generalization of ARMA for non-stationary data.
• ARIMA(0, 1, 0) = Random Walk: A simple random walk (where today’s price is just yesterday’s price plus a random shock) is just an ARIMA(0,1,0). Its first difference is just the random shock, which has no AR or MA terms.

This unifying nature is what makes ARIMA so powerful and widely used. It gives analysts a single, comprehensive framework to handle a vast range of different time series behaviors, from simple stationary data to complex, trending economic indicators. By first taming the data with differencing, it unlocks our ability to apply proven AR and MA modelling techniques, turning chaotic, untamed data into valuable, actionable forecasts.

What do you think? If you were to plot the number of people visiting your favorite local restaurant each day for a year, would you expect that data to be stationary or non-stationary? What kind of trends or patterns might you need to “difference out” to model it?

References

1. https://www.ibef.org/economy/indian-economy-overview
2. https://www.rbi.org.in/Scripts/StatisticsViews.aspx?Id=181
3. https://online.stat.psu.edu/stat510/lesson/4
4. https://www.investopedia.com/terms/a/autoregressive-integrated-moving-average-arima.asp