A Guide to Matrix Multiplication and Its Properties

Imagine you’re an economist trying to model the entire economy of a small country. You have dozens of industries, and each industry’s output (like steel) is an input for another industry (like car manufacturing), which in turn sells to consumers, who then supply labor back to the steel and car industries. How on earth do you track all these interconnected relationships at once? You can’t just use a simple one-variable equation. This is where matrices come in. They are powerful tools for organizing and solving large, complex systems of equations. But to unlock their power, we need to know how they “talk” to each other. The most important, and most nuanced, operation is matrix multiplication. It’s not as simple as multiplying numbers side-by-side. It’s a precise “row-on-column” dance that has its own fascinating rules and properties. Let’s dive into the mechanics of how it works and why it’s the engine behind so much of econometrics and data analysis.

The mechanics of matrix multiplication 🔗

The very first thing to learn about matrix multiplication is that it’s picky. You can’t just multiply any two matrices together. They have to be “conformable,” which is a fancy way of saying they have to fit together in a specific way.

The conformability condition: a rule of “inner dimensions” 🔗

Let’s say you have two matrices, A and B, and you want to find the product AB.

• First, look at the dimensions of A. Let’s say it has $m$ rows and $n$ columns (we write this as $m \times n$).
• Next, look at the dimensions of B. Let’s say it has $p$ rows and $q$ columns (written as $p \times q$).

Here is the one and only rule for conformability: The number of columns in A ($n$) must be exactly equal to the number of rows in B ($p$). If $n = p$, you can multiply them. If $n \neq p$, the operation is undefined. It’s impossible.

Think of it as a matchmaking process. Each of the $n$ elements in a row of A needs a partner from the $p$ elements in a column of B. If the numbers don’t match, someone is left without a partner, and the calculation can’t happen.

When they *are* conformable (when $n = p$), the resulting new matrix, let’s call it C, will have dimensions that come from the “outer” numbers: it will have $m$ rows and $q$ columns (an $m \times q$ matrix).

How each element is calculated: the “sum of products” 🔗

So, you’ve confirmed your matrices are conformable. How do you actually *do* the multiplication? The rule is a “row-on-column” operation, also known as a dot product.

To find the single element in the $i$-th row and $j$-th column of the new matrix C (we’ll call this element $c_{ij}$), you need to:

1. Select the $i$-th row from matrix A.
2. Select the $j$-th column from matrix B.
3. Multiply the first element of that row by the first element of that column.
4. Multiply the second element of that row by the second element of that column.
5. …and so on, all the way down the line.
6. Finally, add all of those products together. That sum is your new element, $c_{ij}$.

Let’s try a simple 2×2 example.

Let $\mathbf{A} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$ and $\mathbf{B} = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix}$

We want to find $\mathbf{C} = \mathbf{AB}$. The new matrix $\mathbf{C}$ will also be 2×2. Let’s find the element in the first row, first column ($c_{11}$):

• Take the first row of A: (1, 2)
• Take the first column of B: (5, 7)
• Calculate the sum of products: $(1 \times 5) + (2 \times 7) = 5 + 14 = 19$.
• So, $c_{11} = 19$.

Now for the element in the first row, second column ($c_{12}$):

• Take the first row of A: (1, 2)
• Take the second column of B: (6, 8)
• Calculate: $(1 \times 6) + (2 \times 8) = 6 + 16 = 22$.
• So, $c_{12} = 22$.

If we continue this, we find $c_{21} = (3 \times 5) + (4 \times 7) = 15 + 28 = 43$, and $c_{22} = (3 \times 6) + (4 \times 8) = 18 + 32 = 50$.

Our final product matrix is $\mathbf{C} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}$.

This row-on-column operation is the workhorse of econometrics. It’s how you apply a set of coefficients (one matrix) to a set of data variables (another matrix) to get your predicted values (the product matrix).

[Image: Diagram showing the first row of matrix A and first column of matrix B being combined with multiplication and addition to create the first element of the product matrix C]

Understanding the transpose of a product 🔗

First, what’s a “transpose”? The transpose of a matrix (written as $\mathbf{A}^T$ or $\mathbf{A}’$) is what you get when you simply “flip” the matrix along its main diagonal. The rows become columns, and the columns become rows. So, an $m \times n$ matrix becomes an $n \times m$ matrix.

Now, what happens if you multiply two matrices (A and B) and *then* take the transpose of the result? You get $(\mathbf{AB})^T$. How does this relate to the transposes of A and B individually?

Your first guess might be that $(\mathbf{AB})^T = \mathbf{A}^T \mathbf{B}^T$. This seems logical, but it’s completely wrong. In fact, it’s often impossible to even calculate $\mathbf{A}^T \mathbf{B}^T$ due to the conformability rule!

The correct and critically important property is this: $(\mathbf{AB})^T = \mathbf{B}^T \mathbf{A}^T$

Notice that the order is reversed. This is famously known as the “socks and shoes rule.”

Think about it: to get ready in the morning, you first put on your socks (operation A), then you put on your shoes (operation B). The final result is $(\mathbf{AB})$. To undo this process (which is what the transpose is like, in a sense), you can’t take your socks off first. You *must* reverse the order: first, take off your shoes ($\mathbf{B}^T$), and *then* take off your socks ($\mathbf{A}^T$).

This rule is not just a quirky math trick. It is fundamental to manipulating matrix equations in statistical proofs, especially when simplifying the complex formulas you find in Ordinary Least Squares (OLS) regression.

Special ops: two powerful types of matrices 🔗

Just like in regular arithmetic we have special numbers like 0 (the additive identity) and 1 (the multiplicative identity), in matrix algebra, we have special *types* of matrices that have powerful properties. Let’s look at two of them.

Orthogonal matrices: the “rotators” 🔗

An orthogonal matrix is a square matrix that has a very special and tidy relationship with its own transpose. Let’s call our matrix A (we often use Q or P for orthogonal matrices, but let’s stick with A).

A is orthogonal if its transpose is also its inverse. This means that if you multiply A by its own transpose ($\mathbf{A}’$), you get the Identity Matrix (I).

The formula is: $\mathbf{A}’\mathbf{A} = \mathbf{A}\mathbf{A}’ = \mathbf{I}$

The Identity Matrix, I, is the matrix equivalent of the number 1. It’s a square matrix with 1s all down the main diagonal and 0s everywhere else. Multiplying any matrix by I (where possible) just gives you the original matrix back, unchanged.

So, why is this special? The term “orthogonal” gives a clue. The columns (and rows) of an orthogonal matrix are all “orthonormal.” This means two things:

1. Ortho- (Orthogonal): Every column vector is perpendicular (at a 90-degree angle) to every other column vector.
2. -Normal (Normalized): Every column vector has a “length” or “magnitude” of 1.

In geometry and data science, multiplying your data by an orthogonal matrix is equivalent to rotating your data in space. It doesn’t stretch, shrink, or warp the data in any way-it just changes its orientation. This property of preserving distances and angles is incredibly useful in advanced econometric techniques like Principal Component Analysis (PCA), where the goal is to rotate a complex dataset to find the most informative “view” of it.

Nilpotent matrices: the “vanishing” matrices 🔗

This one has a more dramatic-sounding name. A nilpotent matrix (let’s call it B) is a square matrix that, when you multiply it by itself enough times, eventually becomes the zero matrix (a matrix filled with nothing but zeros).

The “index” of nilpotency, $k$, is the smallest positive integer power you need to raise B to in order to make it vanish.

The formula is: $\mathbf{B}^k = \mathbf{0}$ for some integer $k \ge 1$.

A simple example is the matrix $\mathbf{B} = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}$.

If we calculate $\mathbf{B}^2$ (which is $\mathbf{B} \times \mathbf{B}$):

$\mathbf{B}^2 = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} (0 \times 0 + 1 \times 0) & (0 \times 1 + 1 \times 0) \\ (0 \times 0 + 0 \times 0) & (0 \times 1 + 0 \times 0) \end{bmatrix} = \begin{bmatrix} 0 & 0 \\ 0 & 0 \end{bmatrix}$

Since $\mathbf{B}^2 = \mathbf{0}$, this matrix B is nilpotent with an index of 2. These “vanishing” matrices are very important in abstract algebra and also appear in economic models that describe systems where an initial shock or effect (like a one-time government stimulus) eventually “dies out” or has no further effect on the system after a certain number of periods.

Essential theorems of matrix multiplication 🔗

Finally, we need to cover some of the unbreakable laws of matrix multiplication. These are theorems that form the bedrock of linear algebra, and they include a few very surprising “gotchas” for beginners.

The big one: multiplication is not commutative 🔗

In regular arithmetic, we know that $5 \times 3$ is the exact same as $3 \times 5$. The order doesn’t matter. This is called the commutative property.

In matrix algebra, this property does not hold.

In general, for two matrices A and B: $\mathbf{AB} \neq \mathbf{BA}$

This is arguably the most important and non-intuitive rule in matrix algebra. There are two reasons why this is true:

1. The Conformability Failure: Let’s say A is a $2 \times 3$ matrix and B is a $3 \times 4$ matrix. You *can* calculate AB (the inner dimensions $3$ and $3$ match), and the result will be a $2 \times 4$ matrix. But what about BA? You would be trying to multiply a $3 \times 4$ matrix by a $2 \times 3$ matrix. The inner dimensions are $4$ and $2$. They don’t match! The operation BA is completely undefined.
2. The Calculation Difference: But what if they are both square, like our 2×2 example from before? The multiplication is defined in both directions. Let’s check.

We already calculated $\mathbf{AB} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}$.

Now let’s try $\mathbf{BA} = \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}$

• $c_{11} = (5 \times 1) + (6 \times 3) = 5 + 18 = 23$
• $c_{12} = (5 \times 2) + (6 \times 4) = 10 + 24 = 34$

We don’t even need to go further. The new matrix starts with 23, while AB started with 19. They are clearly not the same. This non-commutative property is not just a bug; it’s a feature. It reflects the fact that in the real world, the order of operations matters. (For example, “loading the data” then “running the regression” is not the same as “running the regression” then “loading the data”!).

Two critical “zero” theorems 🔗

Here are two more fundamental results that are essential for proofs in econometrics.

Theorem 1: $\mathbf{A}’\mathbf{A} = \mathbf{0}$ if and only if $\mathbf{A} = \mathbf{0}$

This sounds a bit abstract, but let’s break it down. The product $\mathbf{A}’\mathbf{A}$ is very special. If you think about the row-on-column multiplication, the element on the diagonal of $\mathbf{A}’\mathbf{A}$ (say, the element in the $j$-th row and $j$-th column) is calculated by taking the $j$-th row of $\mathbf{A}’$ (which is just the $j$-th *column* of A) and multiplying it by the $j$-th column of A.

This means you are multiplying each element in a column by *itself* and then adding them up. You are getting a sum of squares. For example, the first diagonal element is $a_{11}^2 + a_{21}^2 + … + a_{m1}^2$.

The only way for a sum of squared real numbers to equal zero is if *every single one of those numbers is zero*. Therefore, if the entire matrix $\mathbf{A}’\mathbf{A}$ is the zero matrix, it logically forces every single element in A to be zero. The “if and only if” (iff) means it works both ways. This theorem is the rock-solid foundation that allows econometricians to prove that a set of coefficients is, in fact, zero.

Theorem 2: $\mathbf{AB} = \mathbf{0}$ does *not* imply $\mathbf{A} = \mathbf{0}$ or $\mathbf{B} = \mathbf{0}$

This is our final “gotcha.” In regular arithmetic, if $a \times b = 0$, you know for a fact that either $a=0$ or $b=0$.

Not so with matrices. You can have two non-zero matrices that, when multiplied, completely annihilate each other and produce the zero matrix. Our nilpotent matrix B from earlier is a perfect example. B was not the zero matrix, but $\mathbf{B} \times \mathbf{B} = \mathbf{0}$. These matrices are called “divisors of zero,” and they are a direct consequence of the complex row-on-column multiplication process.

What do you think?

What real-world example (besides socks and shoes!) can you think of where the order of operations is critical, just like $\mathbf{AB} \neq \mathbf{BA}$? And now that you see the mechanics, how do you think matrix multiplication could be used to represent something like a national “supply chain” in economics, where one industry’s outputs become the next industry’s inputs?

References

1. https://www.mathsisfun.com/algebra/matrix-multiplying.html
2. https://online.stat.psu.edu/stat462/node/107/
3. https://nptel.ac.in/courses/111104137
4. https://math.libretexts.org/Bookshelves/Linear_Algebra/A_First_Course_in_Linear_Algebra_(Kuttler)/02%3A_Algebras/2.01%3A_Matrix_Operations