The inverse matrix

A matrix moves every point of the plane. Can another matrix move them all back? Watch the grid warp, then play back home.

Undo

In A matrix moves the plane, the matrix $\mathbf{A}$ above sent the point $(3,2)$ to $(4,5)$. The undo must send $(4,5)$ back to $(3,2)$, and do the same for every other point. A matrix that does this is the inverse of $\mathbf{A}$, written $\mathbf{A}^{-1}$:

Here $\mathbf{I}$ is the identity matrix of Matrix multiplication, the map that leaves every point where it is. Doing $\mathbf{A}$ and then $\mathbf{A}^{-1}$ leaves every point where it started. So does doing them in the other order.

For this matrix,

Check it on the point: $\mathbf{A}^{-1}(4,5)=\tfrac13\,(4+5,\ -4+10)=(3,2)$. That is also the answer to the system $2x-y=4$, $x+y=5$ of Two pictures of a linear system. Once you know $\mathbf{A}^{-1}$, the solution of $\mathbf{A}\mathbf{x}=\mathbf{b}$ is $\mathbf{x}=\mathbf{A}^{-1}\mathbf{b}$.

Notice what the inverse is not. It is not the matrix of reciprocals, $1/2$, $-1$, $1$ and $1$. The undo has to reverse the whole map, not each number on its own.

Finding it by elimination

Where does $\mathbf{A}^{-1}$ come from? Its first column is where $\mathbf{e}_1=(1,0)$ lands under the undo. That is the $\mathbf{x}$ that $\mathbf{A}$ sends to $\mathbf{e}_1$, so it solves $\mathbf{A}\mathbf{x}=\mathbf{e}_1$. The second column solves $\mathbf{A}\mathbf{x}=\mathbf{e}_2$.

So solve both systems at once. Write $\mathbf{A}$ with $\mathbf{I}$ beside it, and run the elimination of Elimination on the whole wide table. This time keep going: clear above each pivot as well as below, then divide each row by its pivot. When the left half has become $\mathbf{I}$, the right half is $\mathbf{A}^{-1}$. This is called Gauss–Jordan elimination.

The steps are: take $\tfrac12$ of row 1 from row 2; add $\tfrac23$ of row 2 to row 1; divide row 1 by 2 and row 2 by 1.5.

The 2 × 2 formula

For a $2\times2$ matrix there is a short cut. Swap the two numbers on the diagonal, change the sign of the other two, and divide everything by one number:

For our $\mathbf{A}$, the number is $2\cdot1-(-1)\cdot1=3$, and the formula gives the $\tfrac13\begin{bmatrix}1&1\\-1&2\end{bmatrix}$ above. Check one entry of $\mathbf{A}^{-1}\mathbf{A}$: row 1 of $\mathbf{A}^{-1}$ times column 1 of $\mathbf{A}$ is $\tfrac13\cdot2+\tfrac13\cdot1=1$. Row 1 times column 2 is $\tfrac13\cdot(-1)+\tfrac13\cdot1=0$.

The number $a_{11}a_{22}-a_{12}a_{21}$ is the determinant of the matrix, written $\det\mathbf{A}$. Here it is 3, which is the area of the parallelogram the unit square became in the first picture. The page The determinant as area explains why.

When there is no undo

Now drag a column, after the clip, towards the line of the other column. The grid squeezes, and when the two columns line up it is flat: the whole plane has landed on one line.

Take $\begin{bmatrix}1&2\\2&4\end{bmatrix}$, whose columns $(1,2)$ and $(2,4)$ lie on one line. It sends $(2,-1)$ to $2\,(1,2)-1\,(2,4)=(0,0)$. It sends $(0,0)$ to $(0,0)$ as well. Two different points land on the same spot, and no undo can send that spot back to both of them. So this matrix has no inverse. The formula agrees: its determinant is $1\cdot4-2\cdot2=0$, and the formula would divide by zero.

The same reasoning works for any square matrix. If $\mathbf{A}\mathbf{x}=\mathbf{0}$ for some $\mathbf{x}$ that is not $\mathbf{0}$, then $\mathbf{x}$ and $\mathbf{0}$ land together, and $\mathbf{A}$ has no inverse. So not every square matrix has one. Those that do are called invertible.

Undo in reverse order

In the morning you put on socks, then shoes. To undo it, you take off the shoes first, then the socks. Matrices work the same way.

Take the turn and the shear of Matrix multiplication: $\mathbf{A}_\text{rot}=\begin{bmatrix}0&-1\\1&0\end{bmatrix}$ turns by 90°, which is $\pi/2$ radians, and $\mathbf{A}_\text{shear}=\begin{bmatrix}1&1\\0&1\end{bmatrix}$ shears. Turning first and then shearing is $\mathbf{A}_\text{shear}\mathbf{A}_\text{rot}=\begin{bmatrix}1&-1\\1&0\end{bmatrix}$, since the first map is written on the right. To undo it, undo the shear first, then the turn:

Here $\mathbf{A}_\text{rot}^{-1}$ turns back by 90° and $\mathbf{A}_\text{shear}^{-1}$ shears back. The other order, $\mathbf{A}_\text{shear}^{-1}\mathbf{A}_\text{rot}^{-1}=\begin{bmatrix}1&1\\-1&0\end{bmatrix}$, is a different matrix, and it does not undo $\mathbf{A}_\text{shear}\mathbf{A}_\text{rot}$. In general,

One more rule. The transpose $\mathbf{A}^\top$ of a matrix swaps its rows and columns, as Transposes and permutations explains. The inverse of the transpose is the transpose of the inverse: $(\mathbf{A}^\top)^{-1}=(\mathbf{A}^{-1})^\top$. For the turn, the inverse happens to be the transpose itself: $\mathbf{A}_\text{rot}^{-1}=\begin{bmatrix}0&1\\-1&0\end{bmatrix}=\mathbf{A}_\text{rot}^\top$.

Solve, don’t invert

The formula $\mathbf{x}=\mathbf{A}^{-1}\mathbf{b}$ is good for thinking, but computer programs rarely use it. Finding $\mathbf{A}^{-1}$ for an $n\times n$ matrix takes about $n^3$ multiply-and-subtract steps. Solving $\mathbf{A}\mathbf{x}=\mathbf{b}$ by elimination takes about $n^3/3$. Elimination is also less disturbed by rounding errors. So a program that needs $\mathbf{A}^{-1}\mathbf{b}$ solves the system instead, and keeps the steps of elimination for the next $\mathbf{b}$, as LU factorisation shows.

Worked example

1. The inverse. $\mathbf{A}=\begin{bmatrix}2&-1\\1&1\end{bmatrix}$ has $\det\mathbf{A}=3$ and $\mathbf{A}^{-1}=\tfrac13\begin{bmatrix}1&1\\-1&2\end{bmatrix}$.
2. Solving with it. $\mathbf{A}^{-1}(4,5)=(9/3,\ 6/3)=(3,2)$.
3. Another one. $\begin{bmatrix}1&2\\3&4\end{bmatrix}$ has determinant $1\cdot4-2\cdot3=-2$, so its inverse is $-\tfrac12\begin{bmatrix}4&-2\\-3&1\end{bmatrix}=\begin{bmatrix}-2&1\\1.5&-0.5\end{bmatrix}$.
4. No inverse. $\begin{bmatrix}1&2\\2&4\end{bmatrix}$ has determinant 0 and sends $(2,-1)$ to $\mathbf{0}$.
5. The quarter turn. Its inverse is the quarter turn the other way, 90° clockwise, which is its transpose.
6. A test. If $\mathbf{A}\mathbf{x}=\mathbf{0}$ has a solution $\mathbf{x}\neq\mathbf{0}$, can $\mathbf{A}$ be invertible? No: $\mathbf{A}^{-1}$ would send $\mathbf{0}$ to both $\mathbf{x}$ and $\mathbf{0}$.
7. Predict. As one column nears the line of the other, what happens to the playback? The grid squeezes, and the undo has to stretch it back more and more, without limit.

Where you’ll meet this

The stereo crossfeed of A matrix moves the plane mixed the channels with $\begin{bmatrix}0.7&0.3\\0.3&0.7\end{bmatrix}$. Its determinant is $0.49-0.09=0.40$, and its inverse is $\begin{bmatrix}1.75&-0.75\\-0.75&1.75\end{bmatrix}$, which unmixes them. Notice the cost: a difference between the channels, $(1,-1)$, comes out as $(2.5,-2.5)$. Any noise in the mixed sound that differs between the channels is made 2.5 times larger. The page Conditioning measures this effect.

A game finds the object under the mouse pointer by running the camera’s matrix backwards, from the screen into the scene. That is an inverse.

Reference card