The four fundamental subspaces

A matrix carries one plane onto another. Watch which part of each point arrives, and which part vanishes.

Two planes

The matrix at the top of the page is

Its second column is twice the first, so every output is a multiple of $(1,3)$: $\mathbf{A}\mathbf{x}=x_1(1,3)+x_2(2,6)=(x_1+2x_2)\,(1,3)$. The whole input plane lands on one line. Its rank is 1.

So two things happen to an input. Some of it arrives on the line of $(1,3)$, and some of it is lost. This page sorts out exactly which part is which. The answer uses four subspaces, two on each side of the matrix.

Column space and null space

You have met two of them. The column space $C(\mathbf{A})$, from Vector spaces and subspaces, is every output: here the line of $(1,3)$, on the output side. The null space $N(\mathbf{A})$, from The null space, is every input sent to zero: here the line of $(2,-1)$, on the input side, since $\mathbf{A}(2,-1)=(2-2,\;6-6)=(0,0)$.

For an $m\times n$ matrix of rank $r$, the column space has dimension $r$ and lives in $\mathbb{R}^m$. The null space has dimension $n-r$ and lives in $\mathbb{R}^n$.

Row space and left null space

The other two come from the rows. The row space is the span of the rows. The rows of $\mathbf{A}^\top$, the transpose from Transposes and permutations, are the columns of $\mathbf{A}$ and the other way round, so the row space is the column space of the transpose, $C(\mathbf{A}^\top)$. Here the rows are $(1,2)$ and $(3,6)$, and $(3,6)=3\,(1,2)$, so the row space is the line of $(1,2)$, on the input side.

The left null space is the null space of the transpose, $N(\mathbf{A}^\top)$: the vectors $\mathbf{y}$ with $\mathbf{A}^\top\mathbf{y}=\mathbf{0}$. Transpose both sides and this says $\mathbf{y}^\top\mathbf{A}=\mathbf{0}^\top$: the entries of $\mathbf{y}$ are weights that combine the rows of $\mathbf{A}$ into zero. Here $3\cdot\text{row 1}-\text{row 2}=\mathbf{0}$, so the left null space is the line of $(3,-1)$, on the output side. The name comes from $\mathbf{y}$ standing on the left of $\mathbf{A}$.

Right angles

Look at the input plane in the picture. The row line and the null line cross at a right angle. That always happens, and the reason is one line long. $\mathbf{A}\mathbf{x}=\mathbf{0}$ says that every row of $\mathbf{A}$, dotted with $\mathbf{x}$, gives 0. By Length and the dot product, that makes $\mathbf{x}$ perpendicular to every row, and so to every combination of rows. In symbols, with $\perp$ for “perpendicular to”:

The same argument on the transpose puts the other pair at a right angle, on the output side:

Here $(1,2)\cdot(2,-1)=2-2=0$ and $(1,3)\cdot(3,-1)=3-3=0$.

The trip across

Now follow one input. Every point of the input plane splits into a part on the row line and a part on the null line, which I write $\mathbf{x}=\mathbf{x}_\text{row}+\mathbf{x}_\text{null}$. The point $(3,1)$ is

The matrix sends the null part to zero and the row part onto the column line:

Drag the input point in the picture and watch its two parts. Slide it along the null line and the image does not move at all. Slide it along the row line and the image runs along the column line. Two different points of the row line never land on the same output, so the matrix carries the row space onto the column space one to one. Everything that is lost is the null part.

Dimensions add up

The row space has the same dimension as the column space: $r$. Elimination only mixes rows, so it never changes their span, and the rows with pivots in the reduced form are a basis of the row space. There are $r$ of them. So for an $m\times n$ matrix of rank $r$:

On each side the two dimensions add up to the whole space: $r+(n-r)=n$ for the inputs and $r+(m-r)=m$ for the outputs. For the matrix at the top, $m=n=2$ and $r=1$, so every one of the four is a line.

Worked example

1. The $2\times4$ matrix of the null space page. $\mathbf{A}=\begin{bmatrix}1&2&1&3\\2&4&3&8\end{bmatrix}$ has $m=2$, $n=4$, $r=2$. Row space: dimension 2. Null space: $4-2=2$, the two special solutions. Column space: 2, so all of $\mathbb{R}^2$. Left null space: $2-2=0$, only $\mathbf{0}$, since the two rows are independent.

2. A $3\times2$ matrix. $\mathbf{B}=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}$ has $m=3$, $n=2$, $r=2$. Row space: 2, all of $\mathbb{R}^2$. Null space: 0. Column space: 2, the plane $b_3=b_1+b_2$. Left null space: $3-2=1$, the line of $(1,1,-1)$, since

It is perpendicular to both columns: $(1,0,1)\cdot(1,1,-1)=0$ and $(0,1,1)\cdot(1,1,-1)=0$.

3. Counting. A $3\times4$ matrix of rank 2 has a row space of dimension 2. A $5\times3$ matrix of rank 3 has a left null space of dimension $5-3=2$.

Predict before you drag: where does $(3,1)$ land? At $(5,15)$, the same place as $(1,2)$.

Where you’ll meet this

Checking measurements. Suppose three sensors report $b_1$, $b_2$ and $b_3$, and the physics says $b_3=b_1+b_2$, as for $\mathbf{B}$ above. Every possible clean reading lies in $C(\mathbf{B})$, so it is perpendicular to the left null vector $(1,1,-1)$: $b_1+b_2-b_3=0$. A reading that fails this check holds an error. Left null vectors are the checks that consistent data must pass.

The best answer when there is none. When $\mathbf{A}\mathbf{x}=\mathbf{b}$ has no solution, the closest output leaves an error that is perpendicular to the column space, so the error lies in the left null space. That is the idea behind least squares, the method that fits a line to scattered data.

Rank-one images. A rank-one matrix of pixel values is one column times one row: every row of the image is a multiple of the same row. Such an image is a plaid: horizontal bands times vertical bands, and a photo can be built up as a sum of a few rank-one pieces.

Reference card