Independence, basis and dimension

Two arrows can draw a grid of their own. Watch the plane’s grid change underneath a point that stays still.

A grid of your own

The usual grid is built from two arrows, $\mathbf{e}_1=(1,0)$ and $\mathbf{e}_2=(0,1)$. The point $(5,4)$ means 5 steps of the first and 4 of the second. Nothing forces those two arrows. With $\mathbf{v}_1=(2,1)$ and $\mathbf{v}_2=(1,2)$ the same point is

so on the slanted grid its address is $(2,1)$: 2 steps of $\mathbf{v}_1$, 1 of $\mathbf{v}_2$. The point did not move. Only the grid changed, and the address changed with it.

Now drag $\mathbf{v}_2$ toward the line of $\mathbf{v}_1$ and watch the readout. With $\mathbf{v}_2=(1.5,1)$ the address of $(5,4)$ is $(-2,6)$. With $\mathbf{v}_2=(1.8,1)$ it is $(-11,15)$. The cells are so thin that it takes huge steps back and forth to reach the point. On the line itself both arrows point the same way, and the grid can’t reach $(5,4)$ at all.

Independent

What went wrong on the line is that one arrow became a mix of the other. A set of vectors is independent when no vector in it is a combination of the others. Each one adds a new direction.

There is a cleaner test. Put the vectors side by side as the columns of a matrix, $\mathbf{V}=[\,\mathbf{v}_1\ \cdots\ \mathbf{v}_n\,]$. A combination with weights $\mathbf{c}=(c_1,\dots,c_n)$ is $\mathbf{V}\mathbf{c}$. If one vector is a mix of the others, move it to the other side and you get a combination equal to $\mathbf{0}$ with weights that are not all zero. So the vectors are independent exactly when $\mathbf{V}\mathbf{c}=\mathbf{0}$ has only the solution $\mathbf{c}=\mathbf{0}$. In the words of the null space page:

By The null space, that means no free columns: a pivot in every column, so the rank equals the number of vectors.

Basis

A basis of a space is a set of vectors that is independent and spans the space. Spanning means every point can be reached; independent means no arrow is wasted.

• $\mathbf{e}_1,\mathbf{e}_2$ is a basis of the plane, the standard basis. So is $\mathbf{v}_1,\mathbf{v}_2$.
• In $\mathbb{R}^n$ the standard basis is $\mathbf{e}_1,\dots,\mathbf{e}_n$, where $\mathbf{e}_q$ has a 1 in place $q$ and 0 elsewhere.
• The plane $x+y+z=0$ of Vector spaces and subspaces has the basis $(1,-1,0)$, $(0,1,-1)$.

A subspace has many bases. A basis is a choice of grid, and the grid is yours to choose.

Coordinates are unique

Given a basis, every point has exactly one address. Suppose a point $\mathbf{x}$ had two, $\mathbf{x}=\mathbf{V}\mathbf{c}$ and $\mathbf{x}=\mathbf{V}\mathbf{d}$. Subtract: $\mathbf{V}(\mathbf{c}-\mathbf{d})=\mathbf{0}$. The basis is independent, so $\mathbf{c}-\mathbf{d}=\mathbf{0}$, and the two addresses are the same. The numbers in $\mathbf{c}$ are the coordinates of $\mathbf{x}$ in that basis.

Coordinates belong to the basis as much as to the point. In the next picture the same points are counted on two grids at once.

Watch each point’s two readouts. The point is one place; its two addresses differ because the grids differ.

Dimension

Every basis of the plane has two vectors, and every basis of space has three. That is not a coincidence: every basis of a space has the same number of vectors. That number is the dimension of the space, written $\dim$.

Here is why, in $\mathbb{R}^n$. Put $k$ vectors in a matrix with $n$ rows. Each row holds at most one pivot, so there are at most $n$ pivots.

• If $k>n$, some column has no pivot, so there is a free variable and the vectors are dependent.
• If $k<n$, there are at most $k$ pivots, fewer than $n$ rows, so elimination leaves a zero row and some targets can’t be reached: the vectors don’t span.

So a basis of $\mathbb{R}^n$ has exactly $n$ vectors, and $\dim\mathbb{R}^n=n$. The same counting gives the dimensions of the subspaces of a matrix. The pivot columns of $\mathbf{A}$ are a basis of its column space, so $\dim C(\mathbf{A})=r$, the rank. The null space has dimension $n-r$, as on the page about the null space.

Worked example

1. Dependent. $(1,2,3)+(2,3,4)-(3,5,7)=(0,0,0)$, with weights $1,1,-1$, not all zero. The three vectors are dependent, and the matrix with them as columns has rank 2.

2. Coordinates in a basis of space. Take the basis $(1,0,0)$, $(1,1,0)$, $(1,1,1)$ and the point $(3,2,1)$:

Work from the bottom: $c_3=1$, then $c_2+1=2$ gives $c_2=1$, then $c_1+2=3$ gives $c_1=1$. The coordinates are $(1,1,1)$.

3. A basis of a plane. Any point $(x_1,x_2,x_3)$ of $x_1+x_2+x_3=0$ is $x_1(1,-1,0)+(x_1+x_2)(0,1,-1)$. For the point $(2,1,-3)$ the coordinates are $(2,3)$: $2(1,-1,0)+3(0,1,-1)=(2,1,-3)$.

4. The slanted grid. $(5,4)=2(2,1)+1(1,2)$: coordinates $(2,1)$.

5. Counting. Five vectors in $\mathbb{R}^4$ can’t be independent. A $4\times6$ matrix of rank 4 has a null space of dimension $6-4=2$.

Where you’ll meet this

Sinusoids as a basis. Fourier’s idea is a change of basis: write a signal in a basis of sinusoids instead of a basis of single samples. Decomposing signals splits signals into simpler pieces that add back up to them, and a basis is the case where the pieces are fixed and the weights are unique.

Image compression. A JPEG image cuts a photo into blocks of 8 by 8 pixels, 64 numbers each. It writes each block in a basis of 64 cosine patterns instead of 64 single pixels. In that basis most coordinates of a smooth block are small, and those can be stored roughly or dropped.

Reference card