Change of basis

A point stays exactly where it is while the grid behind it changes. Watch its coordinates change, though it never moves.

Coordinates belong to a basis

The point in the picture is one place. Described on the usual grid, it is $(5,1)$: 5 steps of $\mathbf{e}_1$ and 1 of $\mathbf{e}_2$. Described on the grid of $\mathbf{v}_1=(1,-1)$ and $\mathbf{v}_2=(2,1)$, it is 1 step of $\mathbf{v}_1$ and 2 of $\mathbf{v}_2$:

So its coordinates in that basis are $(1,2)$. The point did not move; only the description changed. That is the lesson of Independence, basis and dimension, now turned into a method.

Going in and coming out

Put the basis vectors as the columns of $\mathbf{V}=\begin{bmatrix}1&2\\-1&1\end{bmatrix}$. Coordinates $\mathbf{c}$ in that basis rebuild the point:

To go the other way, from the point to its coordinates, undo $\mathbf{V}$ with its inverse from The inverse matrix:

Here $\det\mathbf{V}=1+2=3$ and $\mathbf{V}^{-1}=\tfrac13\begin{bmatrix}1&-2\\1&1\end{bmatrix}$, so $\mathbf{c}=\tfrac13(5-2,\;5+1)=(1,2)$, as the picture read.

Between two bases. To move from coordinates in the basis of $\mathbf{V}$ to coordinates in the basis of another matrix $\mathbf{W}$, go out through $\mathbf{V}$ and back in through $\mathbf{W}^{-1}$. Write the two coordinate vectors as $\mathbf{c}$ and $\mathbf{c}'$:

With $\mathbf{W}=\begin{bmatrix}1&0\\1&1\end{bmatrix}$, $\mathbf{W}^{-1}\mathbf{V}=\begin{bmatrix}1&2\\-2&-1\end{bmatrix}$ sends $(1,2)$ to $(5,-4)$. Check: $5\,(1,1)-4\,(0,1)=(5,1)$, the same point again.

A map in a new basis

A matrix $\mathbf{A}$ describes a map in the usual coordinates. To describe the same map in the basis of $\mathbf{V}$, take three steps: come out of the new coordinates ($\mathbf{V}$), apply the map ($\mathbf{A}$), and go back in ($\mathbf{V}^{-1}$):

That is the similar matrix of Similar matrices and the Jordan form. The point of choosing a basis is to make $\mathbf{B}$ simple. The basis in the picture was not chosen at random: $(1,-1)$ and $(2,1)$ are the eigenvectors of $\mathbf{A}=\begin{bmatrix}1.5&1\\0.5&1\end{bmatrix}$, with eigenvalues 0.5 and 2. So

In the eigenvector basis the map is two separate stretches, which is what the end of the clip shows. That is diagonalisation, from Diagonalisation and matrix powers, seen as a change of basis.

Follow the point through. In coordinates, $(1,2)\to\mathbf{B}(1,2)=(0.5,\,4)$. Back out: $\mathbf{V}(0.5,4)=(0.5+8,\;-0.5+4)=(8.5,\,3.5)$. Directly, $\mathbf{A}(5,1)=(8.5,\,3.5)$. The same answer, by the long road and the short one.

Whatever basis you choose, some things never change: the trace, the determinant and the eigenvalues of $\mathbf{B}$ are those of $\mathbf{A}$.

Orthonormal bases are the easy case

When the basis vectors are perpendicular and of length 1, $\mathbf{V}^{-1}=\mathbf{V}^\top$, as in Orthonormal bases and Gram–Schmidt. Then each coordinate is a dot product: $c_k=\mathbf{v}_k\cdot\mathbf{x}$. For a frame turned by 53.13° (0.9273 rad), the angle whose cosine is 3/5, $\mathbf{V}=\tfrac15\begin{bmatrix}3&-4\\4&3\end{bmatrix}$, the point $(5,10)$ has coordinates

and $\mathbf{V}(11,2)=\tfrac15(33-8,\;44+6)=(5,10)$ brings it back. No inverse to compute.

A transform is a change of basis

Four samples can be written in the basis of waves from Independence, basis and dimension. The impulse $(4,0,0,0)$ has coordinates $(1,2,0,1)$ there:

The signal is the same in both rows. A transform like the DFT does not change the signal; it changes the basis you describe it in, from single samples to waves.

Worked example

1. Coordinates. $\mathbf{V}=\begin{bmatrix}1&2\\-1&1\end{bmatrix}$, $\mathbf{V}^{-1}=\tfrac13\begin{bmatrix}1&-2\\1&1\end{bmatrix}$, and $\mathbf{x}=(5,1)$ has $\mathbf{c}=(1,2)$. Check: $1(1,-1)+2(2,1)=(5,1)$.

2. The map in the eigenbasis. $\mathbf{B}=\mathbf{V}^{-1}\mathbf{A}\mathbf{V}=\mathrm{diag}(0.5,\,2)$, and the long road gives $\mathbf{A}\mathbf{x}=(8.5,3.5)$.

3. A basis that is not made of eigenvectors. With columns $(1,0)$ and $(1,1)$, $\mathbf{B}=\begin{bmatrix}1&1\\0.5&1.5\end{bmatrix}$. It is not diagonal, but its trace is 2.5 and its determinant $1.5-0.5=1$, the same as for $\mathbf{A}$: the eigenvalues are still 2 and 0.5.

4. Two bases. $\mathbf{W}^{-1}\mathbf{V}(1,2)=(5,-4)$ in the basis $(1,1)$, $(0,1)$.

5. A turned frame. $(5,10)$ has coordinates $(11,2)$ in the frame turned by 53.13°.

Try these before you read the answers.

• Coordinates of $(3,3)$ in the basis $(1,1)$, $(1,-1)$? $(3,0)$.
• If $\mathbf{V}=\mathbf{I}$, what changes? Nothing: the coordinates are the entries.
• $\mathbf{V}$ has orthonormal columns. What is $\mathbf{V}^{-1}$? $\mathbf{V}^\top$.
• In the eigenbasis, what does $\mathbf{A}^2$ look like? $\mathrm{diag}(0.25,\,4)$.

Where you’ll meet this

Waves instead of samples. The Fourier transform and the cosine transform of a JPEG block describe the same data in a basis of waves, where smoothing and compression are simple.

Turning a phone. A phone reads its motion sensor in the sensor’s own axes and changes basis into the screen’s axes, as in the turned frame above.

Robot arms. Each joint of a robot arm has its own frame, and the position of the hand is found by changing basis from frame to frame.

Reference card