A matrix moves the plane: matrix times vector

A matrix is a rule that moves every point of the plane at once. Watch the grid and its two arrows glide to their new place.

A matrix moves the plane

The columns of A are where the unit arrows e_₁ and e_₂ land.

Before the matrix acts: the square grid, and the unit arrows e_₁ and e_₂.

tip of e₁

(1, 0)

tip of e₂

(0, 1)

Describe this picture

One panel: a faint square grid with the origin at its centre, and two unit arrows from the origin, $\mathbf{e}_1$ pointing right and $\mathbf{e}_2$ pointing up, with the unit square between them shaded. After a second’s rest the grid glides, over 5 s, from its square starting place to the place the matrix sends it, turning a little and stretching at the same time, and holds there. The arrow $\mathbf{e}_1$ rides out to $(2,1)$ and $\mathbf{e}_2$ to $(-1,1)$ , and the shaded square becomes a parallelogram. The grid lines stay straight and evenly spaced the whole time, and the origin never moves. Readouts give the coordinates of the two arrow tips. When the clip has finished, a dashed ring marks each tip: either can be dragged, or moved with the arrow keys, in steps of half a unit, and the whole grid follows; the matrix is kept in the link.

Where the two arrows land

A point in the plane, say $(3,2)$ , is a set of instructions: take 3 steps along the first unit arrow and 2 steps along the second. You met this reading in Vectors and Linear combinations and span. I’ll call the two unit arrows $\mathbf{e}_1=(1,0)$ and $\mathbf{e}_2=(0,1)$ . A bold $\mathbf{e}$ with a number under it always means one of these arrows on this site, never the number $e$ .

So what happens if I move the two arrows? Every point is built from them, so every point has to move with them. The point $(3,2)$ is still “3 steps along the first arrow and 2 along the second”, only the arrows are different now.

That is the whole idea of a matrix. It is the table of where the two arrows go. In the picture at the top of the page, $\mathbf{e}_1$ lands on $(2,1)$ and $\mathbf{e}_2$ lands on $(-1,1)$ . I write those two landing places side by side, as columns:

\mathbf{A}=\begin{bmatrix}2&-1\\1&1\end{bmatrix}.

The first column is where $\mathbf{e}_1$ goes. The second column is where $\mathbf{e}_2$ goes. I’ll call the columns $\mathbf{a}_1=(2,1)$ and $\mathbf{a}_2=(-1,1)$ . To save space I write a column of numbers on one line, as $(2,1)$ ; it still means a column.

A common slip is to read the rows as the landing places. The first row here is $2,\,-1$ , but $\mathbf{e}_1$ lands on $(2,1)$ , the first column. I chose a matrix whose rows and columns differ, so the slip cannot go unnoticed.

After the clip, drag the tip of either arrow. The whole grid follows it, because every point is built from the two arrows.

Same steps, new arrows

Now follow one point. The point $(3,2)$ starts as 3 steps of $\mathbf{e}_1$ followed by 2 steps of $\mathbf{e}_2$ . Watch the same steps being taken with the new arrows.

The point rides along

x = (3, 2) as steps along e_₁ and e_₂, then the matrix moves the plane.

The point x = (3, 2) means 3 steps of e_₁, then 2 steps of e_₂.

(3, 2)

–

Describe this picture

The square grid with the point $(3,2)$ drawn as a chain of arrows, tip to tail: three copies of $\mathbf{e}_1$ , then two copies of $\mathbf{e}_2$ . The grid then moves as in the first picture, and the chain moves with it: it becomes three copies of $(2,1)$ followed by two copies of $(-1,1)$ , and it ends at $(4,5)$ . Then, in a corner of the panel, the two row sums appear one after the other, $2\cdot3+(-1)\cdot2=4$ and $1\cdot3+1\cdot2=5$ . The picture has no control.

The chain keeps its instructions, 3 steps then 2 steps, and only the arrows change. So the point lands at

3\begin{bmatrix}2\\1\end{bmatrix}+2\begin{bmatrix}-1\\1\end{bmatrix}=\begin{bmatrix}6-2\\3+2\end{bmatrix}=\begin{bmatrix}4\\5\end{bmatrix}.

This is what “a matrix times a vector” means. For a vector $\mathbf{x}=(x_1,x_2)$ ,

\mathbf{A}\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2 .

The numbers in $\mathbf{x}$ are the weights, and the columns of $\mathbf{A}$ are what they weigh. It is the linear combination of Linear combinations and span, with the columns as the arrows.

One row at a time

There is a second way to get the same answer, and it is the one most people compute by hand. Look at the top number of the answer, 4. It came from $2\cdot3$ in the first column and $(-1)\cdot2$ in the second. Those are the first row of $\mathbf{A}$ , $2,\,-1$ , multiplied entry by entry with $3,\,2$ and added up.

\begin{bmatrix}2&-1\\1&1\end{bmatrix}\begin{bmatrix}3\\2\end{bmatrix}=\begin{bmatrix}2\cdot3+(-1)\cdot2\\1\cdot3+1\cdot2\end{bmatrix}=\begin{bmatrix}4\\5\end{bmatrix}.

Multiply matching entries and add: that is the dot product of the row with $\mathbf{x}$ , which Length and the dot product studies on its own. So each number that comes out is one row’s dot product with $\mathbf{x}$ .

To write this for any matrix I need names for the entries. The entry in row $p$ and column $q$ is $a_{pq}$ , and both count from 1: here $a_{11}=2$ , $a_{12}=-1$ , $a_{21}=1$ and $a_{22}=1$ . Entry $p$ of the answer $\mathbf{y}=\mathbf{A}\mathbf{x}$ is

y_p=\sum_{q=1}^{n}a_{pq}\,x_q ,

where $n$ is the number of columns. The columns picture and the rows picture add up the same products, only in a different order. So they always agree.

What makes it linear

Watch the first picture again and look at the grid lines. They stay straight. Lines that were parallel stay parallel. Lines that were evenly spaced stay evenly spaced. And the origin stays where it is. That is what linear means for a map of the plane, and the column rule explains all of it.

The origin first. The point $(0,0)$ takes no steps along either arrow, so wherever the arrows go, it stays put:

\mathbf{A}\mathbf{0}=0\,\mathbf{a}_1+0\,\mathbf{a}_2=\mathbf{0}.

Now a grid line. Its points are $\mathbf{x}$ , $\mathbf{x}+\mathbf{v}$ , $\mathbf{x}+2\mathbf{v}$ and so on, evenly spaced along a step $\mathbf{v}$ . The column rule adds weights, so the matrix sends them to $\mathbf{A}\mathbf{x}$ , $\mathbf{A}\mathbf{x}+\mathbf{A}\mathbf{v}$ , $\mathbf{A}\mathbf{x}+2\mathbf{A}\mathbf{v}$ : again evenly spaced, along the new step $\mathbf{A}\mathbf{v}$ . Every grid line in the same direction has the same step, so they stay parallel. Both facts come from one rule, which holds for any vectors $\mathbf{x}$ , $\mathbf{y}$ and numbers $c$ , $d$ :

\mathbf{A}(c\,\mathbf{x}+d\,\mathbf{y})=c\,\mathbf{A}\mathbf{x}+d\,\mathbf{A}\mathbf{y}.

The unit square shows it too. Its corners are $(0,0)$ , $(1,0)$ , $(1,1)$ and $(0,1)$ . They land at $(0,0)$ , at $\mathbf{a}_1=(2,1)$ , at $\mathbf{a}_1+\mathbf{a}_2=(1,2)$ and at $\mathbf{a}_2=(-1,1)$ . That is the parallelogram in the first picture.

Numbers in, numbers out

A matrix does not have to be square. Its shape is written rows × columns: an $m\times n$ matrix has $m$ rows and $n$ columns. There is one column for each number that goes in, so it takes $n$ numbers in. There is one row for each number that comes out, so it gives $m$ numbers out.

Here is a $3\times2$ matrix:

\mathbf{B}=\begin{bmatrix}1&0\\0&1\\1&1\end{bmatrix}.

It sends $\mathbf{e}_1$ to $(1,0,1)$ and $\mathbf{e}_2$ to $(0,1,1)$ , two arrows in space. So the point $(3,2)$ goes to $3\,(1,0,1)+2\,(0,1,1)=(3,2,5)$ . Two numbers go in and three come out. The whole flat plane lands on a tilted flat sheet in space, where the third number is the sum of the first two.

Fig. The 3×2 matrix B takes the flat plane into space. Every point lands on one tilted sheet, and (3, 2) lands at (3, 2, 5), straight above (3, 2).

Worked example

Take $\mathbf{A}=\begin{bmatrix}2&-1\\1&1\end{bmatrix}$ and $\mathbf{x}=(3,2)$ .

By columns. $\mathbf{A}\mathbf{x}=3\,(2,1)+2\,(-1,1)=(6-2,\ 3+2)=(4,5)$ .
By rows. First row: $2\cdot3+(-1)\cdot2=4$ . Second row: $1\cdot3+1\cdot2=5$ . The same $(4,5)$ .
The second arrow. Where does $(0,1)$ land? It is $\mathbf{e}_2$ , so it lands on the second column, $(-1,1)$ .
Another point. $\mathbf{A}\,(1,-1)=1\,(2,1)-1\,(-1,1)=(3,0)$ . By rows: $2+1=3$ and $1-1=0$ .
The origin. $\mathbf{A}\,(0,0)=(0,0)$ . A matrix never moves the origin.
Predict first. A matrix has columns $(1,0)$ and $(1,1)$ . Where does $(2,3)$ go? Guess from the picture of the two arrows, then check: $2\,(1,0)+3\,(1,1)=(5,3)$ .
Shapes. The $3\times2$ matrix $\mathbf{B}$ above takes 2 numbers to 3: $(3,2)$ goes to $(3,2,5)$ .

Where you’ll meet this

A stereo crossfeed, which makes headphones sound more like speakers, mixes a little of each channel into the other. With the matrix $\begin{bmatrix}0.7&0.3\\0.3&0.7\end{bmatrix}$ , a sound on the left channel only, $(1,0)$ , comes out as $(0.7,0.3)$ : the first column.

A 3-D graphics program moves a model by applying one matrix to every corner of it, many times a second. Every layer of a neural network starts with a matrix times a vector. A phone turns the readings of its motion sensor with a matrix, so that “up” for the sensor matches “up” on the screen.

In signal processing · convolution

A filter in signal processing is a matrix too. Stack shifted copies of the impulse response $h$ as the columns of a matrix $\mathbf{H}$ , and filtering the samples $\mathbf{x}$ is $\mathbf{y}=\mathbf{H}\mathbf{x}$ , the column rule of this page. The page Discrete convolution shows that view. One difference in counting: matrix entries count from 1, while samples $x[n]$ count from 0, so the vector’s entry $x_q$ is the sample $x[q-1]$ . A coefficient with one subscript, like $a_k$ , is a filter coefficient there; two subscripts, $a_{pq}$ , always mean a matrix entry.

Reference card

Quantity	Formula	Notes
Unit arrows	$\mathbf{e}_1=(1,0)$ , $\mathbf{e}_2=(0,1)$	bold, never the number $e$
Columns	$\mathbf{a}_q$ is where $\mathbf{e}_q$ lands	read the columns, not the rows
By columns	$\mathbf{A}\mathbf{x}=x_1\mathbf{a}_1+x_2\mathbf{a}_2$	the entries of $\mathbf{x}$ are weights
By rows	$y_p=\sum_{q=1}^{n}a_{pq}x_q$	one dot product per row
Entries	$a_{pq}$ : row $p$ , column $q$	both count from 1
Linear	$\mathbf{A}(c\mathbf{x}+d\mathbf{y})=c\mathbf{A}\mathbf{x}+d\mathbf{A}\mathbf{y}$	straight, parallel, evenly spaced lines
Origin	$\mathbf{A}\mathbf{0}=\mathbf{0}$	never moves
Shape	$m\times n$ : $n$ numbers in, $m$ out	rows × columns

A matrix moves the plane