Matrices in computer graphics

A game moves thousands of corners every frame, all with matrices. Watch a cube turn, move away and land on a screen.

A move is not linear, in the plane

On A gallery of linear maps, sliding the plane by $(1,0)$ failed the test of linearity, because it moves the origin and every matrix keeps the origin in place. But games and drawing programs move objects all the time, and they want to do everything with matrices, because matrices chain by multiplication.

The trick is one extra coordinate. Write the point $(x,y)$ as $\mathbf{x}_\text{h}=(x,y,1)$, its homogeneous coordinates. Now a move by $(t_x,t_y)$ is a $3\times3$ matrix:

One dimension up, the move is a shear of space, and shears are linear. The plane $z=1$ just slides along inside it. Turns and scalings go in the top left corner, with a 1 in the bottom right: the turn by $\theta$ from A gallery of linear maps becomes $\begin{bmatrix}\cos\theta&-\sin\theta&0\\\sin\theta&\cos\theta&0\\0&0&1\end{bmatrix}$, still written $\mathbf{A}_\text{rot}(\theta)$.

Chaining steps

Every step is now a matrix, so a chain of steps is one product, applied right to left as in Matrix multiplication. A turn about a point $\mathbf{x}_0$ other than the origin takes three steps: move $\mathbf{x}_0$ to the origin, turn, move back.

For a quarter turn about $(2,1)$:

It sends $(3,1)$ to $(2,2)$, and it leaves $(2,1)$ where it is, as a turn about that point must.

The order matters, as it did for the turn and the shear: “move, then turn” is not “turn, then move”.

Perspective

In 3-D a point is $(x,y,z,1)$ and every step is $4\times4$. A camera at the origin looking along the $z$ axis draws a point at $(x/z,\ y/z)$ on a screen at distance 1: things twice as far look half as big. Division is not a matrix, but it can be left to the end. Use

which sends $(x,y,z,1)$ to $(x,y,z,z)$: it copies the depth into the last entry. Then divide by that last entry. Graphics cards work this way: multiply, then divide once per corner.

A vector whose last entry is 0 is a direction, not a place. Moves don’t change it: $\mathbf{A}_\text{move}(5,5)\,(1,2,0)=(1,2,0)$. That is right, since moving an object doesn’t change which way it points.

Worked example

1. Turn about a point. With a quarter turn, $\theta=\pi/2$: $\mathbf{A}_\text{rot}(\pi/2)\,\mathbf{A}_\text{move}(-2,-1)=\begin{bmatrix}0&-1&1\\1&0&-2\\0&0&1\end{bmatrix}$; then $\mathbf{A}_\text{move}(2,1)$ gives $\mathbf{A}_\text{pivot}$ above. Check: $\mathbf{A}_\text{pivot}(3,1,1)=(0-1+3,\;3+0-1,\;1)=(2,2,1)$.

2. The other order. Move by $(2,1)$, then turn about the origin: $\begin{bmatrix}0&-1&-1\\1&0&2\\0&0&1\end{bmatrix}$, which sends $(3,1,1)$ to $(-2,5,1)$.

3. The cube, unturned. Moved 4 units away, its front face has corners $(\pm1,\pm1,3)$ and its back face $(\pm1,\pm1,5)$. On the screen they land at $(\pm0.3333,\pm0.3333)$ and $(\pm0.2,\pm0.2)$: two nested squares, the front one larger in the ratio $5/3$.

4. The cube, turned. The turn by 53.13° (0.9273 rad, the 3-4-5 angle) followed by the move is one matrix:

The corner $(1,1,1)$ goes to $(1.4,\;1,\;3.8)$ and appears on the screen at $(1.4/3.8,\ 1/3.8)=(0.3684,\ 0.2632)$. The corner $(-1,1,1)$ goes to $(0.2,\;1,\;5.4)$ and appears at $(0.0370,\ 0.1852)$.

Try these before you read the answers.

• Is “turn, then move” the same as “move, then turn”? No.
• Where does $(1,1,3)$ land on the screen at distance 1? $(1/3,\ 1/3)$.
• Move the cube twice as far away. How big is its drawing? About half as big.

Where you’ll meet this

Every graphics card. A card draws a triangle by multiplying its corners by three matrices: the model’s place in the world, the camera’s view, and the perspective. A game multiplies the three once per object per frame, and then each corner costs one $4\times4$ product.

Robot arms. Each joint of an arm turns and offsets the next link, one $4\times4$ matrix per joint. The hand’s position is the product of the chain.

Augmented reality. To draw a virtual object on a live camera view, a phone estimates the camera’s $4\times4$ matrix from the picture, many times a second.

Reference card