Skip to content

The singular value decomposition

The SVD and its uses

Every matrix, square or not, is a rotation, a stretch along perpendicular axes and another rotation: the singular value decomposition. This topic uses it to compress images, find principal components in data, solve systems with the pseudo-inverse, and measure how sensitive a system is.

Lessons on this topic

The unit circle carried by A = [3, 0, 4, 5] in three moves. Stretches 6.708 and 2.236.The unit circle carried by A = [3, 0, 4, 5] in three moves. Stretches 6.708 and 2.236.

Intermediate · 25 minYou are hereRead

The singular value decomposition

See every matrix, square or not, as a turn, a stretch along two axes and another turn, with a flip in one of the turns when the matrix flips the plane, and read its rank and subspaces off the pieces.

Animation: a 128 by 128 greyscale picture of a sky, a bright disc and textured ground below a diagonal edge, rebuilt from its first singular pieces. Now 32 pieces: 8224 numbers stored instead of 16 384, 99.91 % of the energy kept.Animation: a 128 by 128 greyscale picture of a sky, a bright disc and textured ground below a diagonal edge, rebuilt from its first singular pieces. Now 32 pieces: 8224 numbers stored instead of 16 384, 99.91 % of the energy kept.

Intermediate · 20 minYou are hereRead

Low-rank approximation

Rebuild a picture from a few rank-one pieces, keep the largest ones for the best approximation, and count when it saves space.

Animation: six points and a line through the origin; each point drops its shadow on the line; a second panel plots the variance of the shadows against the angle. The line is at 26.57 degrees, shadow variance 18. The peak is 18 at 26.57 degrees, along (2, 1).Animation: six points and a line through the origin; each point drops its shadow on the line; a second panel plots the variance of the shadows against the angle. The line is at 26.57 degrees, shadow variance 18. The peak is 18 at 26.57 degrees, along (2, 1).

Intermediate · 25 minYou are hereRead

Principal components

Turn a line through a cloud of points to find the direction of most spread, get it from the covariance matrix or the SVD, and see what it can’t tell you.

The matrix B = [1, 1; 2, 2] flattens the plane, then its pseudo-inverse carries the line back. Target (1, 0), answer (0.1, 0.1).The matrix B = [1, 1; 2, 2] flattens the plane, then its pseudo-inverse carries the line back. Target (1, 0), answer (0.1, 0.1).

Intermediate · 25 minYou are hereRead

The pseudo-inverse

Undo a matrix as far as it can be undone, find the shortest least-squares answer, and damp the weak directions with ridge regularisation.

Two lines crossing near (1, 1), epsilon = 0.01. Condition number 402; a change of 0.01 moves the crossing by up to 1.414.Two lines crossing near (1, 1), epsilon = 0.01. Condition number 402; a change of 0.01 moves the crossing by up to 1.414.

Advanced · 25 minYou are hereRead

Conditioning

Watch the crossing of two nearly parallel lines fly away at the smallest nudge, measure that danger with the condition number, and tell a hard problem from a bad method.

Related topics

Where to go next.

Phasorium

Account

Sign inKeep your place on any device

Subjects

Signal processing
Linear algebra
Probability and statistics
Machine learning

Site

Look