Length and the dot product

How much do two arrows point the same way? Watch one arrow’s shadow on the other grow, vanish and turn negative.

Length

Before I compare two arrows, I need the length of one. The arrow $\mathbf{v}=(3,1)$ goes 3 across and 1 up. Those two moves are the short sides of a right triangle, and the arrow is its long side. Pythagoras gives the length:

The length of a vector is written with double bars, $\lVert\mathbf{v}\rVert$. In two dimensions, $\lVert\mathbf{v}\rVert=\sqrt{v_1^2+v_2^2}$. In three, use Pythagoras twice: the arrow $(1,2,2)$ has length $\sqrt{1+4+4}=3$. In $n$ dimensions you keep going, one square per component:

An arrow of length 1 is a unit vector. To make one that points the same way as $\mathbf{v}$, divide $\mathbf{v}$ by its length: $\mathbf{v}/\lVert\mathbf{v}\rVert=(3,1)/3.162=(0.949,\;0.316)$. Its length is 1, and it keeps only the direction.

The dot product

Now take two vectors and multiply them component by component, then add the products. That number is the dot product:

For $\mathbf{v}=(3,1)$ and $\mathbf{w}=(1,2)$ it is $3\cdot1+1\cdot2=5$. The answer is a number, not a vector.

Dot a vector with itself and you get the sum of its squares, which is its length squared: $\mathbf{v}\cdot\mathbf{v}=\lVert\mathbf{v}\rVert^2$. So length is the dot product’s first use.

The recipe hides what the number means. Here is the picture. Let $\theta$ be the angle between the two arrows. Then

Read the right-hand side in two parts. $\lVert\mathbf{w}\rVert\cos\theta$ is the shadow of $\mathbf{w}$ on the line of $\mathbf{v}$: drop a line from the tip of $\mathbf{w}$ straight down onto that line, at a right angle, and measure from the origin. The dot product is that shadow times the length of $\mathbf{v}$.

Why the two formulas agree

The two arrows and the arrow $\mathbf{w}-\mathbf{v}$ between their tips form a triangle. The law of cosines, from school geometry, gives its third side from the other two and the angle between them:

Now work out the same length from the components. Length squared is a vector dotted with itself, and the dot product multiplies out like ordinary brackets, one component at a time:

The two right-hand sides are equal. Cancel $\lVert\mathbf{v}\rVert^2$ and $\lVert\mathbf{w}\rVert^2$ from both, divide by $-2$, and the sum of products equals $\lVert\mathbf{v}\rVert\,\lVert\mathbf{w}\rVert\cos\theta$. Any two arrows lie in one flat plane, even in many dimensions, so the triangle argument works in every dimension.

Angles and right angles

Go back to the picture at the top of the page and drag $\mathbf{w}$ round slowly. Watch the sign of the product.

• When $\mathbf{w}$ points along $\mathbf{v}$, the angle $\theta$ is 0°, $\cos\theta=1$, and the product is as large as it can be.
• As $\mathbf{w}$ turns away, the shadow shrinks and so does the product.
• When $\theta$ reaches 90° the shadow is a single point and the product is exactly 0.
• Past 90° the shadow points backwards along the line, and the product is negative.

So the sign tells you the rough direction. Positive: the arrows lean the same way, less than 90° apart. Zero: they are at a right angle. Negative: they lean apart, more than 90°.

Turn the formula round and you can find the angle from the components alone:

Two vectors with a dot product of 0 are at a right angle; they are orthogonal, or perpendicular. For example $(3,1)\cdot(-1,3)=-3+3=0$. This test works in any number of dimensions, where you can’t draw a right angle: $(1,2,2)\cdot(2,1,-2)=2+2-4=0$, so these two arrows in space are orthogonal.

A zero dot product does not mean one of the vectors is zero. Both $(3,1)$ and $(-1,3)$ have length $\sqrt{10}$. The product is zero because the arrows are at a right angle.

Two recordings

The same test compares things you can’t draw as arrows at all. Take two short frames of a recording, four samples each. Each frame is a vector in $\mathbb{R}^4$, and the cosine of the angle between them says how alike their shapes are.

Notice that the two arrows in that picture are drawn in their own plane, not along the axes. Four numbers each still make two arrows, and two arrows always lie in one plane.

Two limits

The picture gives two facts that hold for every pair of vectors.

The dot product is never bigger than the two lengths multiplied. A cosine lies between $-1$ and 1, so

This is the Cauchy–Schwarz inequality. The two sides are equal only when the arrows lie on one line. In the clip at the top of the page the product never climbs above $\sqrt{10}\cdot\sqrt5=7.071$, the height the cosine curve reaches when $\mathbf{w}$ lies along $\mathbf{v}$.

The straight path is never longer than the bent one. The sum $\mathbf{v}+\mathbf{w}$ is the third side of the tip-to-tail triangle from Vectors, so

This is the triangle inequality. It follows from the first one. Square the left side:

By Cauchy–Schwarz the middle term is at most $2\,\lVert\mathbf{v}\rVert\,\lVert\mathbf{w}\rVert$, and $\lVert\mathbf{v}\rVert^2+2\,\lVert\mathbf{v}\rVert\,\lVert\mathbf{w}\rVert+\lVert\mathbf{w}\rVert^2$ is a perfect square. So

and taking the square root of both sides gives the triangle inequality.

Worked example

Take $\mathbf{v}=(3,1)$ and $\mathbf{w}=(1,2)$.

1. Dot product. $\mathbf{v}\cdot\mathbf{w}=3\cdot1+1\cdot2=5$.
2. Lengths. $\lVert\mathbf{v}\rVert=\sqrt{10}=3.162$ and $\lVert\mathbf{w}\rVert=\sqrt5=2.236$.
3. Angle. $\cos\theta=5/(\sqrt{10}\,\sqrt5)=5/\sqrt{50}=0.707$, so the angle $\theta$ is 45°, which is $\pi/4\approx0.7854$ rad. Check it another way: $\mathbf{v}$ points 18.4° above the horizontal and $\mathbf{w}$ points 63.4° above it, and 63.4° − 18.4° = 45°.
4. Cauchy–Schwarz. $5\le\sqrt{50}=7.071$.
5. Triangle. $\mathbf{v}+\mathbf{w}=(4,3)$ has length $\sqrt{16+9}=5$, and $5\le\sqrt{10}+\sqrt5=5.398$.
6. Unit vector. $\mathbf{v}/\lVert\mathbf{v}\rVert=(0.949,\;0.316)$.
7. Right angles. $(3,1)\cdot(-1,3)=0$; in space, $(1,2,2)\cdot(2,1,-2)=0$, and $\lVert(1,2,2)\rVert=3$.

Predict before you drag: at which angle is the shadow longest? At 0°, when $\mathbf{w}$ lies along $\mathbf{v}$. And if $\lVert\mathbf{v}\rVert=2$, $\lVert\mathbf{w}\rVert=3$ and $\theta$ is 60°, whose cosine is $\tfrac12$, the product is $2\cdot3\cdot\tfrac12=3$.

Where you’ll meet this

Work done by a force. Push a box with a force of $(5,2)$ newtons while it slides $(3,0)$ metres along the floor. Only the part of the force along the motion does work: $(5,2)\cdot(3,0)=15$ joules. The sideways part, 2 newtons, does none.

How alike are two recordings? The frames $(1,2,1,0)$ and $(0,1,2,1)$ have dot product $0+2+2+0=4$, and each has length $\sqrt6$. Their cosine is $4/6=0.667$, an angle of 48.2°. This number, the cosine similarity, is how text search tools compare documents and how audio tools compare sound frames: 1 means the same shape, 0 means no shared shape (the frames are orthogonal).

Signal processing runs on dot products. Correlation slides one signal along another and takes a dot product at each shift. Each value of The DFT is a dot product of the signal with a probe wave. Matched filters and detection uses Cauchy–Schwarz to prove that the best detector for a known pulse is a dot product with that pulse.

Reference card