The sampling theorem

Sampling multiplies a signal by a comb of spikes, and the comb is a sum of cosines. Here the comb is built one cosine at a time, at a sample rate of 10 kHz. Watch each new cosine bring a new pair of copies of the signal’s spectrum.

A sampled signal is a signal times a comb

The page Sampling & aliasing (10.1) found where one tone lands after sampling. This page looks at a whole spectrum at once. The answer is short: sampling puts a copy of the spectrum at every multiple of the sample rate $f_s$. The sampling theorem is the condition under which those copies do not overlap.

First I need a way to write sampling as an operation on a signal. Take a continuous-time signal $x_c(t)$ and multiply it by the impulse train $\delta_{T_s}(t)=\sum_n\delta(t-nT_s)$ from Transforms of common signals (8.3), a spike every sample period $T_s=1/f_s$. A spike times a signal is the spike scaled by the signal’s value there, as in Impulse, step and ramp (3.1). So

The signal $x_s(t)$ is the samples written as spikes. It is a model: the spikes exist on paper, not on a wire. The model is useful because $x_s(t)$ is a continuous-time signal, so the transforms of Chapter 8 apply to it.

The comb has a series, found in Transforms of common signals (8.3): equal arrows of size $1/T_s$ at every multiple of $f_s$, for positive and negative multiples alike. Pair the arrow at $+kf_s$ with the arrow at $-kf_s$. Each pair is a cosine, because $e^{j\theta}+e^{-j\theta}=2\cos\theta$, as in Complex exponentials & phasors (3.4). So

The signal I use throughout is called $x_c$. Its spectrum $X(f)$ is a triangle from $-4$ kHz to $4$ kHz, with height 1 at $f=0$. Nothing in $x_c$ is above 4 kHz. I call a signal with that property band-limited, and I call the highest frequency it contains $f_\text{max}$, here $f_\text{max}=4$ kHz. I wrote $X$ in hertz, $X(f)$, the form given beside $X(j\omega)$ in From series to transform (8.1). A real signal’s spectrum is mirrored at negative frequency, so a real $x_c$ has both edges, $\pm4$ kHz.

Each cosine of the comb makes a pair of copies

Multiply $x_c$ by the comb one term at a time. The constant 1 leaves $x_c$ alone, so its spectrum stays where it is. Each cosine term is a modulation, from Properties of the Fourier transform (8.2): in hertz, multiplying by $\cos(2\pi f_1t)$ gives half a copy of $X$ at $+f_1$ and half a copy at $-f_1$,

The comb’s cosines carry a factor 2, which turns each half copy into a whole one. So $2\cos(2\pi kf_st)$ makes full copies of $X$ at $+kf_s$ and $-kf_s$.

That is what the picture at the top of the page showed, with $f_s=10$ kHz. Each new cosine in the comb panel brings one new pair of copies in the spectrum panel, in the same colour and label:

• The comb’s first term is a constant, 1. Multiplying by it leaves $x$ alone: one spectrum, centred at 0, and a comb flat at 1.
• Two cosines in, the comb’s peaks reach 5 at 0 and at $\pm0.1$ ms, and the copies stand at $0$, $\pm10$ and $\pm20$ kHz. Each cosine of the comb at $kf_s$ makes a pair of copies at $\pm kf_s$.
• With three cosines the comb’s peaks are 7 and the copies reach $\pm30$ kHz. Every cosine added sharpens the spikes and adds a pair of copies: sampling copies the spectrum to every multiple of $f_s$.

The comb with $K$ cosines peaks at $1+2K$, in units of $1/T_s$, so the peaks are 1, 3, 5 and 7 for $K=0,1,2,3$. The spikes sharpen because more cosines agree at the peaks and disagree everywhere else. With every cosine added, which is what the comb really has, the copies reach every multiple of $f_s$.

Adding all the terms, the spectrum of the samples is

Each copy carries the factor $1/T_s$. The instruments draw every copy relative to the peak of $X$ and leave the factor out. A wallpaper repeats one pattern at every step along the wall: sampling does the same to the spectrum, with $f_s$ as the step.

Copies approach, touch, overlap

The copies sit $f_s$ apart. The copy of $X$ at 0 reaches up to $f_\text{max}$, and the copy at $f_s$ reaches down to $f_s-f_\text{max}$. So between them there is a gap of

A negative value means the copies overlap. For $f_\text{max}=4$ kHz and $f_s=12$ kHz the space is $12-8=4$ kHz, and at 9 kHz it is 1 kHz.

The next instrument lowers $f_s$ from 12 kHz to 6 kHz. The dashed box marks what an ideal low-pass from Frequency response and Bode plots (8.4) would keep. Watch the space between the copies close.

At 12 kHz the copies sit with 4 kHz of clear space between them, and the dashed low-pass could lift the middle one out whole. At 8 kHz, twice the highest frequency, the copies just touch. At 6 kHz they overlap from 2 to 4 kHz. There the heights add to a flat 0.5, and no filter can pull the original back out.

When the clip ends, drag across the spectrum to set the sample rate yourself. The space reaches 0 at 8 kHz, and below that the summed curve loses the triangle’s shape.

Why does the sum read a flat 0.5? At $f_s=6$ kHz, between 2 and 4 kHz, the triangle is $1-f/4$ and the copy centred at 6 kHz is $1-(6-f)/4=(f-2)/4$. Their sum is $1-\tfrac f4+\tfrac f4-\tfrac12=\tfrac12$. The two slopes cancel, and the original triangle is not recoverable there, because every height in the overlap is the sum of two unknowns.

The fold is an overlap

Page 10.1 said that a tone above half the sample rate folds back. The copies explain why. Take the part of $x_c$ at 3.5 kHz and sample at $f_s=6$ kHz.

The triangle at $-3.5$ kHz has height $1-3.5/4=0.125$. The copy centred at 6 kHz carries that point to $-3.5+6=2.5$ kHz, and keeps its height $0.125$. At 2.5 kHz the original triangle has height $1-2.5/4=0.375$, so the two add to $0.375+0.125=0.5$, the flat value of the overlap. This is 10.1’s fold, $f_s-f=6-3.5=2.5$ kHz, seen as two copies that overlap.

The theorem

If the copies do not overlap, the middle one is a clean copy of $X$ and nothing else. An ideal low-pass filter that keeps $\lvert f\rvert < f_s/2$ and has gain $T_s$ lifts it out and cancels the factor $1/T_s$. What is left is $X(f)$, and so $x_c(t)$. That is the whole argument, and it needs the gap $f_s-2f_\text{max}$ to be positive.

Theorem. If $X(f)=0$ for $\lvert f\rvert>f_\text{max}$ and $f_s>2f_\text{max}$, the copies do not overlap, and the samples determine $x_c$: keep the middle copy. Reconstruction (10.3) shows how in time.

This is the Nyquist–Shannon sampling theorem.

Two names go with it, and they are easy to mix up. The number $2f_\text{max}$ is the Nyquist rate: it belongs to the signal. The number $f_N=f_s/2$ is the Nyquist frequency: it belongs to the sampler. The theorem says the sample rate must exceed the signal’s Nyquist rate. That is the same as saying $f_\text{max}<f_N$.

For the example signal, the Nyquist rate is $2\cdot4=8$ kHz. At 10 kHz the Nyquist frequency is 5 kHz, which is above 4 kHz, so there is room: the space is $10-8=2$ kHz.

Exactly twice

The theorem says “greater than”, and the instrument showed the copies just touching at 8 kHz. Why not “at least”? Sample a 2 kHz wave at exactly twice its frequency, $f_s=4$ kHz, so that the samples fall every half period.

The first strip is a sine of size 1 and all its samples are 0, the same as the samples of a signal that is silent. The other two strips are waves of the same frequency and the same size, and they give different dots. No rule can recover the size from the dots, so the theorem asks for strictly more than twice. In the spectrum this is the same event as the copies touching: the spike of the wave at $+f_\text{max}$ lands on the copy of its mirror spike at $-f_\text{max}$.

Real signals need margin

A band-limited signal lasts forever. The reason is the trade of Properties of the Fourier transform (8.2): a signal that is exactly zero above $f_\text{max}$ cannot be zero in time outside a finite stretch. The ideal low-pass of Frequency response and Bode plots (8.4) shows the same thing, because its response never stops. A recording lasts a finite time, so it always has a little energy above any $f_\text{max}$ you pick.

So practice leaves space and filters first. CD audio keeps content up to 20 kHz, so its Nyquist rate is 40 kHz. It samples at 44.1 kHz, which gives a Nyquist frequency of 22.05 kHz and a space of $44.1-40=4.1$ kHz for a real filter to fall in. Anti-aliasing and practical converters (10.4) picks this up.

Worked example

1. Copies. With $f_s=10$ kHz and $f_\text{max}=4$ kHz, the copies are centred at $0,\pm10,\pm20,\dots$ kHz. The space between them is $10-2\cdot4=2$ kHz.
2. Slide. The space $f_s-2f_\text{max}$ is 4 kHz at 12 kHz, 1 kHz at 9 kHz and 0 at 8 kHz (touching). At 7 kHz the copies overlap by 1 kHz, and at 6 kHz by 2 kHz. At 6 kHz the triangle $1-f/4$ and the copy $(f-2)/4$ sum to $0.5$ on $2\le f\le4$.
3. CD. For $f_\text{max}=20$ kHz the Nyquist rate is 40 kHz. At $f_s=44.1$ kHz the Nyquist frequency is $f_N=22.05$ kHz and the space is 4.1 kHz.
4. Telephone. Speech up to 3.4 kHz has a Nyquist rate of 6.8 kHz. At $f_s=8$ kHz the space is $8-6.8=1.2$ kHz.
5. Comb. The partial sum with $K$ cosines peaks at $2K+1$, in units of $1/T_s$: 1, 3, 5, 7 for $K=0,1,2,3$.
6. Exactly twice. A 2 kHz sine sampled at 4 kHz gives samples all 0. The cosine gives $1,-1,1,\dots$. The cosine started 45° late gives $\pm0.7071$.

Where you’ll meet this

Every audio interface, phone and sensor board has a sample rate chosen with this rule. The rate has to be more than twice the highest frequency worth keeping, and a filter before the sampler enforces that limit. Page 10.4 is about those filters.

Two ideas continue from here. Reconstruction (10.3) shows what cutting out the middle copy does in time. Bandpass sampling (10.5) asks what happens to a signal whose frequencies sit in a high band, so that its copies can fit in the gaps. Later, The DTFT (12.2) writes the spectrum of the samples as a function of a digital frequency, and the copies reappear as its period.

Reference card