Vector spaces and subspaces

Add two points of a flat plane, or stretch one. Watch where the results land: on the plane, or off it.

Staying inside

Take the plane of all points whose three coordinates add up to 0, such as $(1,-1,0)$ and $(0,1,-1)$. Add those two and you get $(1,0,-1)$. Its coordinates add up to 0 again, so the sum is still on the plane. Stretch $(1,-1,0)$ by 2 and you get $(2,-2,0)$, still on the plane.

That is no accident. If $x+y+z=0$ for one point and for another, then adding them adds two zeros, and scaling by $c$ gives $c\cdot0$. Both are 0. Nothing you do with adding and scaling can carry you off this plane.

Now move the plane. Take the points whose coordinates add up to 3, such as $(1,1,1)$. Add $(1,1,1)$ to itself and you get $(2,2,2)$, whose coordinates add up to 6. The sum has left the plane. In the picture at the top of the page, move the level $b$ away from 0 and watch the sum and the stretched copy fly off.

Subspaces

A set of vectors $S$ is a subspace when adding and scaling never take you out of it. For every $\mathbf{v}$ and $\mathbf{w}$ in $S$, and every number $c$:

I write ”$\mathbf{v}$ is in $S$” as $\mathbf{v}\in S$. A set with this property is closed under adding and scaling.

One consequence is worth seeing at once. Take any $\mathbf{v}$ in $S$ and scale it by $c=0$. The result is $\mathbf{0}$, so it must be in $S$. So every subspace contains the origin. That is why the plane only kept its points at $b=0$: a plane that misses the origin fails the test before you add anything.

Here are the subspaces of the plane $\mathbb{R}^2$:

• the origin alone, $\{\mathbf{0}\}$;
• any line through the origin;
• the whole plane.

In space, $\mathbb{R}^3$, add the planes through the origin. Every one of these is the span of some vectors, as in Linear combinations and span: a span is closed, because a combination of combinations is again a combination.

Beyond arrows

The test never asked what the vectors look like. It only asked that you can add them and scale them. So the same idea works for things that are not arrows.

Signals. A signal of three samples is a vector in $\mathbb{R}^3$, as in Vectors: component $x_1$ is the sample $x[0]$, since matrix and vector entries count from 1 and samples from 0. The signals whose three samples add up to zero are exactly the plane $x+y+z=0$ from the picture. Add two such signals, or turn one up, and the total stays 0.

Polynomials. The polynomials of degree at most 2, such as $1+2x-x^2$ or $3x^2$, can be added and scaled, and the result is again a polynomial of degree at most 2. They form a space with three numbers per member, just like $\mathbb{R}^3$.

Anything you can add and scale, with the usual rules of arithmetic, is called a vector space. $\mathbb{R}^n$ is the example to keep in mind. I won’t list the rules here; adding and scaling as you already know them is all this course uses.

The column space

Every matrix brings a subspace with it. In A matrix moves the plane, $\mathbf{A}\mathbf{x}$ was a combination of the columns of $\mathbf{A}$, with the entries of $\mathbf{x}$ as weights. Collect every output the matrix can produce:

This is the column space of $\mathbf{A}$: the span of its columns. It is a subspace of the output side, because outputs add and scale: $\mathbf{A}\mathbf{x}+\mathbf{A}\mathbf{y}=\mathbf{A}(\mathbf{x}+\mathbf{y})$ and $c\,\mathbf{A}\mathbf{x}=\mathbf{A}(c\,\mathbf{x})$.

The column space answers the question of Two pictures of a linear system once and for all: $\mathbf{A}\mathbf{x}=\mathbf{b}$ has a solution exactly when $\mathbf{b}\in C(\mathbf{A})$.

Take the $3\times2$ matrix

It sends $(x_1,x_2)$ to $(x_1,\,x_2,\,x_1+x_2)$. Every output has its third entry equal to the sum of the first two, so $C(\mathbf{B})$ is the plane $b_3=b_1+b_2$ in space.

Watch the two targets as the view turns. The point $(2,3,5)$ stays in the plane from every side. The point $(2,3,4)$ only seems to touch it from some angles. $\mathbf{B}\mathbf{x}=(2,3,5)$ has a solution, and $\mathbf{B}\mathbf{x}=(2,3,4)$ has none.

Worked example

1. A line through the origin. The line $y=2x$ is a subspace. Sum: $(1,2)+(2,4)=(3,6)$, and $6=2\cdot3$. Scaling: $3(1,2)=(3,6)$, on the line again.
2. A shifted line. $y=2x+1$ is not: at $x=0$ it gives $y=1$, so it misses $\mathbf{0}$.
3. A half-plane. $\{(x,y):x\ge0\}$ is not: $-1\cdot(1,0)=(-1,0)$ leaves it.
4. Two axes. The set where $xy=0$ is not: $(1,0)+(0,1)=(1,1)$, and $1\cdot1\ne0$.
5. The tilted plane. At $b=0$, $(1,-1,0)+(0,1,-1)=(1,0,-1)$ has total 0. At $b=3$, $(1,1,1)+(1,1,1)=(2,2,2)$ has total $6\ne3$.
6. Column space. $(2,3,5)\in C(\mathbf{B})$ with $\mathbf{x}=(2,3)$: $\mathbf{B}(2,3)=(2,\,3,\,2+3)$. The point $(1,2,3)$ is in it too, with $\mathbf{x}=(1,2)$. The point $(2,3,4)$ is not, because $2+3\ne4$.

Predict before you drag: at which level $b$ do sums stay on the plane? Only at $b=0$.

Where you’ll meet this

Band-limited audio. The signals with nothing above a given frequency form a subspace, as the callout above shows. A telephone channel keeps signals inside the one with nothing above about 3.4 kHz, and any mix of two voices stays inside it.

Removing the average. Subtracting a signal’s average moves it into the subspace of signals that add up to zero. The samples $(1,2,6)$ have average 3; take it away and you get $(-2,-1,3)$, which adds up to 0.

What a system can produce. If a system is a matrix, its column space is the set of every output it can make. A target outside it can’t be reached exactly by any input, and the best you can do is come close.

Reference card