Linear differential equations

At every point of the plane, a matrix sets a velocity. Watch points flow along it: into the origin, or in and back out.

A velocity at every point

Many systems change at a rate set by where they are: a cooling cup, a swinging spring, a circuit. With two quantities $\mathbf{u}=(u_1,u_2)$ changing together, and each rate a fixed mix of both, the rule is

At every point $\mathbf{u}$ of the plane, $\mathbf{A}\mathbf{u}$ is a velocity: the direction and speed of the point at that moment. The picture draws those velocities as arrows, and a solution is a path that follows the arrows from its start.

Along an eigenvector

Most arrows point off at an angle, so paths curve. But on an eigenvector line, $\mathbf{A}\mathbf{u}=\lambda\mathbf{u}$ points along the line itself, so a point there stays on it. Write $\mathbf{u}(t)=c(t)\,\mathbf{v}$. The equation becomes $c'(t)\,\mathbf{v}=\lambda\,c(t)\,\mathbf{v}$, an ordinary equation for one number:

So along an eigenvector the point grows like $e^{\lambda t}$ when $\lambda>0$ and dies away when $\lambda<0$.

The solution as a mix

Split the starting point along the eigenvectors, $\mathbf{u}(0)=c_1\mathbf{v}_1+c_2\mathbf{v}_2$. Each part follows its own line, so

The two matrices in the picture are the matrix $\mathbf{A}=\begin{bmatrix}1.5&1\\0.5&1\end{bmatrix}$ of Eigenvectors, shifted: $\mathbf{A}_\text{node}=\mathbf{A}-3\mathbf{I}$ and $\mathbf{A}_\text{saddle}=\mathbf{A}-\mathbf{I}$. Subtracting $c\,\mathbf{I}$ keeps the eigenvectors $(2,1)$ and $(1,-1)$ and lowers each eigenvalue by $c$. So $\mathbf{A}_\text{node}$ has eigenvalues $-1$ and $-2.5$, and $\mathbf{A}_\text{saddle}$ has $1$ and $-0.5$.

The start $(3,0)$ is $(2,1)+(1,-1)$, so $c_1=c_2=1$. For the saddle:

At $t=1$ this is $(6.043,\;2.112)$. At $t=0$ its slope is $(2-0.5,\;1+0.5)=(1.5,1.5)$, which is $\mathbf{A}_\text{saddle}(3,0)$, as the equation demands.

For the node, $\mathbf{u}(t)=e^{-t}\,(2,1)+e^{-2.5t}\,(1,-1)$, which at $t=1$ is $(0.818,\;0.286)$.

Stability

The eigenvalues say where every path goes.

• Both negative (the node): every part dies away, and every path ends at the origin. The faster part, $e^{-2.5t}$, dies first, so paths slide in along the slow line, $(2,1)$.
• One positive, one negative (the saddle): the $(1,-1)$ part dies and the $(2,1)$ part grows. Paths come in along one line and leave along the other. Only a start exactly on the line of $(1,-1)$ reaches the origin.
• Both positive: every path runs away from the origin.

So the system is stable, every path settles to zero, exactly when every eigenvalue is negative. When eigenvalues are complex, as on Complex eigenvalues, the rule reads: every real part negative.

The matrix exponential

For one number, $u'=au$ is solved by $u(t)=e^{at}u(0)$. The same works for a matrix, with the exponential defined by the same series as for numbers:

Differentiate term by term and you get $\mathbf{A}\,e^{\mathbf{A}t}$, so $\mathbf{u}(t)=e^{\mathbf{A}t}\,\mathbf{u}(0)$ solves the equation. Every power in the series is a matrix power, so when $\mathbf{A}=\mathbf{X}\boldsymbol{\Lambda}\mathbf{X}^{-1}$, as in Diagonalisation and matrix powers,

where $e^{\boldsymbol{\Lambda}t}$ is diagonal with entries $e^{\lambda_kt}$. For the saddle, with the two projections $\mathbf{P}_1$ and $\mathbf{P}_2$ of that page,

At $t=1$ it is about $\begin{bmatrix}2.01&1.41\\0.70&1.31\end{bmatrix}$, and times $(3,0)$ it gives $(6.04,\;2.11)$, the point found above.

One more check: the determinant of $e^{\mathbf{A}t}$ is $e^{t\,\mathrm{tr}\,\mathbf{A}}$, the product of the $e^{\lambda_kt}$. For the saddle at $t=1$ that is $e^{0.5}=1.649$, the area by which the flow stretches the plane in one second.

A spring as a system

A damped spring obeys a second-order equation, for example $y''+3y'+2y=0$. Keep the position and the speed together, $\mathbf{u}=(y,y')$. Then $y'$ is the second entry, and $y''=-2y-3y'$, so

The matrix has trace $-3$ and determinant 2, so its eigenvalues are $-1$ and $-2$, with eigenvectors $(1,-1)$ and $(1,-2)$. Release the spring from $y(0)=1$ at rest: $(1,0)=2\,(1,-1)-(1,-2)$, so

Watch the curve: it sinks to 0 and never crosses it. Both eigenvalues are real and negative, so there is no swing: the spring is overdamped. At $t=1$ the position is $0.600$, and at $t=2$ it is $0.252$.

Worked example

1. The saddle. $\mathbf{u}(t)=e^{t}(2,1)+e^{-0.5t}(1,-1)$; at $t=1$, $(2e+e^{-0.5},\;e-e^{-0.5})=(6.043,\;2.112)$.

2. The node. $\mathbf{u}(t)=e^{-t}(2,1)+e^{-2.5t}(1,-1)$; at $t=1$, $(0.818,\;0.286)$. Both parts die, so the path ends at the origin.

3. The exponential. $e^{\mathbf{A}_\text{saddle}}\approx\begin{bmatrix}2.01&1.41\\0.70&1.31\end{bmatrix}$, with determinant $e^{0.5}=1.649$.

4. The spring. $y(t)=2e^{-t}-e^{-2t}$: $y(0)=1$, $y'(0)=-2+2=0$, $y(1)=0.600$, $y(2)=0.252$.

Try these before you read the answers.

• Eigenvalues $-1$ and $-3$: stable? Yes.
• Eigenvalues $-1$ and $0.2$: stable? No: one direction grows.
• Is $e^{\mathbf{A}}$ the exponential of each entry? No.
• In the node, which line do paths hug as they reach the origin? The slow one, $(2,1)$.

Where you’ll meet this

Circuits and machines. Every linear circuit or mechanism with several places to store energy, such as capacitors, inductors, masses and springs, obeys $\mathrm{d}\mathbf{u}/\mathrm{d}t=\mathbf{A}\mathbf{u}$ when left alone. Its eigenvalues are its natural rates.

Feedback. Whether a feedback loop settles or runs away is read from the real parts of the eigenvalues of its matrix.

Simulation. A computer that simulates such a system at a sample period $T_s$ steps the state by the fixed matrix $e^{\mathbf{A}T_s}$. That step is exact, not an approximation, and it is the idea behind Impulse invariance.

Reference card