Symmetric matrices

Squeeze a circle with a symmetric matrix. Watch it become an ellipse whose two axes meet at a right angle, whatever you do.

Perpendicular axes

A matrix like the one in the picture has a mirror pattern: the entry above the diagonal equals the entry below it. A matrix that equals its own transpose, from Transposes and permutations, is symmetric:

I write a symmetric matrix as $\mathbf{S}$. Every symmetric matrix in the picture turns the circle into an ellipse whose two axes meet at a right angle. The eigenvectors point along the axes, and the eigenvalues are the lengths of the semi-axes (their sizes, if one is negative). Take $s=2$:

Its trace is 7 and its determinant $10-4=6$, so its eigenvalues are 6 and 1, as on Finding eigenvalues. The eigenvectors are $(2,1)$, since $\mathbf{S}(2,1)=(12,6)=6\,(2,1)$, and $(-1,2)$, since $\mathbf{S}(-1,2)=(-1,2)$. And $(2,1)\cdot(-1,2)=-2+2=0$: a right angle.

Compare the matrix of the eigenvectors page, $\begin{bmatrix}1.5&1\\0.5&1\end{bmatrix}$, which is not symmetric. Its eigenvectors $(2,1)$ and $(1,-1)$ have $\cos\theta=1/\sqrt{10}=0.316$, so they meet at 71.57°, not 90°. Right angles are a gift of symmetry, not of every matrix.

Why the axes are perpendicular

Here is the reason, in one line. Let $\mathbf{S}\mathbf{v}_1=\lambda_1\mathbf{v}_1$ and $\mathbf{S}\mathbf{v}_2=\lambda_2\mathbf{v}_2$. Move $\mathbf{S}$ across the dot product with the transpose, $(\mathbf{S}\mathbf{v}_1)\cdot\mathbf{v}_2=\mathbf{v}_1\cdot(\mathbf{S}^\top\mathbf{v}_2)$, and use $\mathbf{S}^\top=\mathbf{S}$:

So $(\lambda_1-\lambda_2)\,\mathbf{v}_1\cdot\mathbf{v}_2=0$. When the two eigenvalues differ, the dot product must be 0, and the eigenvectors are perpendicular. When they are equal, every direction in their plane is stretched alike, so you may choose a perpendicular pair.

Real eigenvalues

A symmetric matrix never has complex eigenvalues. For $\begin{bmatrix}a&b\\b&c\end{bmatrix}$ the quantity under the square root in the formula of Finding eigenvalues is

It is a sum of squares, so the roots are real. The angle of the axes comes out neatly too: the eigenvector of the larger eigenvalue sits at the angle $\theta$ with

For $\mathbf{S}=\begin{bmatrix}5&2\\2&2\end{bmatrix}$, $\tan2\theta=4/3$, so $\theta$ is 26.57°, which is 0.4636 rad: the direction of $(2,1)$.

The spectral theorem

Scale each eigenvector to length 1. For $\mathbf{S}$ above, $\mathbf{q}_1=(2,1)/\sqrt5$ and $\mathbf{q}_2=(-1,2)/\sqrt5$. They are perpendicular and of length 1, so the matrix with them as columns,

has orthonormal columns, as in Orthonormal bases and Gram–Schmidt: $\mathbf{Q}^\top\mathbf{Q}=\mathbf{I}$, so its inverse is its transpose. Diagonalising as in Diagonalisation and matrix powers, with $\mathbf{Q}$ in place of $\mathbf{X}$, gives

This is the spectral theorem: every real symmetric matrix is a turn to its axes ($\mathbf{Q}^\top$), a stretch along each axis ($\boldsymbol{\Lambda}$), and a turn back ($\mathbf{Q}$). No inverse is needed, only a transpose.

A sum of stretches

Multiply $\mathbf{Q}\boldsymbol{\Lambda}\mathbf{Q}^\top$ out by columns times rows, as in Matrix multiplication:

Each $\mathbf{q}\mathbf{q}^\top$ is the projection onto one axis, from Projections. So a symmetric matrix projects a point onto each axis, stretches each piece by its eigenvalue, and adds the pieces.

With numbers:

Worked example

1. Eigenvalues, axes, factors. $\mathbf{S}=\begin{bmatrix}5&2\\2&2\end{bmatrix}$: trace 7, determinant 6, $\lambda=6$ and 1, eigenvectors $(2,1)$ and $(-1,2)$. The pieces add back to $\mathbf{S}$, as above. A check that needs no eigenvectors: the squares of the entries add up to $25+4+4+4=37$, and so do the squares of the eigenvalues, $36+1=37$.

2. Not symmetric, not perpendicular. The eigenvectors $(2,1)$ and $(1,-1)$ of $\begin{bmatrix}1.5&1\\0.5&1\end{bmatrix}$ meet at 71.57°.

3. A data cloud. Two measurements, each with spread 1 and correlation 0.8, have the covariance matrix $\begin{bmatrix}1&0.8\\0.8&1\end{bmatrix}$. Its eigenvalues are 1.8 along $(1,1)$ and 0.2 along $(1,-1)$. The ellipse that holds the cloud at one standard deviation has semi-axes $\sqrt{1.8}=1.342$ and $\sqrt{0.2}=0.447$, tilted at 45°.

4. Two masses on springs. Two equal masses are joined to two walls and to each other by three equal springs, with stiffness over mass $100\,\mathrm{s}^{-2}$. Their positions obey $\mathbf{x}''=-\mathbf{K}\mathbf{x}$ with

Its eigenvalues are 100, along $(1,1)$, and 300, along $(1,-1)$. A motion $\mathbf{x}=\mathbf{v}\cos\omega t$ along an eigenvector needs $\omega^2=\lambda$, so the masses swing together at $\omega=10$ rad/s, or 1.59 Hz, and against each other at $\omega=17.32$ rad/s, or 2.76 Hz.

Try these before you read the answers.

• A symmetric matrix has eigenvalues 3 and 1. At what angle do its eigenvectors meet? 90°.
• Eigenvalues and directions of $\begin{bmatrix}2&1\\1&2\end{bmatrix}$? 3 along $(1,1)$, and 1 along $(1,-1)$.
• Can a real symmetric $2\times2$ matrix have complex eigenvalues? No: $(a-c)^2+4b^2\ge0$.
• If $\mathbf{S}=\mathbf{Q}\boldsymbol{\Lambda}\mathbf{Q}^\top$, what is $\mathbf{S}^{-1}$? $\mathbf{Q}\boldsymbol{\Lambda}^{-1}\mathbf{Q}^\top$, when no eigenvalue is 0.

Where you’ll meet this

Principal axes. A cloud of data points, the stress inside a beam and the way a body resists being spun all have a symmetric matrix. Its eigenvectors are the natural axes of the problem, at right angles, and its eigenvalues say how much happens along each.

Vibration modes. Anything built from masses and springs, from a guitar body to a bridge, vibrates in modes that are the eigenvectors of a symmetric matrix, as in example 4. There the masses are equal, so the modes are perpendicular, and each can be excited and measured on its own.

Least squares. The matrix $\mathbf{A}^\top\mathbf{A}$ of Least squares is always symmetric, since $(\mathbf{A}^\top\mathbf{A})^\top=\mathbf{A}^\top\mathbf{A}$.

Reference card