# Math 31AH: Lecture 19

Let $\mathbf{V}$ be an $n$-dimensional vector space equipped with a scalar product $\langle \cdot,\cdot \rangle.$ Recall from Lecture 16 that an operator $A \in \mathrm{End}\mathbf{V}$ is said to be selfadjoint (or symmetric) if

$\langle \mathbf{v},A\mathbf{w} \rangle = \langle A\mathbf{v},\mathbf{w}\rangle \quad \forall\ \mathbf{v},\mathbf{w} \in \mathbf{V}.$

Also recall from Lecture 18 that $A \in \mathrm{End}\mathbf{V}$ is said to be semisimple if there exists a basis of $\mathbf{V}$ consisting of eigenvectors of $A.$ The goal of this lecture is to prove the following cornerstone result in linear algebra.

Theorem 1 (Spectral Theorem for selfadjoint operators): If $A \in \mathrm{End}\mathbf{V}$ is selfadjoint, then it is semisimple.

The proof of this important theorem occupies the remainder of this lecture. It is a constructive argument that builds an eigenbasis for $A$ one vector at a time. A nice feature of the construction is that the eigenbasis it outputs is an orthonormal basis of $\mathbf{V}.$

Let us begin with an important observation on a special subspecies of selfadjoint operators.

Definition 1: A selfadjoint operator $B \in \mathrm{End}\mathbf{V}$ is said to be nonnegative if the associated quadratic form is nonnegative, i.e. if the function defined by

$Q_B(\mathbf{v}) := \langle \mathbf{v},B \mathbf{v} \rangle, \quad v \in \mathbf{V},$

satisfies $Q_B(\mathbf{v}) \geq 0$ for all $\mathbf{v} \in \mathbf{V}.$

Any nonnegative selfadjoint operator $B$ has the property that membership in its kernel is certified by vanishing of $Q_B.$

Lemma 1: If $B \in \mathrm{End}\mathbf{V}$ is a nonnegative selfadjoint operator, then $\mathbf{v} \in \mathrm{Ker} B$ if and only if $Q_B(\mathbf{v})=0.$

Proof: One direction of this equivalence is obvious: if $\mathbf{v} \in \mathrm{Ker}B,$ then

$Q_B(\mathbf{v}) = \langle \mathbf{v},B\mathbf{v}\rangle = \langle \mathbf{v},\mathbf{0} \rangle = 0.$

The proof of the converse statement is similar to the proof of the Cauchy-Schwarz inequality. More precisely, suppose that $Q_B(\mathbf{v})=0,$ and let $t \in \mathbb{R}$ be any number and let $\mathbf{w} \in \mathbf{V}$ be an arbitrary vector. We have

$Q_B(\mathbf{v}+t\mathbf{w}) = \langle \mathbf{v}+t\mathbf{w},B\mathbf{v}+tB\mathbf{w}\rangle \\= \langle \mathbf{v},B\mathbf{v} \rangle + \langle \mathbf{v},tB\mathbf{w} \rangle + \langle t\mathbf{w},B\mathbf{v} \rangle + \langle t\mathbf{w},tB\mathbf{w} \rangle.$

Using the definition of $Q_B$ together with the fact that $B$ is selfadjoint, this simplifies to

$Q_B(\mathbf{v}+t\mathbf{w}) = Q_B(\mathbf{v}) + 2t\langle B\mathbf{v},\mathbf{w} \rangle + t^2Q_B(\mathbf{w}),$

and since $Q_B(\mathbf{v})=0$ this further simplifies to

$Q_B(\mathbf{v}+t\mathbf{w}) = 2t\langle B\mathbf{v},\mathbf{w} \rangle + t^2Q_B(\mathbf{w}).$

Now, as a function of $t \in \mathbb{R}$ the righthand side of this equation is a parabola, and since $Q_B(\mathbf{w}) \geq 0$ this parabola is upward=opening. Moreover, since the lefthand side satisfies $Q_B(\mathbf{v}+t\mathbf{w}) \geq 0,$ the lowest point of this parabola cannot lie below the line $t=0,$ and this forces

$\langle B\mathbf{v},\mathbf{w} \rangle = 0.$

But the vector $\mathbf{w}$ was chosen arbitrarily, so the above equation holds for any $\mathbf{w} \in \mathbf{V},$ in particular $\mathbf{w}=B\mathbf{v}.$ We thus have

$\langle B\mathbf{v},B\mathbf{v}\rangle = \|B\mathbf{v}\|^2=0,$

which means that $B\mathbf{v}=\mathbf{0},$ i.e. $\mathbf{v} \in \mathrm{Ker}B.$

— Q.E.D.

Now, let $A \in \mathrm{End}\mathbf{V}$ be any selfadjoint operator. We are going to use the Lemma just established to prove that $A$ admits an eigenvector $\mathbf{e}$; the argument even gives a description of the corresponding eigenvalue $\lambda.$

Consider the unit sphere in the Euclidean space $\mathbf{V},$ i.e. the set

$S(\mathbf{V}) = \{ \mathbf{v} \in \mathbf{V} \colon \|\mathbf{v}\|=1\}$

of all vectors of length $1.$ The quadratic form $Q_A(\mathbf{v}) = \langle \mathbf{v},A\mathbf{v}\rangle$ is a continuous function, and hence by the Extreme Value Theorem the minimum value of $Q_A$ on the sphere,

$\lambda = \min\limits_{\mathbf{v} \in S(\mathbf{V})} Q_A(\mathbf{v}),$

does indeed exist, and is moreover achieved at a vector $\mathbf{e} \in S(\mathbf{V})$ at which the minimum is achieved, i.e.

$Q_A(\mathbf{e})=\lambda.$

Theorem 2: The minimum $\lambda$ of $Q_A$ on the unit sphere is an eigenvalue of $A,$ and the minimizer $\mathbf{e}$ lies in the eigenspace $\mathbf{V}_\lambda.$

Proof: By definition of $\lambda$ as the minimum value of $Q_A,$ we have that

$\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda \quad \forall \mathbf{v} \in S(\mathbf{V}).$

Since $\mathbf{v},\mathbf{v} \rangle =1$ for any $\mathbf{v} \in S_1(\mathbf{V}),$ the above inequality can be rewritten as

$\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda\langle \mathbf{v},\mathbf{v} \rangle \quad \forall \mathbf{v} \in S_1(\mathbf{V}).$

But actually, this implies that

$\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda\langle \mathbf{v},\mathbf{v} \rangle \quad \forall \mathbf{v} \in \mathbf{V},$

since every vector in $\mathbf{V}$ is a nonnegative scalar multiple of a vector of unit length (make sure you understand this). We thus have that

$\langle \mathbf{v},(A-\lambda I)\mathbf{v} \rangle \geq 0 \quad \forall v \in \mathbf{V}.$

This says that the selfadjoint operator $B:= A-\lambda I$ is nonnegative. Moreover, we have that

$Q_B(\mathbf{e}) = \langle \mathbf{e},(A-\lambda I)\mathbf{e} \rangle = Q_A(\mathbf{e})-\lambda \langle \mathbf{e},\mathbf{e}\rangle = \lambda - \lambda = 0.$

Thus, by Lemma 1, we have that $\mathbf{e} \in \mathrm{Ker}(A-\lambda)I,$ meaning that

$(A-\lambda I)\mathbf{e} = \mathbf{0}.$

or equivalently

$A\mathbf{e} = \lambda \mathbf{e}.$

— Q.E.D.

Theorem 2 has established that an arbitrary selfadjoint operator $A$ has an eigenvector. However, this seems to be a long way from Theorem 1, which makes the much stronger assertion that $A$ has $n$ linearly independent eigenvectors. In fact, the distance from Theorem 2 to Theorem 1 is not so long as it may seem. To see why, we need to introduce one more very important concept.

Defintion 2: Let $T\in \mathrm{End}\mathbf{V}$ be a linear operator, and let $\mathbf{W}$ be a subspace of $\mathbf{V}.$ We say that $\mathbf{W}$ is invariant under $T$ if

$T\mathbf{w} \in \mathbf{W} \quad \forall\ \mathbf{w} \in \mathbf{W}.$

The meaning of this definition is that if $\mathbf{W}$ is invariant under $T,$ then $T$ may be considered as a linear operator on the smaller space $\mathbf{W},$ i.e. as an element of the algebra $\mathrm{End}\mathbf{W}.$

Let us adorn the eigenvalue/eigenvector pair produced by Theorem 2 with a subscript, writing this pair as $(\mathbf{e}_1,\lambda_1).$ Consider the orthogonal complement of the line spanned by $\mathbf{e}_1,$ i.e. the subspace of $\mathbf{V}$ given by

$\mathbf{V}_2 = \{ \mathbf{v} \in \mathbf{V} \colon \langle \mathbf{v},\mathbf{e}_1 \rangle = 0\}.$

Proposition 1: The subspace $\mathbf{V}_2$ is invariant under $A.$

Proof: We have to prove that if $\mathbf{v}$ is orthogonal to the eigenvector $\mathbf{e}_1$ of $A,$ then so is $A\mathbf{v}.$ This follows easily from the fact that $A$ is selfadjoint:

$\langle A\mathbf{v},\mathbf{e}_1 \rangle = \langle \mathbf{v},A\mathbf{e}_1 \rangle = \langle \mathbf{v},\lambda_1\mathbf{e}_1 \rangle = \lambda_1 \langle \mathbf{v},\mathbf{e}_1 \rangle=0.$

— Q.E.D.

The effect of Proposition 1 is that we may consider $A$ as a selfadjoint operator defined on the $(n-1)$-dimensional subspace $\mathbf{V}_2.$ But this means that we can simply apply Theorem 2 again, with $\mathbf{V}_2$ replacing $\mathbf{V}.$ We will then get a new eigenvector/eigenvalue pair $(\mathbf{e}_2,\lambda_1),$ where

$\lambda_2 = \min\limits_{\mathbf{v} \in S(\mathbf{V}_2} Q_A(\mathbf{v})$

is the minimum value of $Q_A$ on the unit sphere in the Euclidean space $\mathbf{V}_2,$ and $e_2 \in S_(\mathbf{V}_2)$ is a vector at which the minimum is achieved,

$Q_A(\mathbf{e}_2) = \lambda_2.$

By construction, $\mathbf{e}_2$ is a unit vector orthogonal to $\mathbf{e}_1,$ so that in particular $\{\mathbf{e}_1,\mathbf{e}_2\}$ is a linearly independent set in $\mathbf{V}.$ Moreover, we have that $\lambda_1 \leq \lambda_2,$ since $S(\mathbf{V}_2)$ is a subset of $S(\mathbf{V}_1).$