Math 31AH: Lecture 19

Let \mathbf{V} be an n-dimensional vector space equipped with a scalar product \langle \cdot,\cdot \rangle. Recall from Lecture 16 that an operator A \in \mathrm{End}\mathbf{V} is said to be selfadjoint (or symmetric) if

\langle \mathbf{v},A\mathbf{w} \rangle = \langle A\mathbf{v},\mathbf{w}\rangle \quad \forall\ \mathbf{v},\mathbf{w} \in \mathbf{V}.

Also recall from Lecture 18 that A \in \mathrm{End}\mathbf{V} is said to be semisimple if there exists a basis of \mathbf{V} consisting of eigenvectors of A. The goal of this lecture is to prove the following cornerstone result in linear algebra.

Theorem 1 (Spectral Theorem for selfadjoint operators): If A \in \mathrm{End}\mathbf{V} is selfadjoint, then it is semisimple.

The proof of this important theorem occupies the remainder of this lecture. It is a constructive argument that builds an eigenbasis for A one vector at a time. A nice feature of the construction is that the eigenbasis it outputs is an orthonormal basis of \mathbf{V}.

Let us begin with an important observation on a special subspecies of selfadjoint operators.

Definition 1: A selfadjoint operator B \in \mathrm{End}\mathbf{V} is said to be nonnegative if the associated quadratic form is nonnegative, i.e. if the function defined by

Q_B(\mathbf{v}) := \langle \mathbf{v},B \mathbf{v} \rangle, \quad v \in \mathbf{V},

satisfies Q_B(\mathbf{v}) \geq 0 for all \mathbf{v} \in \mathbf{V}.

Any nonnegative selfadjoint operator B has the property that membership in its kernel is certified by vanishing of Q_B.

Lemma 1: If B \in \mathrm{End}\mathbf{V} is a nonnegative selfadjoint operator, then \mathbf{v} \in \mathrm{Ker} B if and only if Q_B(\mathbf{v})=0.

Proof: One direction of this equivalence is obvious: if \mathbf{v} \in \mathrm{Ker}B, then

Q_B(\mathbf{v}) = \langle \mathbf{v},B\mathbf{v}\rangle = \langle \mathbf{v},\mathbf{0} \rangle = 0.

The proof of the converse statement is similar to the proof of the Cauchy-Schwarz inequality. More precisely, suppose that Q_B(\mathbf{v})=0, and let t \in \mathbb{R} be any number and let \mathbf{w} \in \mathbf{V} be an arbitrary vector. We have

Q_B(\mathbf{v}+t\mathbf{w}) = \langle \mathbf{v}+t\mathbf{w},B\mathbf{v}+tB\mathbf{w}\rangle \\= \langle \mathbf{v},B\mathbf{v} \rangle + \langle \mathbf{v},tB\mathbf{w} \rangle + \langle t\mathbf{w},B\mathbf{v} \rangle + \langle t\mathbf{w},tB\mathbf{w} \rangle.

Using the definition of Q_B together with the fact that B is selfadjoint, this simplifies to

Q_B(\mathbf{v}+t\mathbf{w}) = Q_B(\mathbf{v}) + 2t\langle B\mathbf{v},\mathbf{w} \rangle + t^2Q_B(\mathbf{w}),

and since Q_B(\mathbf{v})=0 this further simplifies to

Q_B(\mathbf{v}+t\mathbf{w}) = 2t\langle B\mathbf{v},\mathbf{w} \rangle + t^2Q_B(\mathbf{w}).

Now, as a function of t \in \mathbb{R} the righthand side of this equation is a parabola, and since Q_B(\mathbf{w}) \geq 0 this parabola is upward=opening. Moreover, since the lefthand side satisfies Q_B(\mathbf{v}+t\mathbf{w}) \geq 0, the lowest point of this parabola cannot lie below the line t=0, and this forces

\langle B\mathbf{v},\mathbf{w} \rangle = 0.

But the vector \mathbf{w} was chosen arbitrarily, so the above equation holds for any \mathbf{w} \in \mathbf{V}, in particular \mathbf{w}=B\mathbf{v}. We thus have

\langle B\mathbf{v},B\mathbf{v}\rangle = \|B\mathbf{v}\|^2=0,

which means that B\mathbf{v}=\mathbf{0}, i.e. \mathbf{v} \in \mathrm{Ker}B.

— Q.E.D.

Now, let A \in \mathrm{End}\mathbf{V} be any selfadjoint operator. We are going to use the Lemma just established to prove that A admits an eigenvector \mathbf{e}; the argument even gives a description of the corresponding eigenvalue \lambda.

Consider the unit sphere in the Euclidean space \mathbf{V}, i.e. the set

S(\mathbf{V}) = \{ \mathbf{v} \in \mathbf{V} \colon \|\mathbf{v}\|=1\}

of all vectors of length 1. The quadratic form Q_A(\mathbf{v}) = \langle \mathbf{v},A\mathbf{v}\rangle is a continuous function, and hence by the Extreme Value Theorem the minimum value of Q_A on the sphere,

\lambda = \min\limits_{\mathbf{v} \in S(\mathbf{V})} Q_A(\mathbf{v}),

does indeed exist, and is moreover achieved at a vector \mathbf{e} \in S(\mathbf{V}) at which the minimum is achieved, i.e.

Q_A(\mathbf{e})=\lambda.

Theorem 2: The minimum \lambda of Q_A on the unit sphere is an eigenvalue of A, and the minimizer \mathbf{e} lies in the eigenspace \mathbf{V}_\lambda.

Proof: By definition of \lambda as the minimum value of Q_A, we have that

\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda \quad \forall \mathbf{v} \in S(\mathbf{V}).

Since \mathbf{v},\mathbf{v} \rangle =1 for any \mathbf{v} \in S_1(\mathbf{V}), the above inequality can be rewritten as

\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda\langle \mathbf{v},\mathbf{v} \rangle \quad \forall \mathbf{v} \in S_1(\mathbf{V}).

But actually, this implies that

\langle \mathbf{v},A\mathbf{v} \rangle \geq \lambda\langle \mathbf{v},\mathbf{v} \rangle \quad \forall \mathbf{v} \in \mathbf{V},

since every vector in \mathbf{V} is a nonnegative scalar multiple of a vector of unit length (make sure you understand this). We thus have that

\langle \mathbf{v},(A-\lambda I)\mathbf{v} \rangle \geq 0 \quad \forall v \in \mathbf{V}.

This says that the selfadjoint operator B:= A-\lambda I is nonnegative. Moreover, we have that

Q_B(\mathbf{e}) = \langle \mathbf{e},(A-\lambda I)\mathbf{e} \rangle = Q_A(\mathbf{e})-\lambda \langle \mathbf{e},\mathbf{e}\rangle = \lambda - \lambda = 0.

Thus, by Lemma 1, we have that \mathbf{e} \in \mathrm{Ker}(A-\lambda)I, meaning that

(A-\lambda I)\mathbf{e} = \mathbf{0}.

or equivalently

A\mathbf{e} = \lambda \mathbf{e}.

— Q.E.D.

Theorem 2 has established that an arbitrary selfadjoint operator A has an eigenvector. However, this seems to be a long way from Theorem 1, which makes the much stronger assertion that A has n linearly independent eigenvectors. In fact, the distance from Theorem 2 to Theorem 1 is not so long as it may seem. To see why, we need to introduce one more very important concept.

Defintion 2: Let T\in \mathrm{End}\mathbf{V} be a linear operator, and let \mathbf{W} be a subspace of \mathbf{V}. We say that \mathbf{W} is invariant under T if

T\mathbf{w} \in \mathbf{W} \quad \forall\ \mathbf{w} \in \mathbf{W}.

The meaning of this definition is that if \mathbf{W} is invariant under T, then T may be considered as a linear operator on the smaller space \mathbf{W}, i.e. as an element of the algebra \mathrm{End}\mathbf{W}.

Let us adorn the eigenvalue/eigenvector pair produced by Theorem 2 with a subscript, writing this pair as (\mathbf{e}_1,\lambda_1). Consider the orthogonal complement of the line spanned by \mathbf{e}_1, i.e. the subspace of \mathbf{V} given by

\mathbf{V}_2 = \{ \mathbf{v} \in \mathbf{V} \colon \langle \mathbf{v},\mathbf{e}_1 \rangle = 0\}.

Proposition 1: The subspace \mathbf{V}_2 is invariant under A.

Proof: We have to prove that if \mathbf{v} is orthogonal to the eigenvector \mathbf{e}_1 of A, then so is A\mathbf{v}. This follows easily from the fact that A is selfadjoint:

\langle A\mathbf{v},\mathbf{e}_1 \rangle = \langle \mathbf{v},A\mathbf{e}_1 \rangle = \langle \mathbf{v},\lambda_1\mathbf{e}_1 \rangle = \lambda_1 \langle \mathbf{v},\mathbf{e}_1 \rangle=0.

— Q.E.D.

The effect of Proposition 1 is that we may consider A as a selfadjoint operator defined on the (n-1)-dimensional subspace \mathbf{V}_2. But this means that we can simply apply Theorem 2 again, with \mathbf{V}_2 replacing \mathbf{V}. We will then get a new eigenvector/eigenvalue pair (\mathbf{e}_2,\lambda_1), where

\lambda_2 = \min\limits_{\mathbf{v} \in S(\mathbf{V}_2} Q_A(\mathbf{v})

is the minimum value of Q_A on the unit sphere in the Euclidean space \mathbf{V}_2, and e_2 \in S_(\mathbf{V}_2) is a vector at which the minimum is achieved,

Q_A(\mathbf{e}_2) = \lambda_2.

By construction, \mathbf{e}_2 is a unit vector orthogonal to \mathbf{e}_1, so that in particular \{\mathbf{e}_1,\mathbf{e}_2\} is a linearly independent set in \mathbf{V}. Moreover, we have that \lambda_1 \leq \lambda_2, since S(\mathbf{V}_2) is a subset of S(\mathbf{V}_1).

Lecture 19 coda

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s