Let be an -dimensional vector space equipped with a scalar product Recall from Lecture 16 that an operator is said to be selfadjoint (or symmetric) if
Also recall from Lecture 18 that is said to be semisimple if there exists a basis of consisting of eigenvectors of The goal of this lecture is to prove the following cornerstone result in linear algebra.
Theorem 1 (Spectral Theorem for selfadjoint operators): If is selfadjoint, then it is semisimple.
The proof of this important theorem occupies the remainder of this lecture. It is a constructive argument that builds an eigenbasis for one vector at a time. A nice feature of the construction is that the eigenbasis it outputs is an orthonormal basis of
Let us begin with an important observation on a special subspecies of selfadjoint operators.
Definition 1: A selfadjoint operator is said to be nonnegative if the associated quadratic form is nonnegative, i.e. if the function defined by
satisfies for all
Any nonnegative selfadjoint operator has the property that membership in its kernel is certified by vanishing of
Lemma 1: If is a nonnegative selfadjoint operator, then if and only if
Proof: One direction of this equivalence is obvious: if then
The proof of the converse statement is similar to the proof of the Cauchy-Schwarz inequality. More precisely, suppose that and let be any number and let be an arbitrary vector. We have
Using the definition of together with the fact that is selfadjoint, this simplifies to
and since this further simplifies to
Now, as a function of the righthand side of this equation is a parabola, and since this parabola is upward=opening. Moreover, since the lefthand side satisfies the lowest point of this parabola cannot lie below the line and this forces
But the vector was chosen arbitrarily, so the above equation holds for any in particular We thus have
which means that i.e.
— Q.E.D.
Now, let be any selfadjoint operator. We are going to use the Lemma just established to prove that admits an eigenvector ; the argument even gives a description of the corresponding eigenvalue
Consider the unit sphere in the Euclidean space i.e. the set
of all vectors of length The quadratic form is a continuous function, and hence by the Extreme Value Theorem the minimum value of on the sphere,
does indeed exist, and is moreover achieved at a vector at which the minimum is achieved, i.e.
Theorem 2: The minimum of on the unit sphere is an eigenvalue of and the minimizer lies in the eigenspace
Proof: By definition of as the minimum value of we have that
Since for any the above inequality can be rewritten as
But actually, this implies that
since every vector in is a nonnegative scalar multiple of a vector of unit length (make sure you understand this). We thus have that
This says that the selfadjoint operator is nonnegative. Moreover, we have that
Thus, by Lemma 1, we have that meaning that
or equivalently
— Q.E.D.
Theorem 2 has established that an arbitrary selfadjoint operator has an eigenvector. However, this seems to be a long way from Theorem 1, which makes the much stronger assertion that has linearly independent eigenvectors. In fact, the distance from Theorem 2 to Theorem 1 is not so long as it may seem. To see why, we need to introduce one more very important concept.
Defintion 2: Let be a linear operator, and let be a subspace of We say that is invariant under if
The meaning of this definition is that if is invariant under then may be considered as a linear operator on the smaller space i.e. as an element of the algebra
Let us adorn the eigenvalue/eigenvector pair produced by Theorem 2 with a subscript, writing this pair as Consider the orthogonal complement of the line spanned by i.e. the subspace of given by
Proposition 1: The subspace is invariant under
Proof: We have to prove that if is orthogonal to the eigenvector of then so is This follows easily from the fact that is selfadjoint:
— Q.E.D.
The effect of Proposition 1 is that we may consider as a selfadjoint operator defined on the -dimensional subspace But this means that we can simply apply Theorem 2 again, with replacing We will then get a new eigenvector/eigenvalue pair where
is the minimum value of on the unit sphere in the Euclidean space and is a vector at which the minimum is achieved,
By construction, is a unit vector orthogonal to so that in particular is a linearly independent set in Moreover, we have that since is a subset of
