Let be an
-dimensional vector space equipped with a scalar product
Recall from Lecture 16 that an operator
is said to be selfadjoint (or symmetric) if
Also recall from Lecture 18 that is said to be semisimple if there exists a basis of
consisting of eigenvectors of
The goal of this lecture is to prove the following cornerstone result in linear algebra.
Theorem 1 (Spectral Theorem for selfadjoint operators): If is selfadjoint, then it is semisimple.
The proof of this important theorem occupies the remainder of this lecture. It is a constructive argument that builds an eigenbasis for one vector at a time. A nice feature of the construction is that the eigenbasis it outputs is an orthonormal basis of
Let us begin with an important observation on a special subspecies of selfadjoint operators.
Definition 1: A selfadjoint operator is said to be nonnegative if the associated quadratic form is nonnegative, i.e. if the function defined by
satisfies for all
Any nonnegative selfadjoint operator has the property that membership in its kernel is certified by vanishing of
Lemma 1: If is a nonnegative selfadjoint operator, then
if and only if
Proof: One direction of this equivalence is obvious: if then
The proof of the converse statement is similar to the proof of the Cauchy-Schwarz inequality. More precisely, suppose that and let
be any number and let
be an arbitrary vector. We have
Using the definition of together with the fact that
is selfadjoint, this simplifies to
and since this further simplifies to
Now, as a function of the righthand side of this equation is a parabola, and since
this parabola is upward=opening. Moreover, since the lefthand side satisfies
the lowest point of this parabola cannot lie below the line
and this forces
But the vector was chosen arbitrarily, so the above equation holds for any
in particular
We thus have
which means that i.e.
— Q.E.D.
Now, let be any selfadjoint operator. We are going to use the Lemma just established to prove that
admits an eigenvector
; the argument even gives a description of the corresponding eigenvalue
Consider the unit sphere in the Euclidean space i.e. the set
of all vectors of length The quadratic form
is a continuous function, and hence by the Extreme Value Theorem the minimum value of
on the sphere,
does indeed exist, and is moreover achieved at a vector at which the minimum is achieved, i.e.
Theorem 2: The minimum of
on the unit sphere is an eigenvalue of
and the minimizer
lies in the eigenspace
Proof: By definition of as the minimum value of
we have that
Since for any
the above inequality can be rewritten as
But actually, this implies that
since every vector in is a nonnegative scalar multiple of a vector of unit length (make sure you understand this). We thus have that
This says that the selfadjoint operator is nonnegative. Moreover, we have that
Thus, by Lemma 1, we have that meaning that
or equivalently
— Q.E.D.
Theorem 2 has established that an arbitrary selfadjoint operator has an eigenvector. However, this seems to be a long way from Theorem 1, which makes the much stronger assertion that
has
linearly independent eigenvectors. In fact, the distance from Theorem 2 to Theorem 1 is not so long as it may seem. To see why, we need to introduce one more very important concept.
Defintion 2: Let be a linear operator, and let
be a subspace of
We say that
is invariant under
if
The meaning of this definition is that if is invariant under
then
may be considered as a linear operator on the smaller space
i.e. as an element of the algebra
Let us adorn the eigenvalue/eigenvector pair produced by Theorem 2 with a subscript, writing this pair as Consider the orthogonal complement of the line spanned by
i.e. the subspace of
given by
Proposition 1: The subspace is invariant under
Proof: We have to prove that if is orthogonal to the eigenvector
of
then so is
This follows easily from the fact that
is selfadjoint:
— Q.E.D.
The effect of Proposition 1 is that we may consider as a selfadjoint operator defined on the
-dimensional subspace
But this means that we can simply apply Theorem 2 again, with
replacing
We will then get a new eigenvector/eigenvalue pair
where
is the minimum value of on the unit sphere in the Euclidean space
and
is a vector at which the minimum is achieved,
By construction, is a unit vector orthogonal to
so that in particular
is a linearly independent set in
Moreover, we have that
since
is a subset of