Let us begin Math 31BH by reviewing some aspects of Math 31AH. Last quarter, you encountered the following definition.

**Definition 1:** A **Euclidean space** is a pair consisting of a finite-dimensional real vector space together with a scalar product on that space.

Equivalently, a Euclidean space is a finite-dimensional real inner product space.

Given a Euclidean space , Math 31AH is largely about the associated vector space whose elements are linear operators

(The “End” here comes from the word endomorphism, which is a more general term used to refer to a structure-preserving function from some object to itself). Every linear operator on is completely determined by its action on any basis of , which gives us a concrete description of as a matrix . The form of this matrix depends heavily on the basis , and one of the main goals of Math 31AH was to treat the following problem: given an operator , find a basis of such that is as simple as possible.

It turns out that this problem has a lot to do with how the operator interacts with the scalar product on . You will recall that associated to every is another operator , called the adjoint of , defined by the condition that

Another way to describe the relationship between and is by saying that, for any basis of the matrix is the transpose of the matrix , and vice versa, and indeed sometimes is referred to as the transpose of .

**Definition 2:** An operator is said to be **selfadjoint** (or **symmetric**) if .

We denote by the subset of consisting of symmetric operators.

**Proposition 1:** is a subspace of .

*Proof:* The scalar product is bilinear, i.e. we have

and from this it follows (why?) that the adjoint is a linear operator on linear operators: we have

In particular, if are symmetric operators, we have that

which says that is closed under taking linear combinations, i.e. it is a subspace of .

Q.E.D.

We now come to what is arguably the central result of Math 31AH, the Spectral Theorem.

**Theorem 1:** Given a symmetric operator , there exists an orthonormal basis of such that

,

where are real numbers.

The orthonormal vectors are eigenvectors of the symmetric operator , and the numbers are its eigenvalues.

Once the Spectral Theorem is known, Math 31AH tends to devolve into a veritable orgy of diagonalization in which one is compelled to find the eigenvalues of all manner of particular symmetric operators. Diagonalization is an important skill which has many applications in science and engineering, from quantum mechanics to data science, but it is not the perspective taken in Math 31BH.

To elaborate, the Spectral Theorem gives us a function

which sends a symmetric operator on to the list of its eigenvalues arranged in weakly decreasing order, which is a vector in ,

In Math 31AH, a basic problem is: given , compute . That is, one wants to calculate the output of the function for a given input. In Math 31BH, we would like to analyze the function itself and see what kind of a function it is — what can we say about the function which sends a symmetric matrix to its eigenvalues?

First of all, it is important to understand that the function is non-linear: given , it is typically not the case that , i.e. the eigenvalue vector of is usually not simply the sum of the eigenvalue vector of and the eigenvalue vector of . This does occur if and happen to have a common set of eigenvectors, a situation which is equivalent to saying that they commute, , and that is atypical behavior. So is not a linear transformation, and even though this function arises from the context of linear algebra, we need to go beyond linear algebra to understand it.

Probably most natural question to be addressed is that of continuity: if two symmetric operators are close together, are their eigenvalues also close together? Before we can answer this question, we need to formulate it precisely, and to do this we will briefly leave the world of vector spaces in order to axiomatize the notion of distance.

**Definition 2:** A **metric space** is a pair consisting of a set together with a function

which has the following properties:

- for all , with equality if and only if ;
- for all ;
- for all .

The function is referred to as a distance function or metric on , and indeed the three axioms above are chosen so that has all the features we intuitively associate to the notion of distance: the first axiom says that the distance between two points cannot be negative, and is zero if and only if the two points are the same point; the second axiom says that the distance from home to work is the same as the distance from work to home; the third says that the shortest distance between to points is a straight line. The notion of distance is all we need to define the concepts of limit and continuity.

**Defintion 2:** Let and ) be metric spaces, and let be a function. Given points and , we say that the **limit** of as approaches is , written

if the following holds: for any , there exists a corresponding such that

We say that is **continuous** at if the above holds with . We say that is continuous on a set if it is continuous at every point .

In order to get a feel for the above definition, you should try to prove a familiar result from calculus in this general context: the composition of two continuous functions is continuous. Here is the precise formulation of this result.

**Proposition 1:** Let , and be metric spaces, and let

be functions such that is continuous at and is continuous at Then, the composite function is continuous at .

*Proof:* Try it! If you aren’t able to write down a proof of this theorem, go back to Definition 2 and read it again. Iterate this procedure as many times as necessary, and don’t feel bad if a few iterations are required.

Q.E.D.

We are now quite close to having the conceptual tools needed to meaningfully ask whether the function which sends a symmetric operator to its eigenvalues is continuous. The remaining conceptual step is to understand that the scalar product on a Euclidean space gives us a very natural metric. To see this, let us take an intermediate step.

**Definition 3:** Given a vector space , a** norm** on is a function

which has the following properties:

- We have for all , with equality if and only if is the zero vector;
- We have for all scalars and vectors , where is the usual absolute value function on ;
- We have for all .

A **normed vector space** is a pair consisting of a vector space together with a norm on that space.

Reading through the norm axioms, it is apparent that they are chosen to abstract the features of the familiar absolute value function on the real numbers , which has all of these features. This is relevant because the distance between is the absolute value of their difference,

**Proposition 2:** If is a norm on then defines a metric on .

*Proof:* We have the prove that satisfies the metric axioms. The first distance axiom follows immediately from the first norm axiom. For the second, we have

where we made use of the second norm axiom. Finally, for any three vectors , we have

where we made use of the third norm axiom.

Q.E.D.

In view of Proposition 2, we will have found a metric on as soon as we’ve found a norm. The recipe to get a norm out of the scalar product is inspired by the familiar relation between the absolute value and the dot product in the most familiar of Euclidean spaces, :

First we need the following important lemma.

**Lemma** **1 **(Cauchy-Schwarz inequality): In any Euclidean space, the following inequality holds for every two vectors: we have

with equality if and only if and are linearly dependent.

*Proof: *The proof is not difficult, and actually it’s a beautiful argument involving the quadratic formula, which you’ve known for many years. However it’s a bit long, and I don’t want to break up our flow — so look it up! I’ve provided you with a convenient link above.

Q.E.D.

**Proposition 3:** If is a scalar product on then defines a norm on .

*Proof:* We have to check that the norm axioms are satisfied. The first norm axiom follows immediately from the first scalar product axiom, namely that with equality holding only for the zero vector. For the second norm axiom, we have

Last is the triangle inequality, and to verify this property we use the Cauchy-Schwarz inequality. We have

which gets us to

so that taking the square root on both sides we obtain the triangle inequality,

Q.E.D.

So, is the function which sends a symmetric matrix to its eigenvalues continuous? From the above developments, we are almost at the point where we can meaningfully ask this question: all that remains is to find a metric on the vector space , which reduces to finding a norm on , which in turn reduces to finding a scalar product on . In fact, we will define a scalar product on all of , and the subspace will inherit this scalar product.

The construction of a scalar product on uses the concept of trace, which you encountered in Math 31AH and which we now review. Let be an orthonormal basis of , and define the trace relative to to be the function

given by

which is simply the sum of the diagonal elements of the matrix

**Proposition 3:** For any operator , we have

and for any pair of operators , we have

for all , and

*Proof:* By definition of the adjoint, we have

where symmetry of the scalar product on was used in the second to last equality.

The second identity says that is a linear function from to , and this follows immediately from the definition of together with the bilinearity of the scalar product on (work this out to make sure you understand why!).

For the third identity, using Problem 1 in Assignment 1 we have

and likewise

Thus, the -term in the expansion of coincides with the -term in the expansion of , and since these expansions run over all pairs , they are in fact the same sum.

Q.E.D.

**Theorem 2: **The function is a scalar product on .

*Proof: *We have to check the scalar product axioms; they all follow directly form the properties of the trace established above. For non negativity, we have

This is a sum of squares, hence it is nonnegative, and equal to zero if and only if all of its terms are equal to zero. But if for all then is the zero operator (make sure you understand why).

For symmetry, we have

.

Finally, for (bi)linearity we have

Q.E.D.

In view of Proposition 3, Theorem 2 immediately gives us the following.

**Corollary 1:** The function defines a norm on .

The above norm on operators is called the Frobenius norm, and it is very important in matrix analysis and data science. The Frobenius norm is defined using the trace, and the since our definition of the trace depends on a choice of orthonormal basis , so does our definition of the Frobenius norm. In fact, this dependence is illusory.

**Proposition 4:** For any orthonormal basis of , the function defined by

coincides with .

*Proof:* the linear transformation defined by for is invertible — it’s inverse transforms the basis back into the basis , and moreover , i.e. is an orthogonal transformation of . For any operator , we have

Q.E.D.

In view of the above, we may simply refer to “the trace,” and write for this function without specifying a basis.

Now that has been promoted from a vector space to a Euclidean space, the function which sends a symmetric matrix to its ordered list of eigenvalues is a function between Euclidean spaces, and we can legitimately ask if this function is continuous. Next lecture, we will answer this question.