Math 202A: Lecture 22

Let $A \in \mathrm{Hom}(V,W)$ be a morphism with left and right singular blocks

$V=V_1 \oplus \dots \oplus V_k \oplus \mathrm{Ker}(A) \quad\text{and}\quad W=W_1 \oplus \dots \oplus W_k \oplus \mathrm{Im}(A)^\perp,$

and let $\sigma_1>\dots>\sigma_k>0$ be the corresponding singular values: the restriction of $A$ to $V_i$ has the form $\sigma_iU_i,$ where $U_i \in \mathrm{Hom}(V_i,W_i)$ is an isometric isomorphism. As a decomposition of $A$ itself, we have

$A=\sigma_1U_1P_1 + \dots + \sigma_kU_kP_k,$

where $P_i$ is the orthogonal projection of $V$ onto the left singular block $V_i$ . We have seen previously that $\{U_1,P_1\dots,U_kP_k\}$ is an orthogonal set in $\mathrm{Hom}(V,W)$ with respect to the Frobenius scalar product: we have

$\langle U_iP_i,U_jP_j \rangle = \delta_{ij} \dim V_i = \delta_{ij} \dim W_i.$

In particular, we have the Frobenius norm

$\|A\|= \sum\limits_{i=1}^k \sigma_k^2 \dim V_i.$

Previously, we wrote this in terms of repeated singular values, meaning that we decomposed each $V_i$ and $W_i$ into a direct sum of orthogonal lines instead of incorporating the multiplicity factor $\dim V_i.$ The two ways of thinking about the SVD — blocks and lines — are both correct, but we have to keep in mind that the block decomposition is unique whereas the line decomposition is only unique when the number $k$ of singular values $\sigma_1> \dots > \sigma_k$ is equal to the rank of $A.$ In particular, if we want to use the SVD to define the adjoint, we should use the block version.

Definition 22.1. The adjoint of $A\in \mathrm{Hom}(V,W)$ is the morphism $A^* \in \mathrm{Hom}(W,V)$ defined by

$A^*=\sigma_1U_1^{-1}Q_1 + \dots + \sigma_kU_k^{-1}Q_k,$

where $Q_i \in \mathrm{End}(W)$ is the orthogonal projection of $W$ onto the right singular block $W_i.$

A particularly simple (but important) situation is when $A=U \in \mathrm{Hom}(V,W)$ comes to us as an isometric isomorphism. Then, the singular value decompositions of $V$ and $W$ are

$V=V \quad\text{and}\quad W=W,$

the singular value decomposition of $U$ is

$U=U,$

and $U^*=U^{-1}.$

It is clear from Definition 22.1 that the adjoint is an involution, $(A^*)^*=A.$ Geometrically, this just says that swapping left and right singular blocks twice is the same as not swapping them at all.

Now we consider the case where the source and target spaces of $A \in \mathrm{Hom}(V,W)$ coincide, meaning that $V=W$ and $\mathrm{Hom}(V,W)=\mathrm{Hom}(V,V)= \mathrm{End}(V).$ In this setting there are two further coincidences that could potentially occur: the left and right singular blocks of $A$ could coincide, $V_i=W_i,$ or $A$ could coincide with its adjoint, $A^*=A.$ In the first case we say that $A$ is normal, and in the second we say that $A$ is selfadjoint. A priori, it is not clear what the relationship between normality and selfadjointness might be. Of course, we already know the answer from prolonged exposure to matrix algebra.

Theorem 22.2. Selfadjoint implies normal.

I would like to give a geometric proof of the fact that selfadjointness,

$A=\sum\limits_{i=1}^k \sigma_i U_iP_i = \sum\limits_{i=1}^k \sigma_iU_i^{-1}Q_i=A^*$

implies normality,

$V_i=W_i, \quad 1 \leq i \leq k.$

This amounts to showing that

$\sum\limits_{I=1}^k \sigma_i(U_iP_i-U_i^{-1}Q_i)=0_{\mathrm{End}(V)}$

forces $P_i=Q_i$ in $\mathrm{End}(V)$ , for each $1 \leq i \leq k.$ I have not yet succeeded in finding such an argument, and while I plan to try again on my own time you may be getting tired of me constantly turning things upside down and inside out and generally treating linear algebra results we can look up as if they were open research problems. If so, you may have given up on Math 202A, which is fine with me provided you are working on the mathematics that matters to you. I mean this sincerely and will extend the following provision: if you have fallen behind on 202A homework sets because you are composing your mathematical opus, you can turn in that work for homework credit in 202A.

Look up the definition of a normal matrix in any textbook and you’ll get an algebraic characterization of normality which makes Theorem 22.2 obvious.

Theorem 22.3. $A$ is normal if and only if $A^*A=AA^*.$

Problem 22.1. Prove that selfadjoint implies normal. You have two options: either find the direct geometric argument I failed to produce, or prove the standard result Theorem 22.3 and deduce the claimed implication from it.

Now, on to the spectral theorem. Assume $A \in \mathrm{End}(V)$ is selfadjoint. Then, because $A$ is normal, its left and right singular blocks coincide. The SVD tells us that the restriction of $A$ to the singular block $V_i$ is

$A|_{V_i} = \sigma_iU_i,$

where $U_i \in \mathrm{End}(V_i)$ is an isometric automorphism of $V_i$ , aka a unitary operator on $V_i.$ The same reasoning applied to $A^*$ gives

$A^*|_{V_i} = \sigma_i U_i^{-1}=\sigma_iU_i^*.$

Since $A=A^*,$ we have

$\sigma_iU_i = \sigma_iU_i^*,$

and since $\sigma_i>0$ we can cancel it to get

$U_i^*=U_i.$

In summary, the SVD is reducing the study of selfadjoint operators to the study of unitary selfadjoint operators: we have shown that $A$ being selfdadjoint means that

$A=\sum\limits_{i=1}^k \sigma_iU_i,$

where $U_i \in \mathrm{End}(V_i)$ is both selfadjoint and unitary. So let us step back and consider the highly constrained problem of analyzing a selfajdoint unitary operator $U \in \mathrm{End}(V).$

Since $U$ is unitary, $U^*=U^{-1}$ , and since $U$ is selfadjoint, $U^*=U.$ Combining these gives $U=U^{-1},$ and we conclude that a selfadjoint unitary operator $U \in \mathrm{End}(V)$ satisfies

$U^2=I_V,$

where $I \in \mathrm{End}(V)$ is the identity operator on $V.$ Two obvious solutions to this equation are $U=I$ and latex $U=-I,$ and in fact all solutions are mixtures of these two. More precisely, let

$V^+ = \mathrm{Ker}(U-I) =\{v \in V \colon Uv=v\}$

and

$V^{-}=\mathrm{Ker}(U+I)=\{v \in V \colon Uv=-v\}.$

Theorem 22.4. We have $V=V^+ \oplus V^{-}.$

Note that it is *not* necessarily true that both $V^+$ and $V^-$ are nonzero subspaces, as the examples $U=I$ and $U=-I$ already show.

Now return to the general situation, where we know that $A \in \mathrm{End}(V)$ is selfadjoint, but we are not assuming it is also unitary. What we do know is that

$V = V_1 \oplus \dots \oplus V_k \oplus \mathrm{Ker}(A),$

where the restriction of $A$ to $V_i$ is $\sigma_iU_i$ with $U_i \in \mathrm{End}(V_i)$ selfadjoint and unitary. This gives us the decomposition

$V= \bigoplus_{i=1}^k V_i^+ \oplus V_i^-,$

where $A$ acts in $V_i^+$ as multiplication by $\sigma_i$ acts in $V_i^-$ as multiplication by $-\sigma_i.$ In words, the condition that $A \in \mathrm{End}(V)$ allows us to refine its SVD by decomposing each singular block $V_i$ into a positive part $V_i^+$ and a negative part $V_i^-,$ and since $V_i$ is not the zero space at least one of $V_i^+,V_i^-$ is nonzero.

Theorem 22.5. (Spectral Theorem) Given $A \in \mathrm{End}(V)$ selfadjoint, there exists an orthogonal decomposition

$V = E_1 \oplus \dots \oplus E_l \oplus \mathrm{Ker}(A)$

where $E_1,\dots,E_l$ are nonzero subspaces such that

$A|_{E_i} = \lambda_i I_{E_i},$

where $\lambda_1,\dots,\lambda_l$ are nonzero real numbers. Each $E_i$ is a signed singular block of $A,$ and each $\lambda_i$ is a signed singular value of $A.$

Math 202A: Lecture 22

Like this:

Published by Jonathan Novak

Leave a ReplyCancel reply

Share this:

Like this:

Published by Jonathan Novak

Leave a ReplyCancel reply

Discover more from Jonathan Novak