***Happy Thanksgiving – all problems here due 12/03, 23:59.***
In Lecture 19, we interpreted the SVD of a morphism 
in terms of the Hilbert space geometry of 
equipped with the Frobenius scalar product. Let us start “downstairs,” with the orthogonal decompositions
The left singular lines 
are orthogonal one-dimensional subspaces of 
and the right singular lines 
are orthogonal one-dimensional subspaces in 
The restriction of 
to 
for 
has the form 
with 
a line-to-line isometry and 
Corresponding to this, we have an “upstairs” decomposition
happening in the Hilbert space , where 
and 
is the orthogonal projection of 
onto the line 
In Lecture 19, we proved that 
is an orthonormal set wrt the Frobenius scalar product, hence it can be extended to an orthonormal basis 
of the Hilbert space such that
![[A]_{\{E_i\}}=\begin{bmatrix} \sigma_1 \\ \vdots \\ \sigma_r \\ 0 \\ \vdots \\ 0 \end{bmatrix}.](https://s0.wp.com/latex.php?latex=%5BA%5D_%7B%5C%7BE_i%5C%7D%7D%3D%5Cbegin%7Bbmatrix%7D+%5Csigma_1+%5C%5C+%5Cvdots+%5C%5C+%5Csigma_r+%5C%5C+0+%5C%5C+%5Cvdots+%5C%5C+0+%5Cend%7Bbmatrix%7D.&bg=FFFFFF&fg=000&s=0&c=20201002)
Thus, for any 
the vector 
defined by
has coordinates
![[A_q]_{\{E_i\}}=\begin{bmatrix} \sigma_1 \\ \vdots \\ \sigma_q \\ 0 \\ \vdots \\ 0 \end{bmatrix},](https://s0.wp.com/latex.php?latex=%5BA_q%5D_%7B%5C%7BE_i%5C%7D%7D%3D%5Cbegin%7Bbmatrix%7D+%5Csigma_1+%5C%5C+%5Cvdots+%5C%5C+%5Csigma_q+%5C%5C+0+%5C%5C+%5Cvdots+%5C%5C+0+%5Cend%7Bbmatrix%7D%2C&bg=FFFFFF&fg=000&s=0&c=20201002)
and is simply the orthogonal projection of the vector onto the coordinate space 
Remembering that 
are morphism and not just inert vectors, we ask: what is the significance of 
as a linear transformation from into 
?
Theorem (Eckart-Young-Mirsky): Among all rank 
morphisms in 
the transformation is an optimal approximation to 
More precisely, if 
is any morphism of rank , then 
in Frobenius norm.
At a recent conference, my colleague Jorge Garza-Vargas emphasized to me that this result is crucial in applications of linear algebra to data compression, and that I would be doing you the student a grave injustice if I skipped it. So we will now prove the EYM theorem via a geometric argument which hopefully gives some understanding of why this result matters in data compression.
First of all, the upstairs SVD of is by construction
so it is clear that the rank of is indeed 
Our task is to demonstrate that if is an arbitrary transformation of rank 
then
We will decompose this into a two-part optimization problem by first optimizing over all approximants with a fixed image, and then optimizing over the image. Thus, let 
be a point in the Grassmannian of -dimensional subspaces of
Problem 20.1. Show that the optimization problem
is solved by 
where 
is the orthogonal projection of onto the subspace 
The morphism 
is called the compression of into 
and by construction it has rank 
It remains now to optimize the functional
over 
To do this we will express 
in terms of the singular data of . Let 
and 
be ordered orthonormal bases adapted to the downstairs decompositions
This means that 
and 
are orthonormal bases of and such that
for 
which is summarized by saying that 
are left singular vectors for and 
are right singular vectors for . The remaining vectors 
in 
form an orthonormal basis of the kernel of in , while the remaining vectors 
in 
form an orthonormal basis for the complement of the image of in Now, to any subspace 
we associate the compression numbers 
defined as the norms
of the vectors obtained by projecting each right singular vector of onto Note that 
is in fact independent of which unit vector 
we sample from the line 
since any other unit sample from 
will be a scalar multiple of by a complex number of modulus one. Note that 
with nonnegativity due to the fact that these numbers are norms, and the upper bound due to the fact that they are norms of projections of unit vectors. In fact, there is one more constraint on the compression numbers of a -dimensional subspace.
Problem 20.2. Show that for any we have
Now we are ready to quantify compression error.
Problem 20.3. Show that
Let us write the error formula above as
Here we are seeing that the squared error in approximating by the compression 
is at most the Frobenius norm of 
which is the error we incur if we compress to zero.
Our task now is to maximize the part of the sum which reduces this worst-case error. This is a very simple optimization problem: maximize the quadratic form
subject to the constraints
Remembering that the coefficients of 
satisfy
our optimal strategy is to put as much weight as possible on the largest coefficients by taking
Problem 20.4. Finish the proof of the EYM theorem by showing that the compression numbers of 
realize the above maximizer, and that the corresponding compression is 
