Let 
be any two finite-dimensional Hilbert spaces, and let 
be an arbitrary morphism.
Theorem 19.1 (Geometric SVD) We have orthogonal decompositions
where 
is the rank of 
, the spaces 
are lines, the numbers 
are positive, and the restrictions 
are of the form 
with 
an isometric isomorphism for 
In this theorem, “decomposition” really means two things: first, it refers to simultaneous orthogonal decompositions of the spaces 
and 
into orthogonal lines with chunky remainders 
and 
and second it refers to a corresponding decomposition of the transformation into a combination of line-to-line mappings and a terminal collapsing of the (possibly non-existent) chunk 
into the zero vector of 
The decomposition of may be written compactly as
where 
is the orthogonal projection of onto the subspace 
The best way to consider this operator decomposition is an orthogonal decomposition of in a Hilbert space of transformations, which makes the singular values 
the coordinates of .
More precisely, 
is itself a Hilbert space: letting 
and 
be orthonormal bases, we defined the Frobenius scalar product on by
,
or equivalently
This gives the Frobenius norm
or equivalently
We had a nice time proving early on, from first principles, that the Frobenius scalar product is in fact independent of the Cartesian coordinate systems in and used to construct it. Equivalently, if 
and 
are unitary operators then the Frobenius scalar product satisfies
Now let us consider Theorem 19.1 from the point of view of Hilbert space geometry in equipped with the Frobenius scalar product. Let us write the SVD of as
,
where 
Proposition 19.2. 
is an orthonormal set in 
Proof: Let be an ordered orthonormal basis adapted to the decomposition 
and let be an ordered orthonormal basis adapted to the decomposition 
This means that, for each 
we have
Equivalently, the transformation 
which acts on by first orthogonally projecting onto 
and then mapping this line isometrically onto the line 
is a rank one isometry.
Now compute the Frobenius norm of 
using the adapted basis 
Similarly, suppose 
and consider the scalar product
There are two mutually exclusive cases for the term in this sum corresponding to a given 
First case: 
lies on neither of the lines 
, so 
is the zero vector in Second case: lies on at least one of the lines so at least one of the vectors 
is the zero vector in 
Now, the rank 
of 
satisfies 
but we can extend the orthonormal set 
to an orthonormal basis, relative to the Frobenius scalar product. Then, the SVD simply says that the singular values of are its nonzero coordinates in this basis. In particular, we have the following
Proposition 19.3. For any 
we have
Proof: By Proposition 19.2, this is an instance of the Pythagorean theorem.
A difficult and interesting question is to try to understand how the singular values of a sum 
of two morphisms 
are are determined by the singular values of the summands and 
. More conceptually, this means that we want to define functionals 
on which send a morphism 
to its 
th largest singular value (so there are 
of these functionals). These functionals are nonlinear and it is difficult to say much about them in general. One thing we can say immediately is that
which is a consequence of the fact that 
is the operator norm of 
and indeed most statements that can be made are inequalities rather than identities. However, the following identity is an immediate consequence of the Hilbert space structure on
Proposition 19.4. For any we have
Proof: By Proposition 19.2, this is an instance of the Parallelogram Law.
