Math 202A: Lecture 5

In Lecture 4 we covered the basic algebraic and geometric aspects of Hilbert space and Euclidean space. In this lecture we cover Descartes’ idea to encode points in such spaces as finite lists of numbers — the method of coordinates. A straightforward implementation of this idea only works when finitely many coordinates suffice, so we first need to introduce the notion of dimension, which is done using the concept of linear independence. This material will be mostly familiar from prior exposure to linear algebra, so several proofs are omitted (though I may fill them in later).

Let V be a vector space.

Definition 5.1. A finite set X \subset V is said to be linearly independent if the identity

\sum\limits_{x \in X} \alpha_xx=0_V

holds precisely when \alpha_x = 0 for each x \in X.

Remark that Definition 5.1 applies in the case where X is the empty set, which is a finite set of cardinality zero.

Definition 5.2. We say that V is finite-dimensional if there exists a nonnegative integer n such that that V contains a linearly independent set of cardinality n, but does not contain a linearly independent set of cardinality n+1. In this case the number n is called the dimension of V.

For example, the zero space V=\{0_V\} is a finite-dimensional vector space of dimension zero.

Now let V be a finite-dimensional vector space, fixed for the remainder of the lecture. Let X \subset V be a linearly independent set such that |X|=\dim V.

Theorem 5.3. (Coordinate Theorem) For every vector v \in V, there are scalars \alpha_x such that

v=\sum\limits_{x \in X} \alpha_x x.

Moreover, if also

v=\sum\limits_{x \in X} \beta_xx,

then \beta_x = \alpha_x for each x \in X.

A set X \subset V as in Theorem 5.3 is said to be a basis of V.

Now suppose V is equipped with a scalar product, i.e. it is either a Hilbert space (scalar field \mathbb{C}) or a Euclidean space (scalar field \mathbb{R}).

Theorem 5.4. (Gram-Schmidt) There exists a linearly independent set X \subset V of cardinality |X|=\dim V such that each x,y \in X satisfy

\langle x,y \rangle = \begin{cases} 1, \text{ if } x=y \\ 0, \text{ if }x \neq y \end{cases}.

A set X \subset V as in Theorem 5.4 is said to be an orthonormal basis of V.

Theorem 5.5. (Cartesian Coordinates) Let X \subset V be an orthonormal basis. Then, for ever v \in V we have

v = \sum\limits_{x \in X} \langle x,v \rangle x.

Proof: By the coordinate theorem, we have

v = \sum\limits_{y \in X} \alpha_yy,

and therefore

\langle x,v \rangle =\sum\limits_{y \in X} \alpha_y\langle x,y\rangle=\alpha_x.

-QED

The |X|=\dim V scalar products \langle x,v\rangle are called the Cartesian coordinates of v \in V relative to the orthonormal basis X.

Now suppose we choose an ordering (aka labeling, aka enumeration) x_1,\dots,x_n of X. Then, X is called an ordered orthonormal basis of V and the Cartesian coordinates of any vector v \in V relative to X can be stored as an ordered list,

[v] = \begin{bmatrix} \langle x_1,v \rangle \\ \vdots \\ \langle x_n,v\rangle \end{bmatrix},

i.e. as a spatial vector in \mathbb{F}^n where \mathbb{F} \in \{\mathbb{R},\mathbb{C}\}. In this way abstract vectors in a Euclidean space or Hilbert space can be represented as spatial vectors once an ordered orthonormal basis is chosen. However, for most purposes it is not necessary to choose an ordering on X and one may simply work directly with unordered Cartesian coordinates. For example, we have

\langle v,w \rangle = \sum\limits_{x \in X} \overline{\langle x,v\rangle}\langle x,w\rangle = \sum\limits_{x \in X}\langle v,x\rangle \langle x,w\rangle.

for every pair of vectors v,w \in V, and in particular

\|v\|^2 = \sum\limits_{x \in V} |\langle x,v\rangle|^2.

The use of Cartesian coordinates without imposing an ordering is a middle path between the asceticism of never using any coordinates and the indulgence of always using ordered coordinates. Another important computation is change of coordinates. Suppose that Y\subset V is a novel orthonormal basis generating its own Cartesian coordinates

v = \sum\limits_{y \in Y} \langle y,v\rangle y.

To express the novel coordinates we compute

v = \sum\limits_{y \in Y} \left\langle \sum\limits_{x \in X}\langle x,y\rangle x,v\right\rangle y = \sum\limits_{y \in Y} \left(\sum\limits_{x \in X} \overline{\langle x,y\rangle} \langle x,v\rangle \right)y.

Thus, by uniqueness of coordinates in a given basis, we conclude that

\langle y,v \rangle = \sum\limits_{x \in X} \overline{\langle x,y\rangle} \langle x,v\rangle =\sum\limits_{x \in X} \langle y,x\rangle\langle x,v\rangle

holds for every y \in V. This is how change-of-basis is done without ordering bases.

Definition 5.6. A function A colon V \to W from one Hilbert space to another is said to be linear if A(0_V)=0_W and

A(\alpha_1 v_1 + \alpha_2 v_2) = \alpha_1 Av_1 +\alpha_2v_2.

For some reason, linear functions between Hilbert spaces are called linear transformations; they are generally denoted by capital letters and the usual brackets surrounding the argument to the function are omitted.

Previously, we discussed functions between finite sets and showed how these can be encoded as binary matrices (“one-hot encoding”). A non-trivial Hilbert space is an uncountably infinite set and functions between such objects can be very complicated. However, linear functions between finite-dimensional Hilbert spaces are not much more complicated than functions between finite sets: they are encoded by matrices whose entries may be any elements of the ground field.

Let X \subset V and Y \subset W be two Hilbert spaces with specified orthonormal bases, and let A \colon V \to W be a linear transformation. For any v \in V, we have

Av = A\sum\limits_{x \in X} \langle x,v \rangle x=\sum\limits_{x \in X} \langle x,v \rangle Ax.

Thus, A is uniquely determined by the |X|=\dim V vectors Ax, x \in X, and for each of these we have

Ax = \sum\limits_{y \in Y} \langle y,Ax\rangle.

It follows that A is uniquely determined by the |Y||X|=(\dim W)(\dim V) scalar products

\langle y,Ax \rangle, \quad x \in X,\ y \in Y,

which we call the matrix elements of A relative to X and Y. If one wishes, orderings x_1,\dots,x_n and y_1,\dots,y_m of X and Y may be chosen and the matrix elements of A may be arranged into the m \times n matrix [A] with entries

[A]_{ij} = \langle y_j, Ax_i\rangle,

but for most purposes it is not necessary to do this and one can work directly with the matrix elements of A relative to two unordered orthonormal bases.

As an example, let us discuss change of basis. In the category of finite sets with functions f \colon X \to Y the only thing that can occur is a relabeling of the points of X and Y, which will change the one-hot encoding of f by permuting its rows and columns. For linear maps between Hilbert spaces V and W, in which X and Y are sitting as orthonormal bases, we may consider genuinely different orthonormal basis \tilde{X} \subset V and \tilde{Y} \subset W. We then have

\langle \tilde{x}, A \tilde{y}\rangle = \left\langle \sum\limits_{y \in Y} \langle y,\tilde{y}\rangle y,\sum\limits_{x \in X} \langle x,\tilde{x}\rangle Ax \right\rangle=\sum\limits_{y \in Y}\sum\limits_{x \in X} \langle \tilde{y},y \rangle \langle y,Ax \rangle \langle x,\tilde{x} \rangle,

a formula which gives the new matrix elements in terms of the old ones and the Cartesian coordinates of \tilde{X},\tilde{Y} relative to the original bases X,Y.

Now let us consider linear transformations A \colon V \to \tilde{V} and B \colon \tilde{v} \to W, where V,\tilde{V},W are Hilbert spaces with orthonormal bases X,\tilde{X},Y respectively. Then, the composition BA:=B \circ A is a function X \to Y, and you can (and should) check that it is a linear transformation. We compute the matrix elements of the composition transformation in terms of the matrix elements of its constituents as follows:

\langle y,BA x\rangle=\left\langle y,\sum\limits_{\tilde{x} \in \tilde{X}} \langle \tilde{x},Ax\rangle B\tilde{x} \right\rangle =\sum\limits_{\tilde{x} \in \tilde{X}} \langle y,B\tilde{x}\rangle \langle \tilde{x},Ax\rangle.

Published by Jonathan Novak

Mathematician at UC San Diego.

Leave a Reply