Orthogonalization

For a matrix , the hermitian conjugate or adjoint is denoted as :

If is real, then . A matrix is hermitian if , and unitary if .

1. Projectors

For , we define the projection of onto as:

The square matrix s.t. is called a projector:

• We must have (since reprojecting does nothing). So, it is idempotent. In fact, every idempotent matrix is a projector.
• is also a projector, and . This can be shown by the definition of idepmotent: . In fact, and are complementary projectors.
• defines complementary subspaces and . This is because, if ; hence, .
• If and are orthogonal, then is an orthogonal projector.

1.1 Orthogonal Projectors

is an orthogonal projector iff .

Proof

To prove that :

1. Assume .
2. Then let , and .
3. .
4. is , so .

To prove that :

1. Assume .
2. Choose an orthonormal basis for , s.t. spans and spans .
3. .
4. .
5. .
6. Hence, .

If is the orthonormal basis for ,then can be written as . Then is an orthogonal projector matrix onto the column space of , regardless of how was obtained, as long as its columns are orthonormal.

For a projector for a non-orthogonal basis :

1. Let for some .
2. .
3. .
4. .
5. .
6. .

2. Gram-Schmidt Process

We can revisit gram-schmidt with orthogonal projectors. We seek :

We can now define an algorithm:

1for(int j = 1; j < n; j++) {
2    u[j] = a[j];
3}
4
5for(int j = 1; j < n; j++) {
6    r[j][j] = norm(u[j]);
7    q[j] = u[j] / r[j][j];
8    for (int k = j + 1; k < n; k++) {
9        r[j][k] = conj(q[j]) * u[k];
10        u[k] = u[k] - r[j][k] * q[j];
11    }
12}

2.1 Lauchli Matrices

Consider the Lauchli matirx . We notice that for small :

• .
• .

This implies that the columns of are linearly dependent for small . We can use the classical Gram-Schmidt process to orthogonalize the columns of .

1. .
2. .
3. .