Spectral Decomposition

1. Symmetric Matrices

Let with . So, is symmetric. Then:

1. All eigvenvalues of are real.
2. All eigenvectors corresponding to different eigvenvalues are orthogonal.
3. The geometric multiplicity of each eigvenvalue is equal to its algebraic multiplicity. Therefore, the matrix is diagonalisable.

(!) Multiplicity Reminder

The geometric multiplicity of is the dimension of the eigenspace of . The algebraic multiplicity of is the exponent of in the characteristic polynomial of , .

1.1 Proof of Proposition 1

Let with . Also, for some , let . Let us prove that :

• Hence, as .
• Hence, .

1.2 Proof of Proposition 2

Let with . Also, let be eigenvectors of corresponding to respectively. Let us prove that :

• Let and .
• As , we have
• Hence, as .

1.3 Consequence

So, when and then is diagonalisable and all its eigenvalues are real with eigenvectors corresponding to different eigenvalues being orthogonal.

We can assume that eigenvectors for , since if we have this by property 2, and if this follows from gram-schmidt. Hence, has unit vecotrs as its eigenvalues that are pairwise orthogonal.

2. Orthogonal Matrices

Properties of orthogonal matrices where :

1. for all .
2. preserves the scalar product .

(!) Proof of Property 1

• Hence, .

(!) Proof of Property 2

• .

3. Spectral Decomposition

Let where . Then, let where the columns of form an orthonormal basis for or an orthonormal set of eigenvectors of . Now, consider:

This is the spectral decomposition of .

3.1 Applications

Let be a matrix of data, where each row is a row of data. We can make this into a symmetrix matrix with , and .

4. Complex Matrices

Let . If have , then the propositions for real symmetric matrices are satisfied for :

• All eigenvalues of are real. (However, their eigenvectors may be complex.)
• Eigenvectors corresponding to different eigenvalues are orthogonal.
• The geometric multiplicity of each eigenvalue is equal to its algebraic multiplicity.

Hence, is diagonalisable.