Introduction, Norms & Revision

A vector can be thought of as a direction and magnitude, or as the difference between two coordinates in an orthonormal coordinate system. Remember:

• .
• . (This only works in an orthonormal coordinate system.)

(!) Orthonormal Coordinate System

An orthonormal coordinate system is one where the basis vectors are mutually orthogonal and have unit length. The dot product of any two basis vectors is if they are different, and if they are the same.

For these notes, we assume an orthonormal basis on .

1. Vector Norms

A vector norm on is the generalization of the Euclidean length of a vector. It is a real valued map satisfying:

1. For any nonzero vector , .
2. For any scalar and , .
3. For any , (triangle inequality).

Thus, (from the 2nd axiom). The Euclidean length of vectors satisfies the above axioms.

A norm on induces a metric on by , which is translation invariant: .

1.1 L_p Norms

For , the norm of any vector is defined as:

Hence, we can calculate (using orthonormal basis):

• norm: .
• norm: . (Euclidean norm, invariant on changing the orthornormal basis.)
• norm: .

We can show that :

(!) Proposition

For any , :

• Choose s.t. . Hence, .
• By the Cauchy-Schwarz Equation (), .
• As , .

Corollary: For , we have constants s.t. . Hence, any two norms , are equivalent. This means any sequence of vectors in converging w.r.t. one norm also converge w.r.t. any other norm.

Significance: Suppose , then is bounded by : s.t. .

(!) Proposition

If for , then s.t. :

• .
• Also, all sequences of vectors in converging w.r.t. one norm also converge w.r.t. any other norm. This can be shown by: in implies that s.t. in .

(!) Proposition

as for all :

• .
• Hence, .
• As this sum contains , . Also, .
• Hence: .
• We can also see that .
• By the sandwhich theorem, we get as .

2. Matrix Norms

A matrix may have a norm which must satisfy the following axioms:

1. for any nonzero matrix .
2. for any scalar .
3. .
4. . (sub-multiplicative axiom)

The vector norms , , induce corresponding matrix norms. Take , and where . Then:

• .
• .
• .

2.1 Compatibility of Norms

A matrix norm is said to be consistent with vector norms and if . And, if then the matrix norm is compatible with : . The size of the output is bounded by the size of the input the size of the matrix.

It turns out that the vector norm is compatible with the matrix norm, the vector norm is compatible with the matrix norm, and the vector norm is compatible with the matrix norm. Why?

A matrix norm is subordinate to the vector norm if . The matrix norm is subordinate to the vector norm, the matrix norm is subordinate to the vector norm, and the matrix norm is subordinate to the vector norm.

(!) Proposition

Matrix satisfies :

3. Linear Maps

A vector space has a linear structure given by scalar multiplation and vector addition. Given vector spaces and , a linear map is linear if for all and .

Any basis of and gives you a unique way of representing linear w.r.t. the two basis. Let and be basis for and , respectively. If is a matrix representing , we have .

If matrix represents and represents , then represents . Also:

• .

The identity (basis change) function is represented by the identity matrix . For example, .

4. Complex Norms

In general, complex vector spaces are isomorphic to , where all scalars are replaced with complex numbers. There is one exception: norms. With real values , the absolute value (norm) is . With complex values , the absolute value (norm) is . This translates to vector norms: for , .

For a vector , the conjugate is . Similarly, for a matrix , the conjugate is .