Numerical Computation

Lecture 9: Vector Norms and Matrix Condition Numbers

United International College

Review

• During our Last Class we covered:– Operation count for Gaussian Elimination, LU

Factorization– Accuracy of Matrix Methods– Readings: • Pav, section 3.4.1• Moler, section 2.8

Today

• We will cover:– Vector and Matrix Norms– Matrix Condition Numbers– Readings: • Pav, section 1.3.2, 1.3.3, 1.4.1• Moler, section 2.9

Vector Norms

• A vector norm is a quantity that measures how large a vector is (the magnitude of the vector).

• For a number x, we have |x| as a measurement of the magnitude of x.

• For a vector x, it is not clear what the “best” measurement of size should be.

• Note: we will use bold-face type to denote a vector. ( x )

Vector Norms

• Example: x = ( 4 -1 )– is the standard Pythagorean length of x. This

is one possible measurement of the size of x.

22 )1(4

x

Vector Norms

• Example: x = ( 4 -1 )– |4| + |-1| is the “Taxicab” length of x. This is another

possible measurement of the size of x.

x

Vector Norms

• Example: x = ( 4 -1 )– max(|4|,|-1|) is yet another possible measurement of

the size of x.

x

Vector Norms

• A vector norm is a quantity that measures how large a vector is (the magnitude of the vector).

• Definition: A vector norm is a function that takes a vector and returns a non-zero number. We denote the norm of a vector x by

The norm must satisfy:– Triangle Inequality:– Scalar: – Positive: ,and = 0 only when x is the zero vector.

• Our previous examples for vectors in Rn :• Manhattan• Euclidean• Chebyshev

• All of these satisfy the three properties for a norm.

Vector Norms

Vector Norms Example

• Definition: The Lp norm generalizes these three norms. For p > 0, it is defined on Rn by:

• p=1 L1 norm• p=2 L2 norm • p= ∞ L∞ norm

Vector Norms

Distance

• Class Practice: – Find the L2 distance between the vectors x = (1, 2,

3) and y = (4, 0, 1). – Find the L ∞ distance between the vectors x = (1,

2, 3) and y = (4, 0, 1).

Distance

• The answer depends on the application. • The 1-norm and ∞-norm are good whenever

one is analyzing sensitivity of solutions. • The 2-norm is good for comparing distances

of vectors.• There is no one best vector norm!

Which norm is best?

• In Matlab, the norm function computes the Lp norms of vectors. Syntax: norm(x, p)>> x = [ 3 4 -1 ];>> n = norm(x,2)n = 5.0990>> n = norm(x,1)n = 8>> n = norm(x, inf)n = 4

Matlab Vector Norms

• Definition: Given a vector norm the matrix norm defined by the vector norm is given by:

• Example:

Matrix Norms

• Example:

• What does a matrix norm represent? • It represents the maximum “stretching” that A

does to a vector x -> (Ax).

Matrix Norms

• || A || > 0 if A ≠ O• || c A || = | c| * ||A || if A ≠ O• || A + B || ≤ || A || + || B ||• || A B || ≤ || A || * ||B || • || A x || ≤ || A || * ||x ||

Matrix Norm Properties

• Multiplication of a vector x by a matrix A results in a new vector Ax that can have a very different norm from x.

• The range of the possible change can be expressed by two numbers,

• =||A||

• Here the max, min are over all non-zero vectors x.

Matrix

• Definition: The condition number of a nonsingular matrix A is given by: κ(A) = M/mby convention if A is singular (m=0) then κ(A) = ∞.

• Note: If we let Ax = y, then x = A-1 y and

Matrix Condition Number

111

1

max

1minmin

A

y

yAyA

y

x

xAm

• Theorem: The condition number of a nonsingular matrix A can also be given as: κ(A) = || A || * || A-1||

• Proof: κ(A) = M/m. Also, M = ||A|| and by the previous slide m = 1 / (||A-1 ||). QED

Matrix Condition Number

Properties of the Matrix Condition Number

• For any matrix A, κ(A) ≥ 1.• For the identity matrix, κ(I) = 1.• For any permutation matrix P, κ(P) =1.• For any matrix A and nonzero scalar c , κ(c A) = κ(A).• For any diagonal matrix D = diag(di),

κ(D) = (max|di|)/( min | di| )

What does the condition number tell us?

• The condition number is a good indicator of how close is a matrix to be singular. The larger the condition number the closer we are to singularity.

• It is also very useful in assessing the accuracy of solutions to linear systems.

• In practice we don’t really calculate the condition number, it is merely estimated , to perhaps within an order of magnitude.

Condition Number And Accuracy

• Consider the problem of solving Ax = b. Suppose b has some error, say b + δb. Then, when we solve the equation, we will not get x but instead some value near x, say x + δx.

A(x + δx) = b + δb• Then, A(x + δx) = b + δb

Condition Number And Accuracy

• Class Practice: Show:

Condition Number And Accuracy

• The quantity ||δb||/||b|| is the relative change in the right-hand side, and the quantity ||δx||/||x|| is the relative error caused by this change.

• This shows that the condition number is a relative error magnification factor. That is, changes in the right-hand side of Ax=b can cause changes κ(A) times as large in the solution.