Post on 14-Mar-2016
description
Numerical Computation
Lecture 9: Vector Norms and Matrix Condition Numbers
United International College
Review
• During our Last Class we covered:– Operation count for Gaussian Elimination, LU
Factorization– Accuracy of Matrix Methods– Readings: • Pav, section 3.4.1• Moler, section 2.8
Today
• We will cover:– Vector and Matrix Norms– Matrix Condition Numbers– Readings: • Pav, section 1.3.2, 1.3.3, 1.4.1• Moler, section 2.9
Vector Norms
• A vector norm is a quantity that measures how large a vector is (the magnitude of the vector).
• For a number x, we have |x| as a measurement of the magnitude of x.
• For a vector x, it is not clear what the “best” measurement of size should be.
• Note: we will use bold-face type to denote a vector. ( x )
Vector Norms
• Example: x = ( 4 -1 )– is the standard Pythagorean length of x. This
is one possible measurement of the size of x.
22 )1(4
x
Vector Norms
• Example: x = ( 4 -1 )– |4| + |-1| is the “Taxicab” length of x. This is another
possible measurement of the size of x.
x
Vector Norms
• Example: x = ( 4 -1 )– max(|4|,|-1|) is yet another possible measurement of
the size of x.
x
Vector Norms
• A vector norm is a quantity that measures how large a vector is (the magnitude of the vector).
• Definition: A vector norm is a function that takes a vector and returns a non-zero number. We denote the norm of a vector x by
The norm must satisfy:– Triangle Inequality:– Scalar: – Positive: ,and = 0 only when x is the zero vector.
• Our previous examples for vectors in Rn :• Manhattan• Euclidean• Chebyshev
• All of these satisfy the three properties for a norm.
Vector Norms
Vector Norms Example
• Definition: The Lp norm generalizes these three norms. For p > 0, it is defined on Rn by:
• p=1 L1 norm• p=2 L2 norm • p= ∞ L∞ norm
Vector Norms
Distance
• Class Practice: – Find the L2 distance between the vectors x = (1, 2,
3) and y = (4, 0, 1). – Find the L ∞ distance between the vectors x = (1,
2, 3) and y = (4, 0, 1).
Distance
• The answer depends on the application. • The 1-norm and ∞-norm are good whenever
one is analyzing sensitivity of solutions. • The 2-norm is good for comparing distances
of vectors.• There is no one best vector norm!
Which norm is best?
• In Matlab, the norm function computes the Lp norms of vectors. Syntax: norm(x, p)>> x = [ 3 4 -1 ];>> n = norm(x,2)n = 5.0990>> n = norm(x,1)n = 8>> n = norm(x, inf)n = 4
Matlab Vector Norms
• Definition: Given a vector norm the matrix norm defined by the vector norm is given by:
• Example:
Matrix Norms
• Example:
• What does a matrix norm represent? • It represents the maximum “stretching” that A
does to a vector x -> (Ax).
Matrix Norms
• || A || > 0 if A ≠ O• || c A || = | c| * ||A || if A ≠ O• || A + B || ≤ || A || + || B ||• || A B || ≤ || A || * ||B || • || A x || ≤ || A || * ||x ||
Matrix Norm Properties
• Multiplication of a vector x by a matrix A results in a new vector Ax that can have a very different norm from x.
• The range of the possible change can be expressed by two numbers,
• =||A||
• Here the max, min are over all non-zero vectors x.
Matrix
• Definition: The condition number of a nonsingular matrix A is given by: κ(A) = M/mby convention if A is singular (m=0) then κ(A) = ∞.
• Note: If we let Ax = y, then x = A-1 y and
Matrix Condition Number
111
1
max
1minmin
A
y
yAyA
y
x
xAm
• Theorem: The condition number of a nonsingular matrix A can also be given as: κ(A) = || A || * || A-1||
• Proof: κ(A) = M/m. Also, M = ||A|| and by the previous slide m = 1 / (||A-1 ||). QED
Matrix Condition Number
Properties of the Matrix Condition Number
• For any matrix A, κ(A) ≥ 1.• For the identity matrix, κ(I) = 1.• For any permutation matrix P, κ(P) =1.• For any matrix A and nonzero scalar c , κ(c A) = κ(A).• For any diagonal matrix D = diag(di),
κ(D) = (max|di|)/( min | di| )
What does the condition number tell us?
• The condition number is a good indicator of how close is a matrix to be singular. The larger the condition number the closer we are to singularity.
• It is also very useful in assessing the accuracy of solutions to linear systems.
• In practice we don’t really calculate the condition number, it is merely estimated , to perhaps within an order of magnitude.
Condition Number And Accuracy
• Consider the problem of solving Ax = b. Suppose b has some error, say b + δb. Then, when we solve the equation, we will not get x but instead some value near x, say x + δx.
A(x + δx) = b + δb• Then, A(x + δx) = b + δb
Condition Number And Accuracy
• Class Practice: Show:
Condition Number And Accuracy
• The quantity ||δb||/||b|| is the relative change in the right-hand side, and the quantity ||δx||/||x|| is the relative error caused by this change.
• This shows that the condition number is a relative error magnification factor. That is, changes in the right-hand side of Ax=b can cause changes κ(A) times as large in the solution.