Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number •...
Transcript of Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number •...
![Page 1: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/1.jpg)
NLA Reading Group Spring ’13by Can Kavaklıoğlu
Lecture 12Conditioning and Condition Numbers
![Page 2: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/2.jpg)
Outline
• Condition of a problem• Absolute condition number• Relative condition number• Examples
• Condition of matrix-vector multiplication
• Condition number of a matrix• Condition of system of equations
![Page 3: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/3.jpg)
Notation
Problem: f: X →Y
normed vector spaceSome usually non-linear, continious function
Problem instance: combination of x∈X and f
![Page 4: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/4.jpg)
Problem Condition Types
Small perturbation in x
Small perturbation in f(x) Large perturbation in f(x)
well-conditioned ill-conditioned
1, 10, 100 10^6, 10^16
![Page 5: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/5.jpg)
Absolute Condition Number
Small perturbation in x
Assuming and are infinitesimal
![Page 6: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/6.jpg)
Absolute Condition Number
If f is differentiable, we can evaluate Jacobian of f at x
with equality at limit
||J(x)|| represents norm of J(x) induced by norms of X and Y
![Page 7: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/7.jpg)
Relative Condition Number
if f is differentiable,
![Page 8: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/8.jpg)
Examples
![Page 9: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/9.jpg)
Condition of Matrix-Vector Multiplication
Problem: compute Ax from input x with fixed
![Page 10: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/10.jpg)
Condition of Matrix-Vector Multiplication
If A is square and non-singular using
Loosen relative condition number to a bound independent of x
If A is not square use pseudoinverse A+
![Page 11: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/11.jpg)
Condition of Matrix-Vector Multiplication
![Page 12: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/12.jpg)
Condition number of A relative to norm ||•||
Condition Number of a Matrix
If A is singular
![Page 13: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/13.jpg)
Condition of a System of Equations
Fix b and perturb A, in problem:
![Page 14: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/14.jpg)
Condition of a System of Equations
![Page 15: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/15.jpg)
NLA Reading Group Spring ’13by Can Kavaklıoğlu
Lecture 13Floating Point Arithmetic
![Page 16: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/16.jpg)
Outline
• Limitations of Digital Representations
• Floating Point Number
• Machine Epsilon
• Floating Point Arithmetic
• Complex Floating Point Arithmetic
![Page 17: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/17.jpg)
Limitations of Digital Representations
Finite number of bits
Two limitations
• Precision: IEEE double between 1.79 x 10^308 and 2.23 x 10^-308
• Overflow / underflow
• Interval representation: IEEE interval [1 2]:
interval [2 4]:
gap size:
Finite subset of real/complex numbers
![Page 18: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/18.jpg)
Floating Point Number
F: subset of real numbers, including 0
β: base/radix
t: precision (23 single, 53 double precision - IEEE)
fraction or mantissa
integer in range
eexponent: arbitrary integer
Idelized system: ignores underflow and overflow. F is a countably infinite set and it is self similar: F= βF
![Page 19: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/19.jpg)
Machine Epsilon
Resolution of F:
IEEE single IEEE double
Rounding:
![Page 20: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/20.jpg)
Floating Point Arithmetic
Every operation of floating point arithmetic is exact up to a relative error of size at most machine epsilon
![Page 21: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/21.jpg)
Different Machine Epsilon andComplex Floating Point Arithmetic
• Some (very old) hardware may not support IEEE machine epsilon
• It may be possible to use a larger machine epsilon value
• Complex arithmetic is performed using two floating point numbers
• Machine epsilon needs to be adjusted
![Page 22: Lecture 12 Conditioning and Condition Numberscemgil/Courses... · • Relative condition number • Examples • Condition of matrix-vector multiplication • Condition number of](https://reader033.fdocuments.net/reader033/viewer/2022050513/5f9dca8fe2d5cd32881d418f/html5/thumbnails/22.jpg)
The end
thanks