Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.
-
Upload
wilfred-greer -
Category
Documents
-
view
222 -
download
6
Transcript of Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.
![Page 1: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/1.jpg)
Backward Thinking
Confessions of a Numerical Analyst
Keith Evan Schubert
![Page 2: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/2.jpg)
Simple Problem
Consider the problem ax=b
The resulting x value is
A1.50 1.50
1.01 0.990
b
3.00
2.00
x 1.00
1.00
![Page 3: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/3.jpg)
Simple Problem 2
Consider the problem ax=b
The resulting x value is
A1.5 1.5
1.0 0.99
b
3.0
2.0
x 2.0
0.0
![Page 4: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/4.jpg)
What’s Up?
The condition number (sensitivity to perturbations) is about 400.
A condition number of 1 is perfect. Perturbation is 0.01, so 0.01*400=4. Components of x can vary by this much!
![Page 5: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/5.jpg)
What Can We Do?
Rather than solve it the standard way• X=a\b
• X=(ATA)-1atb
Consider the following:• X=(ATA+i)-1atb =.01
Then:x
1.0
1.0
![Page 6: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/6.jpg)
Lucky Guess?
-1 -0.5 0 0.5 10
0.5
1
1.5
2
x 2
-1 -0.5 0 0.5 10
0.5
1
1.5
2
x 1
![Page 7: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/7.jpg)
Does It Always Work?
No Consider X0 Consider i
2 (i is singular value of A)
X± Picking the wrong value can get junk
![Page 8: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/8.jpg)
Skyline
Consider a 1 dimensional picture Use height instead of color Result looks like the silhouette of a city’s
skyline Have smog which blurs and softens Don’t know exactly how much blur Want to get sharp edges
![Page 9: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/9.jpg)
Getting Garbage
![Page 10: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/10.jpg)
Getting Improvement
![Page 11: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/11.jpg)
Why Backward? Forward errors
• Explicitly account for each error source• (X+1)(y+2)=xy+(y1+x2+12)
Backward errors• Check that my algorithm acting on data will give me
a solution that is “near” to the actual system acting on a nearby set of data
• I.E. My algorithm with good data should do about as well as a perfect calculation on ok data
•
error Ax bA x b
![Page 12: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/12.jpg)
Picture Please!
ActualData (x)
NearbyData (x*)
Perfect Calculations
My Algorithm
Inherent errorsin A
b
Errors due toalgorithm
b*
best
![Page 13: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/13.jpg)
Least Squares
Usually we don’t have an invertible matrix Need to find an estimated solution Criterion: minimize ||ax-b|| Normal equation
• ATA x = ATb
Solution• X = (ATA)-1atb
![Page 14: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/14.jpg)
Backward Error
Criterion: minimize ||Ax-b||/(||A|| ||x||+||b||) Normal Equations
•
Solution:•
ATA I xATb
A Ax b
x A x b
x ATA I 1ATb
![Page 15: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/15.jpg)
Non Convex
![Page 16: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/16.jpg)
Finding The Root
g A Ax b
x A x b
![Page 17: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/17.jpg)
Look At Critical Region
![Page 18: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/18.jpg)
Informal Algorithm
Get (A,b) svd(A) [u1 u2],,v
U1b b1
Use rootfinder (bisection, Newton, etc.) to get in [-n
2,0]
vT(2- I)-1 b1 x
![Page 19: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/19.jpg)
What You GetBlurred Image
![Page 20: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/20.jpg)
Least SquaresLeast Squares Solution
![Page 21: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/21.jpg)
Total Least SquaresTotal Least Squares Solution
![Page 22: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/22.jpg)
TikhonovTikhonov Solution
![Page 23: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/23.jpg)
Backward ErrorMin Max Backward Error Solution
![Page 24: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/24.jpg)
OriginalActual Image
![Page 25: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/25.jpg)
ComparisonActual Image Blurred Image
Least Squares Solution Tikhonov Solution
Degenerate Min Min Solution Min Max Solution
Min Max Backward Error Solution Total Least Squares Solution
![Page 26: Backward Thinking Confessions of a Numerical Analyst Keith Evan Schubert.](https://reader030.fdocuments.net/reader030/viewer/2022032723/56649d0a5503460f949dc964/html5/thumbnails/26.jpg)
Final Thoughts
BE is always optimistic in that it presumes that the real system is “better”
Even with this it is “robust” There is a perturbed version of this
algorithm which can be either optimistic or pessimistic
That version is not fully proven