Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative...
Transcript of Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative...
![Page 1: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/1.jpg)
Iterative methods for nonlinear systems of equations:
an introduction
Laboratori de Càlcul Numèric (LaCàN)Dep. de Matemàtica Aplicada III
Universitat Politècnica de Catalunyawww-lacan.upc.es
![Page 2: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/2.jpg)
2
Contents
§ Problem statement§ Motivation§ Functional iteration§ Method of direct iteration§ Picard’s method§ Newton’s method§ Quasi-Newton methods
![Page 3: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/3.jpg)
3
Problem statement
§ Starting point
§ General form of nonlinear system
(1 nonlinear equation with 1 unknown)
(linear system of order n)
f transforms vectors into vectors
α is a zero of f if
![Page 4: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/4.jpg)
4
Problem statement
In more detail:
In general, all components of f are nonlinear w.r.t. all components of x
![Page 5: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/5.jpg)
5
Problem statement
§ Particular form of nonlinear system
with and
Similar structure to linear system
Can be (trivially) transformed into the general form:
![Page 6: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/6.jpg)
6
Motivation
Mathematical models in engineeringLinear models§ Response of physical system proportional to external actions§ Simple models§ A first approximation to the real behaviour
Examples: linear systems of equations; linear PDEs
¬ Nonlinear models§ No proportionality between actions and response§ More complex models§ More realistic description of real beaviourExamples: nonlinear systems of equations; nonlinear PDEs
![Page 7: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/7.jpg)
7
Functional iteration
§ Analogy with root finding in 1-D:1-D problem n-D problem
§ Consistency: function φ must verify
(zeros of f) (fixed points of φ)
Nonlinear equation(s)
Initial approximation
Iterative scheme
![Page 8: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/8.jpg)
8
Functional iteration
§ Convergence: contractive mapping theorem
Let φ: D D, D a closed subset of R . If there exists λ ∈ [0,1) such that
then:(a) there exists a unique fixed point α of φ in D.
(b) for any initial approximation x in D, the sequence {x } generated by x = φ(x )remains in D and converges linearly to α with constant λ.
n
0k kk+1
![Page 9: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/9.jpg)
9
Method of direct iteration(or successive approximations)
Advantages§ Very simple technique (evaluate f once per iteration)Drawbacks§ Contractivity of φ not guaranteed§ Convergence is typically linear (if it converges!)
Problem in general form
Iteration function
Iteration scheme
![Page 10: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/10.jpg)
10
Method of direct iteration(or successive approximations)
![Page 11: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/11.jpg)
11
Picard’s method(or secant matrix method)
Problem in particular form
If matrix is inversible,
define the iteration function
so the iteration scheme is
Attention: do not invert matrix!
![Page 12: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/12.jpg)
12
Picard’s method(or secant matrix method)
§ Solve one linear system per iteration
§ Matrix A(x) and vector b(x) one iteration behind
If matrix is inversible,
Practical algorithm
![Page 13: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/13.jpg)
13
Picard’s method(or secant matrix method)
Matrix A(x) is a secant matrix
![Page 14: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/14.jpg)
14
Picard’s method(or secant matrix method)
Advantages§ If A(x) has a special structure (e.g. banded SPD),
it can be exploited when solving the linear systems
Drawbacks§ Matrix A(x) may be singular for some x
§ Convergence is typically linear (if it converges!)
§ Computational cost: matrix A(x) and vector b(x) change at every iteration
![Page 15: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/15.jpg)
15
Newton’s method(or Newton-Raphson’s method)
Problem in general form
Correction of non-converged approximation:
First-order Taylor’s series expansion:
![Page 16: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/16.jpg)
16
Newton’s method(or Newton-Raphson’s method)
Jacobian matrix
Involves the computation of derivatives
![Page 17: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/17.jpg)
17
Newton’s method(or Newton-Raphson’s method)
Practical algorithm
Iteration function is
Jacobian matrix not inverted in practice!
![Page 18: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/18.jpg)
18
Newton’s method(or Newton-Raphson’s method)
![Page 19: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/19.jpg)
19
Newton’s method(or Newton-Raphson’s method)
For problem in particular form, the iterative scheme is
similar to iterative schemes for linear systems
The Jacobian matrix does not retain structure of A(x)
![Page 20: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/20.jpg)
20
Newton’s method(or Newton-Raphson’s method)
Advantages§ Convergence is quadratic (for J(α) not singular)
Drawbacks§ Matrix J(x) may be singular for some x
§ Computational cost: at every iteration, (1) compute matrix J(x) and vector f(x) and (2) solve linear system
§ If A(x) has a special structure (e.g. banded SPD), it is lost when computing J(x)
![Page 21: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/21.jpg)
21
Quasi-Newton methods§ Secant method in 1-D
Similar to Newton’s method
with tangent approximated
by secant defined by the last two iterations k-1 and k
![Page 22: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/22.jpg)
22
Quasi-Newton methods
![Page 23: Iterative methods for nonlinear systems of equations: an introduction · 2011-05-31 · Iterative methods for nonlinear systems of equations: an introduction Laboratori de Càlcul](https://reader034.fdocuments.net/reader034/viewer/2022042201/5ea1aadcc021d070602b9ba7/html5/thumbnails/23.jpg)
23
Quasi-Newton methods§ Extension to nonlinear systems
Jacobian matrix is approximated by a secant matrix
defined by the last two iterations k-1 and k
n unknowns and only n equations additional conditions on matrix S required
2