Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf ·...
Transcript of Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf ·...
![Page 1: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/1.jpg)
Numerical Methods in Scientific Computing
NGSSC
Uppsala University
January 2012
Maya Neytcheva, IT, Uppsala [email protected] – p. 1/??
![Page 2: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/2.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Why this course ?
What to expect ?
Maya Neytcheva, IT, Uppsala [email protected] – p. 2/??
![Page 3: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/3.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Sci. Comp. - Numerical simulations
data structure
Finite diff.Finite elementsFinite volume
Boundary elements
A set of pde’s
Integral equations
Something else
languages
Physical phenomenon
Mathematical model
Num. discretization
which method
matrix properties
Computer implement.
data structures
parallel computingsequential computing
Num.solut.meth.(NLA)
Maya Neytcheva, IT, Uppsala [email protected] – p. 3/??
![Page 4: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/4.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Sci. Comp. - Numerical simulations
data structure
Finite diff.Finite elementsFinite volume
Boundary elements
A set of pde’s
Integral equations
Something else
languages
Physical phenomenon
Mathematical model
which method
matrix properties
Computer implement.
data structures
parallel computingsequential computing
Num.solut.meth.(NLA)
Num. discretization
Stability wrt discretization parameters
Discretization error
Model error
Solution error (iterations, stop.crit.)
Robustness
Method parameters
parallelization
Execution bottlenecks, memory
Scalability
Maya Neytcheva, IT, Uppsala [email protected] – p. 4/??
![Page 5: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/5.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Day 1 and 2:
TARGET 1: solve a linear system of equations
Ax = b
GOAL: get a global overview of what the needs are, how farwe can go, and with which methods.➜ how large systems we want to solve nowadays;➜ what kind of systems;➜ what are the methods of choice;➜ how far we can go with a particular technique.
Maya Neytcheva, IT, Uppsala [email protected] – p. 5/??
![Page 6: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/6.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Day 3 and 4:
TARGET 2: Discretization of ordinary and partial differentialequationsGOAL: Understand how the type of the problem influences thechoice of discretization parameters➜ Systems of ordinary differential equations (ODEs)➜ Elliptic PDEs;➜ Parabolic PDEs;➜ Hyperbolic PDEs.
Maya Neytcheva, IT, Uppsala [email protected] – p. 6/??
![Page 7: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/7.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Day 5:
TARGET 3: Monte Carlo methodsGOAL: Get basic understanding of the MC methods and theirpotential (and limitations) in particular for high dimensionalproblems.
Maya Neytcheva, IT, Uppsala [email protected] – p. 7/??
![Page 8: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/8.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Day 1 and 2:
➜ Introduction
➜ Direct solution methodsfor linear systems – dense and sparse matrices
➜ Iterative solution methods
➤ Basic techniques
➤ Projection methods
➣ The Conjugate Gradient method (CG)
➣ The Generalized Conjugate Gradient method (GCG)
➤ Preconditioning
➣ (Block-)ILU
➣ Approximate inverses
➣ Block preconditioners (block-triangular, block-factorized,
Schur complement-based)
➣ Multilevel methods
Maya Neytcheva, IT, Uppsala [email protected] – p. 8/??
![Page 9: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/9.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Partial list of application fieldsStiff ODEsDifferential-Algebraic EquationsLinear programming (simplex, interior point methods)OptimizationNonlinear equationsElliptic PDEsEigensolutionsTwo-point boundary value problemsLeast Squares calculations
Maya Neytcheva, IT, Uppsala [email protected] – p. 9/??
![Page 10: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/10.jpg)
NGSSC-SciComp, January, 2012, Uppsala
More application fieldsacoustic scattering demography network flowair traffic control economics oceanographyastrophysics electrical eng. petroleum eng.biochemical electric nets reactor modellingchemical eng. climate/pollution studies statisticschemical kinetics fluid flow structural engcircuit physics laser optics survey datacomputer simulations linear programming signal processing
Maya Neytcheva, IT, Uppsala [email protected] – p. 10/??
![Page 11: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/11.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Discretization and linearization
of nonlinear PDEs
Linear system
Ax = b
Nonsingular Well-conditioned
Small
LargeNonsymmetric
Symmetric
Singular
Real
Complex
Normal
Nonnormal Ill-conditioned
Discretization
of linear PDEs
Optimal
control
Stochastic
processes
Image
analysis
Dense
Sparse
StructuredNot structured
Maya Neytcheva, IT, Uppsala [email protected] – p. 11/??
![Page 12: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/12.jpg)
NGSSC-SciComp, January, 2012, Uppsala
When talking about the solution of a linear system ofequations:
- computer demands (computing time and memoryconsumption)
- numerical efficiency (number of iterations)
- computational complexity
- robustness wrt to (problem, discretization and method)parameters
Maya Neytcheva, IT, Uppsala [email protected] – p. 12/??
![Page 13: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/13.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues:
Cramer’s rule
Ax = b, A(n× n), det(A) 6= 0
a11 · · · a1,i−1 a1,i a1,i+1 · · · a1,n
· · · · · · · · · · · · · · · · · · · · ·
ai1 · · · ai,i−1 ai,i ai,i+1 · · · ai,n
· · · · · · · · · · · · · · · · · · · · ·
an1 · · · an,i−1 an,i an,i+1 · · · an,n
x1
· · ·
xi
· · ·
xn
=
b1
· · ·
bi
· · ·
bn
Maya Neytcheva, IT, Uppsala [email protected] – p. 13/??
![Page 14: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/14.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues:
Cramer’s rule
xi =1
det(A)
a11 · · · a1,i−1 b1 a1,i+1 · · · a1,n
· · · · · · · · · · · · · · · · · · · · ·
ai1 · · · ai,i−1 bi ai,i+1 · · · ai,n
· · · · · · · · · · · · · · · · · · · · ·
an1 · · · an,i−1 bn an,i+1 · · · an,n
Maya Neytcheva, IT, Uppsala [email protected] – p. 14/??
![Page 15: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/15.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues
Consider products of n elements of A,
a1,α1 , a2,α2 , · · · , an,αn ,
where α1, α2, · · · , αn is a permutation of 1, 2, · · · , n.The number of all these products is n! .
det(A) =∑
i=1
n!
n∏
j=1
(−1)γ aj,αj,
thus, the computational complexity to solve the system is n!.To be more precise: (n+ 1)(n!) = (n+ 1)! multiplications and(n+ 1)(n!) = (n+ 1)! additions.
Maya Neytcheva, IT, Uppsala [email protected] – p. 15/??
![Page 16: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/16.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues:
computing det(A)
Method Multiplications Additions
Gaussian Elimination 13n
3 + n2 − 13n
13n
3 + 12n
2 − 56n
Gauss-Jordan Elimination 13n
3 + n2 − 52n+ 2 1
3n3 − 3
2n2 + 1
Solving using the factorsobtained by Gaussianelimination
56n
3 + 2n2 − 56n
43n
3 − 12n
2 − 56n
Solving using the factorsobtained by Gauss-Jordanelimination
32n
3 + 12n− 1 3
2n3 − 2n2 − 1
2n
Cramer’s Rule n! n!
Maya Neytcheva, IT, Uppsala [email protected] – p. 16/??
![Page 17: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/17.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues:
Matrix inversion Gauss-Jordan elimination O(n3)
Strassen algorithm O(n2.807)
Coppersmith-Winograd algorithm O(n2.376)
Determinant Laplace expansion O(n!)
LU decomposition O(n3)
Bareiss algorithm O(n3)
Fast matrix multiplication O(n2.376)
Maya Neytcheva, IT, Uppsala [email protected] – p. 17/??
![Page 18: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/18.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues: Cramer
against Gauss
A comparison of the amount of time to solve Ax = b on a Cray J90. The Cray J90performs one trillion operations per second (one teraflop).
n Gaussian Elimination Cramer’s Rule
2 6 x 10−12 secs 6 x 10−12 secs
3 1.7 x 10−11 secs 2.4 x 10−11 secs
4 3.6 x 10−11 secs 1.2 x 10−10 secs
5 6.5 x 10−11 secs 7.2 x 10−10 secs
6 1.06 x 10−10 secs 5.04 x 10−09 secs
10 4.3 x 10−10 secs 3.99168 x 10−05 secs
20 3.06 x 10−9 secs 1.622 years
100 3.433 x 10−7 secs 2.9889 x 10138 centuries
1000 3.3433 x 10−4 secs
Maya Neytcheva, IT, Uppsala [email protected] – p. 18/??
![Page 19: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/19.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Computational complexity issues: Cramer
against Gauss
In November 2011, it was announced that Japan hadachieved 10.51 petaflops with its K computer.Will tera-, peta-computers change much?
Maya Neytcheva, IT, Uppsala [email protected] – p. 19/??
![Page 20: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/20.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Factorials...
In 2001, the value of 1000! was currently too large to bestored as a single number in the memory of a computer.(Computational Science: Tools for a Changing World by R.A.Tapia, C. Lanius, 2001, Rice.)
The scientific calculator in Windows XP is able to calculatefactorials up to at least 100000!.(look-up tables)
Maya Neytcheva, IT, Uppsala [email protected] – p. 20/??
![Page 21: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/21.jpg)
mine
20-1
![Page 22: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/22.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Grand challenge problems
Environment• High-resolution weather forecasting: more accurate, faster
and longer range predictions• Pollution studies, including cross-pollutant interactions• Global atmosphere-ocean-biosphere and long range cli-
mate modelling⇒ Risø National Laboratory & Technical University of Den-mark, Roskilde
Maya Neytcheva, IT, Uppsala [email protected] – p. 21/??
![Page 23: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/23.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Danish Eulerian model: to perform numerical simulations andto study the air pollution over Europe.Task: establish reliable control strategies for the air pollution.
![Page 24: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/24.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Danish Eulerian model: to perform numerical simulations andto study the air pollution over Europe.Task: establish reliable control strategies for the air pollution.Basic scheme: pollutants are emitted in the air from varioussources
![Page 25: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/25.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Danish Eulerian model: to perform numerical simulations andto study the air pollution over Europe.Task: establish reliable control strategies for the air pollution.Basic scheme: pollutants (many of them) are emitted in the airfrom various sources (many of them) and
transported by the wind - (diffusion, advection, horizontalvertical)
get deposited on the Earth surface
transform due to chemical reactionsfactors: winds, temperature, humidity, day/night, ...
Maya Neytcheva, IT, Uppsala [email protected] – p. 22/??
![Page 26: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/26.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
∂cs∂t
+−
3∑
i=1
∂ui cs∂xi
+
3∑
i=1
∂
∂xi
(
Kxi
∂cs∂xi
)
+Es+kscs+Q(c1, · · · , cq),
where cs, s = 1, · · · , q are the unknown concentrations of qspecies of the air pollutants to be followed.
Maya Neytcheva, IT, Uppsala [email protected] – p. 23/??
![Page 27: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/27.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Demands:
Maya Neytcheva, IT, Uppsala [email protected] – p. 24/??
![Page 28: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/28.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Demands:
Spatial domain: 4800 km2, 96× 96 or 480× 480 regularmesh (horizontal resolution)
Maya Neytcheva, IT, Uppsala [email protected] – p. 24/??
![Page 29: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/29.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Demands:
Spatial domain: 4800 km2, 96× 96 or 480× 480 regularmesh (horizontal resolution)10 vertical layers, 1 km each
35 pollutants
Maya Neytcheva, IT, Uppsala [email protected] – p. 24/??
![Page 30: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/30.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Demands:
Spatial domain: 4800 km2, 96× 96 or 480× 480 regularmesh (horizontal resolution)10 vertical layers, 1 km each
35 pollutants3D: 35 ∗ 10 ∗ 962 = 3225600 unknowns3D: 35 ∗ 10 ∗ 4802 = 80640000 unknowns
about 36000 time-steps, ’only one month simulated’
Maya Neytcheva, IT, Uppsala [email protected] – p. 24/??
![Page 31: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/31.jpg)
NGSSC-SciComp, January, 2012, Uppsala
DEM - air pollution model
Demands:
Spatial domain: 4800 km2, 96× 96 or 480× 480 regularmesh (horizontal resolution)10 vertical layers, 1 km each
35 pollutants3D: 35 ∗ 10 ∗ 962 = 3225600 unknowns3D: 35 ∗ 10 ∗ 4802 = 80640000 unknowns
about 36000 time-steps, ’only one month simulated’Computers at DCAMM: 16 PEs on Newton:
60 000 sec = 1000 min ≈ 17 h Is this much or not?
Maya Neytcheva, IT, Uppsala [email protected] – p. 24/??
![Page 32: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/32.jpg)
NGSSC-SciComp, January, 2012, Uppsala
The topics of the Sci.Comp. course - a
cross-disciplinary point
Conducting a numerical simulation:
Modelling
Discretization
Choice of a solution method
Computer implementation
Postprocessing
![Page 33: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/33.jpg)
NGSSC-SciComp, January, 2012, Uppsala
The topics of the Sci.Comp. course - a
cross-disciplinary point
Modelling:modelling error
![Page 34: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/34.jpg)
NGSSC-SciComp, January, 2012, Uppsala
The topics of the Sci.Comp. course - a
cross-disciplinary point
Modelling: modelling error
Discretization: discretization error (space, time,stability...)
![Page 35: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/35.jpg)
NGSSC-SciComp, January, 2012, Uppsala
The topics of the Sci.Comp. course - a
cross-disciplinary point
Modelling: modelling error
Discretization: discretization error (space, time,stability...)
Choice of a solution method: iteration error (robustnesswrt discretization parameters, )
Maya Neytcheva, IT, Uppsala [email protected] – p. 25/??
![Page 36: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/36.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Large dense matricesAn idea what matrix dimensions might have been consideredvery large for a dense, direct matrix computation through theyears:
n Year Source20 1950 Wilkinson
200 1965 Forsythe&Moler2000 1980 LINPACK
20000 1995 LAPACK> 200 000 today (Umeå)
J. Wilkinson, The algebraic eigenvalue problem, 1965G. Forsythe& C. Moler, Computer solutions of linear algebraicsystems, 1967.
Maya Neytcheva, IT, Uppsala [email protected] – p. 26/??
![Page 37: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/37.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Matrix factorizations (dense matrices)
Schur’s form: A = UTRU
LU
LLT or UTU
LDU
Maya Neytcheva, IT, Uppsala [email protected] – p. 27/??
![Page 38: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/38.jpg)
NGSSC-SciComp, January, 2012, Uppsala
LU factorization A(m,n)
for k = 1, 2 · · ·m− 1
d = 1/a(k)kk
for i = k + 1, · · ·m
ℓ(k)ik
= −a(k)ik
d
for j = k + 1, · · ·n
a(k)ij = a
(k)ij + ℓika
(k)kj
end
end
end
The operational count for the LU factorization can be obtained by integrating the loops:
FlopsLU =
m−1∫
1
m∫
k
n∫
k
djdidk ≈ n3/3 (m = n)
Maya Neytcheva, IT, Uppsala [email protected] – p. 28/??
![Page 39: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/39.jpg)
NGSSC-SciComp, January, 2012, Uppsala
The LDU factorization
Theorem:
If all leading principal minors of A are nonsingular, then thereexist unique lower-triangular matrix L, diagonal matrix D andupper-triangular matrix U , such that A = LDU .
Maya Neytcheva, IT, Uppsala [email protected] – p. 29/??
![Page 40: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/40.jpg)
NGSSC-SciComp, January, 2012, Uppsala
How do we compute A = LDU?
Note: If ej is the jth unit vector, then Mej = M(:, j).
Assume that the first j − 1 columns of L, j − 1 elements of D and j − 1 rows of U arecomputed.
1
∗
. . .
∗ ∗ 1
? ? ? 1
? ? ? ?. . .
? ? ? ? ? 1
d1
. . .
dj−1
?
. . .
?
1 ∗ ∗ ? ? ?
. . . ∗ ? ? ?
1 ? ? ?
1 ? ?
. . . ?
1
Maya Neytcheva, IT, Uppsala [email protected] – p. 30/??
![Page 41: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/41.jpg)
NGSSC-SciComp, January, 2012, Uppsala
How do we compute A = LDU?
We equate the jth column of A = LDU :Denote v = DUej . Then Aej = Lv
v(1 : j) = L(1 : j, 1 : j)−1A(1 : j, j) - known data
d(j) = v(j)
U(i, j) = v(i)/d(i), i = 1 : j − 1
L(j + 1 : n, j)v(1 : j) = A(j + 1, j)
Maya Neytcheva, IT, Uppsala [email protected] – p. 31/??
![Page 42: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/42.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Why bother about dense factorizations and the tricks behind?
Maya Neytcheva, IT, Uppsala [email protected] – p. 32/??
![Page 43: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/43.jpg)
NGSSC-SciComp, January, 2012, Uppsala
What about this...
"Dense Matrix Factorization of Linear Complexity for Impedance Extraction ofLarge-Scale 3-D Integrated Circuits"Wenwen Chai, Dan Jiao School of Electrical and Computer Engineering, PurdueUniversityIEEE Xplore, July 2010
Abstract: A fast LU factorization of linear complexity is developed to directly solve adense system of linear equations for the interconnect extraction of any arbitrary shaped3-D structure embedded in inhomogeneous materials. The proposed solver successfullyfactorizes dense matrices that involve more than one million unknowns in fast CPU runtime and modest memory consumption. Comparisons with state-of-the-art integralequation- based interconnect extraction tools have demonstrated its clear advantages.
Maya Neytcheva, IT, Uppsala [email protected] – p. 33/??
![Page 44: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/44.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Factorizing symmetric positive definitematrices
Maya Neytcheva, IT, Uppsala [email protected] – p. 34/??
![Page 45: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/45.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Factorizing symmetric positive definite matrices
Factorize A = LLT , L – lower-triangularCholesky factorization
Maya Neytcheva, IT, Uppsala [email protected] – p. 35/??
![Page 46: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/46.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Factorizing symmetric marices
The mathematician after whom the Cholesky factorisation isnamed
Maya Neytcheva, IT, Uppsala [email protected] – p. 36/??
![Page 47: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/47.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Major Andre-Louis Cholesky(1875-1918)
Cholesky was born in France, and worked in the Geodesicsection of the Geographic service to the French army’sartillery branch.At this time the system of triangulation used in France, andbased on the meridian line of Paris, was being revised; newmethods were needed in order to facilitate what was not yet aquick or convenient process.Cholesky invented computation procedures based on themethod of least squares, for the solution of certain data-fittingproblems in geodesy, to be put into practice in his triangulationof the French and British parts of Crete, and in his work inAlgeria and Tunisia.His mathematical work was posthumously published on hisbehalf in 1924 by a fellow officer, Benoit.
Maya Neytcheva, IT, Uppsala [email protected] – p. 37/??
![Page 48: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/48.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Cholesky factorization ...
% Maya’s version of Cholesky - to compare execution time
% ---------------------------------------------------------------
function [U]=my_chol(A)
A = triu(A);
n = size(A,1);
for k=1:n,
A(1:k-1,k) = A(1:k-1,1:k-1)’\A(1:k-1,k);
A(k,k) = sqrt(A(k,k) - A(1:k-1,k)’ * A(1:k-1,k));
end
U = triu(A);
return
Maya Neytcheva, IT, Uppsala [email protected] – p. 38/??
![Page 49: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/49.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Cholesky factorization ...
size(A) chol chol RatioMatlab mine
10 1.9 10−5 0.00334 1.76 102
50 5 10−5 0.00224 45
100 1.5 10−4 0.00517 34.5
500 0.0053 0.234918 44.32
1000 0.0259 2.117073 81.87
5000 2.7078 391.9548 144.8
Maya Neytcheva, IT, Uppsala [email protected] – p. 39/??
![Page 50: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/50.jpg)
NGSSC-SciComp, January, 2012, Uppsala
GAXPY Cholesky
for k = 1 : n
if k > 1
A(k : n, k) = A(k : n, k)−A(k : n, k − 1) ∗A(k, 1 : k − 1)T
endif
A(k : n, k) = A(k : n, k)/√
A(k, k)
end
Maya Neytcheva, IT, Uppsala [email protected] – p. 40/??
![Page 51: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/51.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Outer Product Cholesky
for k = 1 : n
A(k, k) =√
A(k, k)
A(k + 1 : n, k) = A(k + 1 : n, k)−A(n : k, k − 1)/A(k, k)
for j = k + 1 : n
A(j : n, j) = A(j : n, j)−A(j : n, j)A(j, k)
end
end
Maya Neytcheva, IT, Uppsala [email protected] – p. 41/??
![Page 52: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/52.jpg)
NGSSC-SciComp, January, 2012, Uppsala
put together
for k = 1 : n
if k > 1
A(k : n, k) = A(k : n, k)−A(k : n, k − 1) ∗A(k, 1 : k − 1)T
endif
A(k : n, k) = A(k : n, k)/√
A(k, k)
end
for k = 1 : n
A(k, k) =√
A(k, k)
A(k + 1 : n, k) = A(k + 1 : n, k)−A(n : k, k − 1)/A(k, k)
for j = k + 1 : n
A(j : n, j) = A(j : n, j)−A(j : n, j)A(j, k)
end
end
Maya Neytcheva, IT, Uppsala [email protected] – p. 42/??
![Page 53: Numerical Methods in Scientific Computinguser.it.uu.se/.../NGSSC_SciComp_Introduction_Psli.pdf · Computer implement. data structures parallel computing sequential computing Num.solut.meth.(NLA)](https://reader034.fdocuments.net/reader034/viewer/2022050102/5f41a0546ed73952cb7f0208/html5/thumbnails/53.jpg)
NGSSC-SciComp, January, 2012, Uppsala
Example of implementing Cholesky
factorization
for k=1:nxeuitb(A(1:k-1,k),A(1:k-1,1:k-1),A(1:k-1,k))A(k,k) = sqrt(A(k,k) - A(1:k-1,k)ˆT * A(1:k-1,k))
end
Computes U (which overwrites A).
BLAS xeuitb(X,U,B) computes X = U−1B
Maya Neytcheva, IT, Uppsala [email protected] – p. 43/??