Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on...
-
Upload
stella-lawson -
Category
Documents
-
view
216 -
download
1
Transcript of Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on...
![Page 1: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/1.jpg)
Peter RichtarikSchool of Mathematics
Optimization with Big Data * in a billion dimensional space on a foggy day
Extreme* Mountain Climbing=
![Page 2: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/2.jpg)
BIG DATA
• digital images & videos• transaction records• government records• health records• defence• internet activity (social media, wikipedia, ...)• scientific measurements (physics, climate models, ...)
BIG Volume BIG Velocity BIG Variety
Sources
BIG Volume BIG Velocity BIG Variety
![Page 3: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/3.jpg)
Western General Hospital(Creutzfeldt-Jakob Disease)
Arup (Truss Topology Design)
Ministry of Defence dstl lab(Algorithms for Data Simplicity)Royal Observatory
(Optimal Planet Growth)
![Page 4: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/4.jpg)
GOD’S Algorithm = Teleportation
![Page 5: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/5.jpg)
If you are not a God...
x0x1
x2 x3
![Page 6: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/6.jpg)
Optimization as Lock Breaking
Setup: Combination maximizing F opens the lock
x = (x1, x2, x3, x4) F(x) = F(x1, x2, x3, x4)
A number representing the
“quality” of a combination
Optimization Problem: Find combination maximizing F
![Page 7: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/7.jpg)
Optimization Algorithm
![Page 8: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/8.jpg)
How to Open a Lock with Billion Interconnected Dials?
F : Rn R# variables/dials = n = 109
x1
x2
Assumption:F = F1 + F2 + ... + Fn
-----------------------Fj depends on the neighbours of xj only
x3
x4
Example:F1 depends on x1, x2, x3 and x4
F2 depends on x1 and x2, ...
xn
![Page 9: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/9.jpg)
Optimization Methods
Computing Architectures• Multicore CPUs• GP GPU accelerators• Clusters / Clouds
• Effectivity• Efficiency• Scalability• Parallelism• Distribution• Asynchronicity• Randomization
![Page 10: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/10.jpg)
Optimization Methods for Big Data
• Randomized Coordinate Descent– P. R. and M. Takac: Parallel coordinate descent
methods for big data optimization, ArXiv:1212.0873 [can solve a problem with 1 billion variables in 2 hours using 24
processors]• Stochastic (Sub) Gradient Descent
– P. R. and M. Takac: Randomized lock-free methods for minimizing partially separable convex functions
[can be applied to optimize an unknown function]• Both of the above
M. Takac, A. Bijral, P. R. and N. Srebro: Mini-batch primal and dual methods for SVMs, ArXiv:1302.xxxx
![Page 11: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/11.jpg)
Theory vs Reality
![Page 12: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/12.jpg)
start
settle for this
holy grail
Parallel Coordinate Descent
![Page 13: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/13.jpg)
TOOLSProbability
Machine LearningMatrix Theory
HPC
![Page 14: Peter Richtarik School of Mathematics Optimization with Big Data * in a billion dimensional space on a foggy day Extreme* Mountain Climbing =](https://reader038.fdocuments.net/reader038/viewer/2022110321/56649ced5503460f949b9e8d/html5/thumbnails/14.jpg)