STA141C: Big Data & High Performance Statistical Computing ...
Transcript of STA141C: Big Data & High Performance Statistical Computing ...
![Page 1: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/1.jpg)
STA141C: Big Data & High Performance StatisticalComputing
Lecture 0: Course information
Cho-Jui HsiehUC Davis
April 3, 2018
![Page 2: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/2.jpg)
Course Information
Website: http://www.stat.ucdavis.edu/~chohsieh/teaching/
STA141C_Spring2018/main.html
My office: Mathematical Sciences Building (MSB) 4232
Office hours: 1pm–2pm Tuesday
My email: [email protected]
TA:
Chun-Jui (Gary) Chen ([email protected])Justin Wang ([email protected])
![Page 3: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/3.jpg)
Course Information
The goal of this:
How to write a good program for data analyticsLearn to implement statistical models for big dataLearn to use some open source (python) toolsHow to parallelize your code
We’ll use python for this course
![Page 4: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/4.jpg)
Course Structure
Statistical Programming (in python)
We assume you already know how to use pythonBasic algorithms and data structure, and how to use them in python
Numerical Optimization for Statistical Problems
Briefly review basic optimization algorithmsHow to use them to solve real world problemslinear regression, classification, neural network
Parallel Computing in Python
Multicore computingDistributed computing
Numerical Linear Algebra for Statistics
Matrix decomposition for huge matricesPageRank, Clustering, Word2vec
Feature generation
Text, images
![Page 5: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/5.jpg)
Prerequisites
Basic python programming skill
(STA 141B or ECS python course)
Basic math and statistics
(linear algebra, matrix multiplication, eigen-decomposition)
![Page 6: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/6.jpg)
Grading Policy
Grading Policy
Homework (65%)
Final project (35%)
Homeworks:
Homework will be some programming problems
You’ll need to write a report for each homework.
Use python to write the programming part.
Final project:
Form a group of ≤ 4 people
Work on a real data mining problem or a data mining contest.
Project proposal due at the 5-th week (TBD)
Final project report due at the end of this quarter (TBD)
![Page 7: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/7.jpg)
Discussion sessions
No discussion sessions for the first week
Later on TAs will be giving some tutorial or reviewing homeworksolutions in discussion sessions.
![Page 8: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/8.jpg)
Preview: Numerical Optimization for Statistical Problems
Statistical Programming (in python)
Numerical Optimization for Statistical Problems
Parallel Computing in Python
Numerical Linear Algebra for Statistics
Feature generation
![Page 9: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/9.jpg)
Preview: Numerical Optimization for Statistical Problems
Optimization:minw
f (w)
f : objective function to be optimized
Example: linear regression
minw
n∑i=1
(wTxi − yi )2
Generalize to arbitrary loss:
minw
n∑i=1
loss(wTxi − yi )2
Generalize to arbitrary function:
minw
n∑i=1
loss(gw (xi ) − yi )2
![Page 10: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/10.jpg)
Preview: Numerical Optimization for Statistical Problems
Optimization:minw
f (w)
f : objective function to be optimizedExample: linear regression
minw
n∑i=1
(wTxi − yi )2
Generalize to arbitrary loss:
minw
n∑i=1
loss(wTxi − yi )2
Generalize to arbitrary function:
minw
n∑i=1
loss(gw (xi ) − yi )2
![Page 11: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/11.jpg)
Preview: Numerical Optimization for Statistical Problems
Optimization:minw
f (w)
f : objective function to be optimizedExample: linear regression
minw
n∑i=1
(wTxi − yi )2
Generalize to arbitrary loss:
minw
n∑i=1
loss(wTxi − yi )2
Generalize to arbitrary function:
minw
n∑i=1
loss(gw (xi ) − yi )2
![Page 12: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/12.jpg)
Preview: Numerical Optimization for Statistical Problems
Optimization:minw
f (w)
f : objective function to be optimizedExample: linear regression
minw
n∑i=1
(wTxi − yi )2
Generalize to arbitrary loss:
minw
n∑i=1
loss(wTxi − yi )2
Generalize to arbitrary function:
minw
n∑i=1
loss(gw (xi ) − yi )2
![Page 13: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/13.jpg)
Preview: Numerical Optimization for Statistical Problems
(Iterative) algorithms:
Gradient descentStochastic gradient descentBlock-coordinate descent· · ·
Implement to solve large problems
![Page 14: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/14.jpg)
Preview: Numerical Optimization for Statistical Problems
(Iterative) algorithms:
Gradient descentStochastic gradient descentBlock-coordinate descent· · ·
Implement to solve large problems
![Page 15: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/15.jpg)
Preview: Numerical Optimization for Statistical Problems
Statistical Programming (in python)
Numerical Optimization for Statistical Problems
Parallel Computing in Python
Numerical Linear Algebra for Statistics
Feature generation
![Page 16: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/16.jpg)
Preview: Parallel Computing in Python
Basic concept for multicore/distributed parallelism
How to implement (multi-core) parallelism in python
![Page 17: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/17.jpg)
Preview: Numerical Optimization for Statistical Problems
Statistical Programming (in python)
Numerical Optimization for Statistical Problems
Parallel Computing in Python
Numerical Linear Algebra for Statistics
Feature generation
![Page 18: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/18.jpg)
Numerical Linear Algebra for Statistics
SVD, eigen-decomposition
How to decompose huge matrices
Applications
PageRank, Clustering, Word2vec
![Page 19: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/19.jpg)
Preview: Numerical Optimization for Statistical Problems
Statistical Programming (in python)
Numerical Optimization for Statistical Problems
Parallel Computing in Python
Numerical Linear Algebra for Statistics
Feature generation
![Page 20: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/20.jpg)
Feature generation
Features for text data
Features for image data
![Page 21: STA141C: Big Data & High Performance Statistical Computing ...](https://reader031.fdocuments.net/reader031/viewer/2022012101/6169f09211a7b741a34d0c8e/html5/thumbnails/21.jpg)
Coming up
Numpy programming, basic algorithm/data structure
Questions?