Post on 02-Aug-2015
Mathematics of Big Data
Dmitry P. VetrovHead of Big Data and Information Retrieval Department,
Faculty of Computer Science, HSE.
Outline
• Intro to Machine learning
• Big Data specifics
• Bayesian framework and its extensions
• Learning from incomplete data
• Deep learning
• Stochastic optimization
• Tensor decompositions
Bayesian methods research group
Founded in 2007. Currently consists of 8 students, 5 PhD students, 1 researcher and 1 associate professor.
Bayesian methods research group
What is machine learning?
Simple example
Areas of application
Stages
Entering the Age of Big Data
Data
Computationalspeed
First Steps towards Mathematics of Big Data
Bayesian Framework
Bayesian Learning and Inference
Advantages of Bayesian inference
Graphical Models
Application
Incomplete data
Incomplete data
EM algorithm
General idea of SVM
Latent variable SVM
Semantic image segmentation
Weak annotation
Example: Latent DirichletAllocation
LDA: model
Deep learning
Secret of Success of Neural Nets
Stochastic Optimization
Advanced Stochastic Optimization
Tensor perspective
Advantages of TT Decomposition
Word2vec project
A Surprising effect
Latent semantic model
Results
Computer can now assign different semantic representations to different occurrences of same word depending on the context
Conclusion
References
• (Osokin15) A. Osokin, D. Vetrov. Submodular Relaxation for Inference in Markov Random Fields. In IEEE TPAMI, 2015.
• (Novikov14) A. Novikov, A. Rodomanov, A. Osokin, D. Vetrov. Putting MRF on a Tensor Train. In ICML2014
• (Bartunov14) S. Bartunov, D. Vetrov. Variational Inference for Sequential Distance Dependent Chinese Restaurant Process. In ICML2014
• (Shapovalov15) R. Shapovalov, A. Osokin, D. Vetrov, P. Kohli. Multi-utility Learning: Structured-output Learning with Multiple Annotation-specific Loss Functions. In EMMCVPR15
• (Kirillov14) A. Kirillov, K. Lobacheva, M. Gavrikov, A. Osokin, D. Vetrov. Deep Part-Based Shape Model with Latent Variables. In GraphiCon14