Mathematics of Big Data

Dmitry P. VetrovHead of Big Data and Information Retrieval Department,

Faculty of Computer Science, HSE.

Outline

• Intro to Machine learning

• Big Data specifics

• Bayesian framework and its extensions

• Learning from incomplete data

• Deep learning

• Stochastic optimization

• Tensor decompositions

Bayesian methods research group

Founded in 2007. Currently consists of 8 students, 5 PhD students, 1 researcher and 1 associate professor.

Bayesian methods research group

What is machine learning?

Simple example

Areas of application

Stages

Entering the Age of Big Data

Data

Computationalspeed

First Steps towards Mathematics of Big Data

Bayesian Framework

Bayesian Learning and Inference

Advantages of Bayesian inference

Graphical Models

Application

Incomplete data

Incomplete data

EM algorithm

General idea of SVM

Latent variable SVM

Semantic image segmentation

Weak annotation

Example: Latent DirichletAllocation

LDA: model

Deep learning

Secret of Success of Neural Nets

Stochastic Optimization

Advanced Stochastic Optimization

Tensor perspective

Advantages of TT Decomposition

Word2vec project

A Surprising effect

Latent semantic model

Results

Computer can now assign different semantic representations to different occurrences of same word depending on the context

Conclusion

References

• (Osokin15) A. Osokin, D. Vetrov. Submodular Relaxation for Inference in Markov Random Fields. In IEEE TPAMI, 2015.

• (Novikov14) A. Novikov, A. Rodomanov, A. Osokin, D. Vetrov. Putting MRF on a Tensor Train. In ICML2014

• (Bartunov14) S. Bartunov, D. Vetrov. Variational Inference for Sequential Distance Dependent Chinese Restaurant Process. In ICML2014

• (Shapovalov15) R. Shapovalov, A. Osokin, D. Vetrov, P. Kohli. Multi-utility Learning: Structured-output Learning with Multiple Annotation-specific Loss Functions. In EMMCVPR15

• (Kirillov14) A. Kirillov, K. Lobacheva, M. Gavrikov, A. Osokin, D. Vetrov. Deep Part-Based Shape Model with Latent Variables. In GraphiCon14