Introduction to ArtificialIntelligence & Machine Learning

Nicolas ThomeProfessor at Cnam

Computer science dptStatistical Learning team (MSDMA)

1.Definition of AI and ML

2.Unsupervised learning

3.Supervised learning

Outline

Artificial Intelligence• Building machines able to solve problems, work & react like humans

• Requiring understanding of the problem

• Very general, being able to • Acquire and understand information from the world, environment => perception

• Image, audio, text, … and any sensor / measurement (physics )

Artificial Intelligence• Building machines able to solve problems, work & react like humans

• Requiring understanding of the problem

• Very general, being able to • Perform action in the world

• Robot, chatbot, playing games, etc

Artificial Intelligence & big data• Big data => huge number of data

• Impossible to manually to process such data=> Obvious need for automatic processing

• Big data applications: essentially all data sience domains• Email filtering, Online recommendations

• Voice recognition, Face recognition

• Medical diagnosis

• Autonomous driving

Artificial Intelligence & Machine learning

AI ambiguous

Historical Artificial Intelligence• Traditional IA (1950-1990): symbolic problems

• Constraint satisfaction problem (CSP) => Optimization/ search issues

• games (chess, go), Travelling salesman problem, etc

• Ex: Travelling salesman problem (TSP)

• Find the shortest path to visit all n cities• Exhaustive search: O(n!) • Explodes very quickly with n

Historical Artificial Intelligence: Expert systems

• Knowledge base collected by experts, expressed by if-then rules

• Inference: deduce new facts from knowledge

Artificial Intelligence & Machine learning

• Traditional AI: explicit rules, handcrafted programs• Difficult to build and maintain knowledge database

• For many pbs: impossible to explicitly express rules (ex: image classification)

• ML: rules learned from data, emerged from data

Machine learning: methods and supervision

• Unsupervised vs supervised learning

Machine learning: methods and supervision

• Reinforcement learning

Machine learning & generalization

• Inductive learning: training database => extract rules•Apply to new data

•Machine Learning ≠ optimization

Under-fitting vsoverfitting

Machine learning: representation

• For many tasks: input representations not adequate

Deep learning: learning representations

• ML on hacrafted features • DL on raw data

1.Definition of AI and ML

2.Unsupervised learning

3.Supervised learning

Outline

Unsupervised learning

• General motivation: learning the structure of data

• Useful for: • Clustering

• Visualization

• Learning representations, manifold learning etc …

K-Means

K-Means

K-Means

K-Means: python example on MNIST for clustering

Cluster with min entropy

K-Means: python example on MNIST for clustering

Cluster with max entropy

Principal Component Analysis

Principal Component Analysis

Principal Component Analysis

Unuspervised learning

• And many other methods…• Generative models, e.g. Gaussian Mixture Models (GMMs)

• Maximum likelihood vs Maximum a Posteriori

1.Definition of AI and ML

2.Unsupervised learning

3.Supervised learning

Outline

Supervised learning

• General methods • Decision trees and variants (random forest)

• K-NN (nearest neighbor): For each test example, simply find its closest example• Or compute k-NN, and apply majority class voting

Supervised learning

=> Train a model with gradient descent

Supervised learning: gradient descent

Supervised learning

Neural Networks

Deep Neural Networks

Deep Neural Networks & expressivity

Deep Neural Networks: Training with backprop

Backprop: chain rule

Deep Neural Networks: specific architectures

Deep Neural Networks: specific architectures

Deep Neural Networks: specific architectures

Deep Neural Networks: specific architectures

Deep learning History

Deep learning History

Deep learning History

Deep learning since 2012

Deep learning since 2012

Deep learning since 2012: ressources

Deep learning & AI: ongoing issues

Deep learning & AI:ongoing issues