Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ......

49

Transcript of Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ......

Page 1: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 2: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Auto ML & UI ML

Model Interpretability

ML Ops

3+1 = ?

Page 3: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 4: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Machine Learning Process

Model

Page 5: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Machine Learning on Azure

Domain specific pretrained modelsTo reduce time to market

Azure Databricks

Machine Learning VMs

Popular frameworks To build advanced deep learning solutions

TensorFlowPytorch Onnx

Azure Machine Learning

LanguageSpeech

…

SearchVision

Productive servicesTo empower data science and development teams

Powerful infrastructureTo accelerate deep learning

Scikit-Learn

PyCharm Jupyter

Familiar Data Science toolsTo simplify model development

Visual Studio Code Command line

CPU GPU FPGA

From the Intelligent Cloud to the Intelligent Edge

Page 6: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

What is Azure Machine Learning service?

Set of Azure

Cloud ServicesPython

SDK

✓ Prepare Data

✓ Build Models

✓ Train Models

✓Manage Models

✓ Track Experiments

✓Deploy Models

That enables

you to:

Page 7: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://azure.microsoft.com/en-us/services/machine-learning/

https://github.com/microsoft/recommenders

https://azure.microsoft.com/en-us/services/machine-learning/
https://github.com/microsoft/recommenders
Page 8: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Automated machine learning

Page 9: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Step 1Step 2Step 3

Page 10: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Configure

Choose

Compute Target

Run

Sit back

& RelaxSelect

Algorithm

Hyper-parameter

tuning

Test

Best Model

Produced!

Auto ML

https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-auto-train

Page 11: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

How much is this car worth?

Azure Machine Learning Automated machine learning

Page 12: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Mileage

Condition

Car brand

Year of make

Regulations

…

Parameter 1

Parameter 2

Parameter 3

Parameter 4

…

Gradient Boosted

Nearest Neighbors

SVM

Bayesian Regression

LGBM

…

Mileage Gradient Boosted Criterion

Loss

Min Samples Split

Min Samples Leaf

Others Model

Which algorithm? Which parameters?Which features?

Car brand

Year of make

Model creation is typically a time consuming process

Page 13: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Which algorithm? Which parameters?Which features?

Mileage

Condition

Car brand

Year of make

Regulations

…

Gradient Boosted

Nearest Neighbors

SGD

Bayesian Regression

LGBM

…

Nearest Neighbors

Criterion

Loss

Min Samples Split

Min Samples Leaf

XYZ Model

Iterate

Gradient Boosted N Neighbors

Weights

Metric

P

ZYX

Mileage

Car brand

Year of make

Car brand

Year of make

Condition

Track

Model creation is typically a time consuming process

Page 14: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Track

Which algorithm? Which parameters?Which features?

Iterate

Model creation is typically a time consuming process

Page 15: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Enter data

Define goals

Apply constraints

Output

Automated Machine Learning accelerates model development

Input Intelligently test multiple models in parallel

Optimized model

Page 16: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Feature Importance

Mileage

Condition

Car brand

Year of make

Regulations

0 1

Feature Importance

Mileage

Condition

Car brand

Year of make

Regulations

95% model

Understand the inner workings of ML by analyzing feature importance

70% model

Page 17: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://github.com/Azure/MachineLearningNotebooks

https://github.com/Azure/MachineLearningNotebooks
Page 18: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

ONNX

Page 19: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Open Neural Network Exchange

Page 20: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://onnx.ai/

https://github.com/microsoft/onnxruntime

https://onnx.ai/
https://github.com/microsoft/onnxruntime
Page 21: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 22: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

What

Why

Page 23: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 24: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 25: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Why Trust Model?

Page 26: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Why Trust Model?

?

Page 27: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

(Husky Vs Wolf)

Page 28: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 29: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 30: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 31: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 32: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

FeaturesFeature 1 Feature 2 Engineered Feature 3

Page 33: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Series 1

Series 2

Series 3

0

1

2

3

4

5

Category 1Category 2

Category 3Category 4

0-1 1-2 2-3 3-4 4-5

0

0.5

1

1.5

2

2.5

3

3.5

4

0 0.5 1 1.5 2 2.5 3 3.5

Page 34: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

azureml.contrib.

interpret

azureml.train.

automl.

automlexplainerazureml.interpret

Page 35: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 36: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://github.com/interpretml/interpret-community

https://github.com/interpretml/interpret-community
Page 37: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Model Deployment

Page 38: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Choosing Model Deployment Modes

Deployment Mode When to choose

ACI – Azure Container Instances • Faster Deployment

• Mainly used in Development and Testing scenarios

AKS – Azure Kubernetes Service • Preferable for Production deployments

• Autoscaling

• Logging & Model Data collection

• Best performance for web-service calls

Azure IoT Edge • Low latency decision making

• Reduce network bandwidth by making predictions locally

• Intermittent network connectivity

• Highly sensitive IP

Batch Deployment mode • For making decisions on the groups of records offline

• Large scale batch decisioning

Page 39: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

DevOps for machine learning

Page 40: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

DevOps loop for data science

Prepare

Data

Prepare

Register and

Manage Model

Build

Image

…

Build model

(your favorite IDE)

Deploy Service

Monitor Model

Train &

Test Model

Page 41: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Model management in Azure Machine Learning

Create/retrain model

Create scoring

files and

dependencies

Create and register image

MonitorRegister model

Cloud Light

Edge

Heavy

Edge

Deploy

image

Page 42: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://github.com/microsoft/MCW-ML-Ops

https://microsoftcloudworkshop.com/

https://github.com/microsoft/MCW-ML-Ops
https://microsoftcloudworkshop.com/
Page 43: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image
Page 44: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Image Source: https://www.thoughtco.com/facts-about-jellyfish-4102061

https://www.thoughtco.com/facts-about-jellyfish-4102061
Page 45: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Image Source: https://www.thoughtco.com/facts-about-jellyfish-4102061

https://www.thoughtco.com/facts-about-jellyfish-4102061
Page 46: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Image Source: https://www.thoughtco.com/facts-about-jellyfish-4102061

https://www.thoughtco.com/facts-about-jellyfish-4102061
Page 47: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

Domain Expertise

Less Labelled Data

Rapid Training

Page 48: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

https://blogs.microsoft.com/ai/machine-teaching/

https://docs.bons.ai/guides/inkling-guide.html

https://www.microsoft.com/en-us/research/group/machine-teaching-group/

https://blogs.microsoft.com/ai/machine-teaching/
https://docs.bons.ai/guides/inkling-guide.html
https://www.microsoft.com/en-us/research/group/machine-teaching-group/
Page 49: Auto ML & UI ML Model Interpretability · Hyper-parameter tuning Test Best Model Produced! ... DevOps loop for data science Prepare Data Prepare Register and Manage Model Build Image

How to Prepare Your Business Model for the · PDF fileHow to Prepare Your Business Model for the Cloud ... •Inbound marketing and online presence: ... sustainability within new business

How to Prepare Your Business Model for the · PDF fileHow to Prepare Your Business Model for the Cloud ... •Inbound marketing and online presence: ... sustainability within new business

Syntax - Stata · Title ml — Maximum likelihood estimation Syntax ml model in interactive mode ml model method progname eq eq ::: if in weight, model options svy diparm options

Syntax - Stata · Title ml — Maximum likelihood estimation Syntax ml model in interactive mode ml model method progname eq eq ::: if in weight, model options svy diparm options

how to prepare a architecture model in a Fablab

how to prepare a architecture model in a Fablab

Ebx Ml Collaboration Modeling Met a Model

Ebx Ml Collaboration Modeling Met a Model

txWPeg - multimedia.3m.com · Helmet Brand (a) Helmet Model (b) P3 Adapter (c) H31 H510 H520 H540 3M™ 1465 E ML ML ML ML AF ML ML ML ML 3M™ Airstream AH1, AH4, AH7, AE ML ML ML

txWPeg - multimedia.3m.com · Helmet Brand (a) Helmet Model (b) P3 Adapter (c) H31 H510 H520 H540 3M™ 1465 E ML ML ML ML AF ML ML ML ML 3M™ Airstream AH1, AH4, AH7, AE ML ML ML

ML 800 Series Laser - Industrial Fiber Optics, Inci-fiberoptics.com/pdf/12_0098-ml8xxseries.pdf · IndustrIal FIber OptIcs Model Numbers: ML 800 ML 810 ML 868 ML 869 ML 869A ML 800

ML 800 Series Laser - Industrial Fiber Optics, Inci-fiberoptics.com/pdf/12_0098-ml8xxseries.pdf · IndustrIal FIber OptIcs Model Numbers: ML 800 ML 810 ML 868 ML 869 ML 869A ML 800

micro203.weebly.com · 2019-10-06 · Prepare 100 ml of nutrient broth in a 250- ml flask. Using the graduated cylinder, add 100 ml of distilled water to the flask. Read the preparation

micro203.weebly.com · 2019-10-06 · Prepare 100 ml of nutrient broth in a 250- ml flask. Using the graduated cylinder, add 100 ml of distilled water to the flask. Read the preparation

MODEL 6430T, 6460 & 6460T POTATO PEELER manual.pdfFORM 37005 (Apr. 2006) MODEL 6430T, 6460 & 6460T POTATO PEELER MODEL ML-137602 6430T ML-19630 6460 ML-137604 6460T ML-104562 6460C

MODEL 6430T, 6460 & 6460T POTATO PEELER manual.pdfFORM 37005 (Apr. 2006) MODEL 6430T, 6460 & 6460T POTATO PEELER MODEL ML-137602 6430T ML-19630 6460 ML-137604 6460T ML-104562 6460C

Antorchas de Soldadura (€ - ES) lists/52299017... · 4 Antorchas de soldadura MIG refrigeradas por gas Datos técnicos MODEL TWIST ML 150/161 ML 240/241 ML 250 ML 300 ML 360 ML

Antorchas de Soldadura (€ - ES) lists/52299017... · 4 Antorchas de soldadura MIG refrigeradas por gas Datos técnicos MODEL TWIST ML 150/161 ML 240/241 ML 250 ML 300 ML 360 ML

ML-Leaks: Model and Data Independent Membership Inference ...

ML-Leaks: Model and Data Independent Membership Inference ...

MTC Model BT-ML Battery Transporter Specification …€¦ · Model BT Battery Transporter Specification Sheet . Figure #2. BT-ML (Back View) Safety Design Safety of operation is

MTC Model BT-ML Battery Transporter Specification …€¦ · Model BT Battery Transporter Specification Sheet . Figure #2. BT-ML (Back View) Safety Design Safety of operation is

ELECTRIC COMBI, CONVECTION & STEAM OVENSFORM 34039 (7-99) ELECTRIC COMBI, CONVECTION & STEAM OVENS MODEL HCE6H ML-126174 HCE10H ML-126175 HCE10F ML-126176 HCE20H ML-126577 HCE20F ML-126578

ELECTRIC COMBI, CONVECTION & STEAM OVENSFORM 34039 (7-99) ELECTRIC COMBI, CONVECTION & STEAM OVENS MODEL HCE6H ML-126174 HCE10H ML-126175 HCE10F ML-126176 HCE20H ML-126577 HCE20F ML-126578

Architecture of ML Systems 10 Model Selection · Architecture of ML Systems 10 Model Selection Matthias Boehm Graz University of Technology, Austria Computer Science and Biomedical

Architecture of ML Systems 10 Model Selection · Architecture of ML Systems 10 Model Selection Matthias Boehm Graz University of Technology, Austria Computer Science and Biomedical

Model-based ML for Retrospective Event Detection

Model-based ML for Retrospective Event Detection

Syllabus Chemistrybkssps.com/wp-content/downloads/n5925494f6f8a6.pdfTo prepare crystal of potash alum July – 2017 2. Volumetric analysis a. Prepare 250 ml of M/10 oxalic acid solution

Syllabus Chemistrybkssps.com/wp-content/downloads/n5925494f6f8a6.pdfTo prepare crystal of potash alum July – 2017 2. Volumetric analysis a. Prepare 250 ml of M/10 oxalic acid solution

MTC Model BT-ML Battery Transporter Specification Sheet · Specification Sheet for the MTC Model BT-ML Battery Transporter Keywords battery transporter, industrial battery transporter,

MTC Model BT-ML Battery Transporter Specification Sheet · Specification Sheet for the MTC Model BT-ML Battery Transporter Keywords battery transporter, industrial battery transporter,

THE TPACK MODEL TO PREPARE AND EVALUATE LESSON …

THE TPACK MODEL TO PREPARE AND EVALUATE LESSON …

ADENDO AL MANUAL DEL OPERADOR · en los hombros, costillas y area del rabo. • Prepare diluciones en serie usando 10 mL de agua de lavado en 90 mL de bufer. • Analice 1 mL del

ADENDO AL MANUAL DEL OPERADOR · en los hombros, costillas y area del rabo. • Prepare diluciones en serie usando 10 mL de agua de lavado en 90 mL de bufer. • Analice 1 mL del

0.2 ml PCR tube FRM Example protocol · 2017-02-24 · Prepare priming buffer 500 µl RBF 500 µl Nuclease-free water 0.2 ml PCR tube 0.2 ml PCR tube Before start checklist MinION™

0.2 ml PCR tube FRM Example protocol · 2017-02-24 · Prepare priming buffer 500 µl RBF 500 µl Nuclease-free water 0.2 ml PCR tube 0.2 ml PCR tube Before start checklist MinION™

· 2019. 4. 15. · Helmet Brand (a) Helmet Model (b) P3 Adapter (c) H31 H510 H520 H540 AF ML ML ML ML 3M™ Airstream AH1, AH4, AH7, AE ML ML ML L HT-701, HT-702, HT-705, HT-707 3M™

· 2019. 4. 15. · Helmet Brand (a) Helmet Model (b) P3 Adapter (c) H31 H510 H520 H540 AF ML ML ML ML 3M™ Airstream AH1, AH4, AH7, AE ML ML ML L HT-701, HT-702, HT-705, HT-707 3M™