Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2...
Transcript of Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2...
![Page 1: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/1.jpg)
© 2018 KNIME AG. All rights reserved.
Guided Analytics for Machine Learning Automation
Christian Dietz, Simon Schmid
KNIME
![Page 2: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/2.jpg)
© 2018 KNIME AG. All rights reserved. 2
Automating Everything?
Automating Data Proc:• Feature Selection• Feature Construction• Data Cleaning
Automating Analytics:• Parameter Optimization• Model Selection• Ensemble Construction
Automating Data Integration:• Parsing• Record Matching• …
Human Input Needed!• Data Selection – Is this relevant?!• Analysis Goal – What is interesting?• Exploration – This looks weird?...
![Page 3: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/3.jpg)
© 2018 KNIME AG. All rights reserved. 3
Automating Everything?
Automating Data Proc:• Feature Selection• Feature Construction• Data Cleaning
Automating Analytics:• Parameter Optimization• Model Selection• Ensemble Construction
Automating Data Integration:• Parsing• Record Matching• …
Human Input Needed!• Data Selection – Is this relevant?!• Analysis Goal – What is interesting?• Exploration – This looks weird?...
![Page 4: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/4.jpg)
© 2018 KNIME AG. All rights reserved. 5
Building a Guided Automation Workflow
Interaction Points
Automated
![Page 5: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/5.jpg)
© 2018 KNIME AG. All rights reserved. 6
Guided Automation: Automation + Interaction
![Page 6: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/6.jpg)
© 2018 KNIME AG. All rights reserved. 7
Guided Automation: Automation + Interaction
![Page 7: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/7.jpg)
© 2018 KNIME AG. All rights reserved. 8
Guided Automation: Automation + Interaction
![Page 8: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/8.jpg)
© 2018 KNIME AG. All rights reserved. 9
Guided Automation: Automation + Interaction
![Page 9: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/9.jpg)
© 2018 KNIME AG. All rights reserved. 10
Guided Automation: Automation + Interaction
KNIME Server
WebPortal
Workflow
![Page 10: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/10.jpg)
© 2018 KNIME AG. All rights reserved. 11
Guided Automation on KNIME Server
Live Demo
![Page 11: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/11.jpg)
© 2018 KNIME AG. All rights reserved. 12
Scoring Workflow
![Page 12: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/12.jpg)
© 2018 KNIME AG. All rights reserved. 13
Scoring Workflow
KNIME Server
![Page 13: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/13.jpg)
© 2018 KNIME AG. All rights reserved. 14
Customize the Blueprint for Text Processing
Added by in-house expert.
![Page 14: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/14.jpg)
© 2018 KNIME AG. All rights reserved. 15
Thank You!
Download workflow from knime.com and get started!
![Page 15: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/15.jpg)
16© 2018 KNIME AG. All rights reserved.
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.
![Page 16: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/16.jpg)
© 2018 KNIME AG. All rights reserved. 18
![Page 17: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/17.jpg)
© 2018 KNIME AG. All rights reserved. 19
Guided Analytics for Machine Learning Automation
![Page 18: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/18.jpg)
© 2018 KNIME AG. All rights reserved. 20
Guided Analytics for Machine Learning Automation
![Page 19: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/19.jpg)
© 2018 KNIME AG. All rights reserved. 21
Guided Analytics for Machine Learning Automation
![Page 20: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/20.jpg)
© 2018 KNIME AG. All rights reserved. 22
![Page 21: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/21.jpg)
© 2018 KNIME AG. All rights reserved. 23
![Page 22: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/22.jpg)
© 2018 KNIME AG. All rights reserved. 24
![Page 23: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/23.jpg)
© 2018 KNIME AG. All rights reserved. 25
Guided Analytics for Machine Learning Automation
![Page 24: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/24.jpg)
© 2018 KNIME AG. All rights reserved. 26
![Page 25: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/25.jpg)
© 2018 KNIME AG. All rights reserved. 27
![Page 26: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/26.jpg)
© 2018 KNIME AG. All rights reserved. 28
![Page 27: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/27.jpg)
© 2018 KNIME AG. All rights reserved. 29
![Page 28: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/28.jpg)
© 2018 KNIME AG. All rights reserved. 30
![Page 29: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/29.jpg)
© 2018 KNIME AG. All rights reserved. 31
What do we Automate?
• Data Cleansing
– Missing value handling, calculate statistics, outlier detection
• Feature Engineering
– Mathematical transformations, feature combinations, dimensionality reduction and more
• Feature Selection
– Forward feature selection, backward feature elimination, genetic algorithm etc.
![Page 30: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/30.jpg)
© 2018 KNIME AG. All rights reserved. 32
What do we Automate?
Automatically try a lot of model parameters.
Model Learning
• Parameter Optimization
![Page 31: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/31.jpg)
© 2018 KNIME AG. All rights reserved. 33
What do we Automate?
• Model Selection
– Try many models, but in an automated way.
![Page 32: Guided Analytics for Machine Learning Automation...© 2018 KNIME AG. All rights reserved. 2 Automating Everything? Automating Data Proc: • Feature Selection • Feature Construction](https://reader033.fdocuments.net/reader033/viewer/2022042220/5ec604ac5638540e6d6ee4c0/html5/thumbnails/32.jpg)
© 2018 KNIME AG. All rights reserved. 34
What do we Automate?
• Model Selection and Parametrization
Model Learning
Model Learning
Model Learning
Check out Daria’s Blogpost “Stuck in the Nine Circles of Hell? Try Parameter Optimization & A Cup Of Tea” (05/28/18)