Machine Learning The Key Ingredient to Self-Driving Data Center
-
Upload
sergey-a-razin -
Category
Data & Analytics
-
view
478 -
download
3
Transcript of Machine Learning The Key Ingredient to Self-Driving Data Center
![Page 1: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/1.jpg)
Machine Learning as The Key Ingredient to
“Self-Driving” Data Center Sergey A. Razin
CTO, SIOS Technology Corp.
![Page 2: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/2.jpg)
2
Who is SIOS?
![Page 3: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/3.jpg)
Who is SIOS?
3
![Page 4: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/4.jpg)
How did IT
start?
![Page 5: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/5.jpg)
How did IT start?
5
![Page 6: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/6.jpg)
How did IT started?
6
![Page 7: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/7.jpg)
How did IT started?
7
![Page 8: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/8.jpg)
![Page 9: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/9.jpg)
9
![Page 10: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/10.jpg)
10
API
![Page 11: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/11.jpg)
11
![Page 12: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/12.jpg)
Operational Scale & Complexity in Virtualized IT Underlies the Problem
12
# of Data Points for 7 day (1 week) (over 60B for 10,000 VMs)
VMs
# of Objects
# of Metrics
0
10,000,000,000
20,000,000,000
30,000,000,000
40,000,000,000
50,000,000,000
60,000,000,000
70,000,000,000
1001,000
3,0005,000
10,000
Axi
s Ti
tle
100 1,000 3,000 5,000 10,000
VMs 100 1,000 3,000 5,000 10,000
# of Objects 2,000 15,000 50,000 100,000 200,000
# of Metrics 604,800,000 4,536,000,000 15,120,000,000 30,240,000,000 60,480,000,000
![Page 13: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/13.jpg)
13
What is Machine
Learning?
![Page 14: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/14.jpg)
Does ML exist today?
14
![Page 15: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/15.jpg)
Gap In Tools
15
![Page 16: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/16.jpg)
16
![Page 17: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/17.jpg)
17
Goodbye, Big Data!
Hello, Smart Data!
![Page 18: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/18.jpg)
We have a problem here…
We can’t see it UNLESS we have tools
![Page 19: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/19.jpg)
Conventional Tools
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 19
Data
Still Dealing With Noise
Conventional Tools
![Page 20: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/20.jpg)
State of The Productized Art
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 20
![Page 21: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/21.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 21
![Page 22: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/22.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 22
![Page 23: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/23.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 23
![Page 24: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/24.jpg)
What is the problem? Why Conventional Tools Can’s Solve It?
• Understanding of Data Semantics because: • Data is Big
• Data is complex
• Data is high-dimensional (with a lot of interrelationships and interplay)
• Data is features rich
• individual measurable property of a phenomenon being observed (examples of the data features: latencies, IOPS, etc.)
• Solution required different type of tool
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 24
![Page 25: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/25.jpg)
What if there was a solution?
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 25
![Page 26: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/26.jpg)
Topological Behavior Analysis
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 26
![Page 27: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/27.jpg)
Topological Data Analysis (TDA)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 27
TDA
Mathematics (Topology) Data
Shape
![Page 28: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/28.jpg)
Topological Data Analysis (TDA)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 28
NOISE
![Page 29: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/29.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 29
Topological Data Analysis (TDA)
DATA HAS SHAPE
![Page 30: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/30.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 30
DATA HAS MEANING
Topological Data Analysis (TDA)
![Page 31: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/31.jpg)
Now back to user’s problem…
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 31
![Page 32: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/32.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 32
Topological Behavior Analysis (TBA)
![Page 33: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/33.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 33
Topological Behavior Analysis (TBA)
![Page 34: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/34.jpg)
IT Operations Is A Big Data Problem
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 34
Noise
![Page 35: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/35.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 35
SIOS iQ – Applying Big Data Principles Topological Behavior Analysis
![Page 36: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/36.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 36
Topological Behavior Analysis & Machine Learning
![Page 37: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/37.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 37
Topological Behavior Analysis & Machine Learning
![Page 38: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/38.jpg)
Copyright @ 2015 SIOS Technology Corp. All rights reserved. 38
Topological Behavior Analysis (TBA)
![Page 39: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/39.jpg)
39
What’s Next?
![Page 40: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/40.jpg)
Self-Driving Datacenter
40
![Page 41: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/41.jpg)
41
Why should IT
brush up on
Algorithms?
![Page 42: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/42.jpg)
Expert Knowledge + Machine Intelligence
42
![Page 43: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/43.jpg)
43
Stay Relevant - Education
Machine Learning Course: https://www.coursera.org/learn/machine-learning
![Page 44: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/44.jpg)
44
Stay Relevant - Education
Data Scientist Certification Program: https://www.coursera.org/specialization/jhudatascience/1
![Page 45: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/45.jpg)
DIY – Machine Learning
http://www.r-project.org/
http://www.cs.waikato.ac.nz/ml/weka/
http://madlib.net/
https://mahout.apache.org/
http://jubat.us/en/
![Page 46: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/46.jpg)
46
How about those latencies?
Anomaly Detection – K-means Clustering
![Page 47: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/47.jpg)
47
Outlier Detection Using K-means
Anything outside 3 standard deviations is an outlier
![Page 48: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/48.jpg)
Social Graph -> IT Graph
http://neo4j.com/
FlockDB
https://github.com/twitter/flockdb
http://giraph.apache.org/
![Page 49: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/49.jpg)
49
Neo4j
https://www.vmware.com/support/pubs/sdk_pubs.html
API
![Page 50: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/50.jpg)
50
Want to see it
in Action? http://us.sios.com/
![Page 51: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/51.jpg)
SIOS iQ™ Machine Learning Analytics for optimizing
VMware environments
![Page 52: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/52.jpg)
52
SIOS iQ™ Machine Learning Analytics for optimizing VMware environments
Participate in our SIOS iQ™
usability study and have a coffee on us!
![Page 53: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/53.jpg)
53
• Always Interested
Job Opportunities @ SIOS
Great People. Exciting and Innovative Projects.
![Page 54: Machine Learning The Key Ingredient to Self-Driving Data Center](https://reader033.fdocuments.net/reader033/viewer/2022051404/58ef658f1a28ab50358b4575/html5/thumbnails/54.jpg)