The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of...
Transcript of The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of...
![Page 1: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/1.jpg)
The Essentials of Effective Machine Learning
![Page 2: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/2.jpg)
Scott ErnstDirector of Data
Science & Engineering
![Page 3: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/3.jpg)
Reinventing Employee Scheduling Software
A better, happier, more efficient hourly workforce
![Page 4: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/4.jpg)
Computational Astrophysics
What happens
when stars collide?
“Magnetohydrodynamic Shock-wave Stability Simulations”
![Page 5: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/5.jpg)
ML-Enhanced CG Animation & VFX
Can CG Characters
Walk without Animators?
“Supervised & Unsupervised Behavioral Character Locomotion”
![Page 7: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/7.jpg)
Dinosaur Data Science“Predict behavioral information from massive tracksite in Switzerland”
![Page 8: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/8.jpg)
Excavation Site Map
![Page 9: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/9.jpg)
What to do?
● No Legacy Constraints
● Freedom of Choice
● Support from Leadership
● Help from Peers
![Page 10: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/10.jpg)
The Essentials of Effective Machine Learning
![Page 11: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/11.jpg)
Machine LearningA very real example
![Page 12: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/12.jpg)
data = load_data()1
The Very Real Example
![Page 13: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/13.jpg)
data = load_data()
model = Model()
1
2
The Very Real Example
![Page 14: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/14.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
1
2
3
The Very Real Example
![Page 15: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/15.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
1
2
3
4
The Very Real Example
![Page 16: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/16.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
1
2
3
4
5
The Very Real Example
![Page 17: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/17.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example
![Page 18: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/18.jpg)
$ python example.py
Accuracy: 94.74%
The Very Real Example
![Page 19: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/19.jpg)
VictoryMachine Learning is Easy!
![Page 20: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/20.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example
![Page 21: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/21.jpg)
data = load_data()
model = FancierModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Improved Very Real Example
![Page 22: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/22.jpg)
The Improved Very Real Example
$ python improved_example.py
Accuracy: 95.66%
$ python example.py
Accuracy: 94.74%
![Page 23: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/23.jpg)
Improved VictoryMachine Learning is very Easy!
![Page 24: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/24.jpg)
data = load_data()
model = FancierModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Improved Very Real Example
![Page 25: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/25.jpg)
data = load_data()
model = EvenFancierModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Even More Improved Very Real Example
![Page 26: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/26.jpg)
The Even More Improved Very Real Example
$ python even_more_improved_example.py
Accuracy: 97.09%
$ python improved_example.py
Accuracy: 95.66%
![Page 27: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/27.jpg)
Even Bigger VictoryMachine Learning is really Easy!
![Page 28: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/28.jpg)
data = load_data()
model = EvenFancierModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Even More Improved Very Real Example
![Page 29: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/29.jpg)
data = load_data()
model = SuperFancyModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Super Improved Very Real Example
![Page 30: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/30.jpg)
The Super Improved Very Real Example
$ python super_improved_example.py
Accuracy: 97.11%
$ python even_more_improved_example.py
Accuracy: 97.09%
![Page 31: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/31.jpg)
Super VictoryMachine Learning is super Easy!
![Page 32: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/32.jpg)
So What Now?
![Page 34: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/34.jpg)
(555) 555-1234 [email protected] https://linkedin.coim/in/mlmaster
Resume
Scott Ernst
Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency.
![Page 35: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/35.jpg)
(555) 555-1234 [email protected] https://linkedin.coim/in/mlmaster
Resume
Scott Ernst
Skills
Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency.
![Page 36: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/36.jpg)
(555) 555-1234 [email protected] https://linkedin.coim/in/mlmaster
Resume
Scott Ernst
Skills
Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency.
● Model
Expert in Machine Learning with:
● FancyModel
● FancierModel
● SuperFancyModel
![Page 37: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/37.jpg)
Machine Learning is a collection of tools
![Page 38: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/38.jpg)
Just because you can use a hammer
doesn’t mean you can build a house.
![Page 39: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/39.jpg)
Some Historical Perspective
![Page 40: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/40.jpg)
DOTCOM1995-2001
BUBBLE
![Page 41: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/41.jpg)
Answer to the Ultimate Question of Life, the
Universe, and Everything...
Circa 2000
![Page 42: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/42.jpg)
Web Development
![Page 43: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/43.jpg)
Success & Failures
![Page 44: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/44.jpg)
One of the biggest failures at Boo was to assume that [web development] was not a technology issue. Up through launch and beyond, the [web] team was first reporting to business development and then to marketing.
Boo.com Postmortem
- Tristan Louis, CTO
![Page 45: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/45.jpg)
Joel Spolsky
Co-Founder & CEO
Founder
![Page 46: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/46.jpg)
The Joel TestYes/No Questions
For Assessing the Quality of a Software Team
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do programmers have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers?
11. Do new candidates write code during their interview?
12. Do you do hallway usability testing?
![Page 47: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/47.jpg)
Answer to the Ultimate Question of Life, the
Universe, and Everything...
Circa 2019
![Page 48: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/48.jpg)
Machine Learning & AI
![Page 49: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/49.jpg)
What is the “Joel Test”
for Machine Learning?
![Page 50: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/50.jpg)
Keeping in mind that
Success is not the absenceof complete failure
![Page 51: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/51.jpg)
RoIReturn on Investment
![Page 52: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/52.jpg)
Bad RoI
OutcomeInvest
+$100K-$200K
$0 $0.4M $0.8M $1.2M-$1.2M -$0.8M -$0.4M
![Page 53: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/53.jpg)
Better RoI
OutcomeInvest
+$600K-$200K
$0 $0.4M $0.8M $1.2M-$1.2M -$0.8M -$0.4M
![Page 54: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/54.jpg)
In Machine Learning
Time is the Investment
![Page 55: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/55.jpg)
Bad RoI
OutcomeInvest
+$100K
$0 $0.4M $0.8M $1.2M
-100k person hours
![Page 56: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/56.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Better RoI
+$600K
-60k person hours
![Page 57: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/57.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Much Better RoI
+$600K
-600 person hours
![Page 58: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/58.jpg)
The Holistic Process
![Page 59: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/59.jpg)
Machine Learning Process (Simplified)
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
![Page 60: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/60.jpg)
Machine Learning Process (Simplified)
COLLECT
1PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5VALIDATE
6
![Page 61: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/61.jpg)
Machine Learning Process (Simplified)
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
![Page 62: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/62.jpg)
A Lot More to it than .fit().predict()
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
![Page 63: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/63.jpg)
“80% of what we call analytics is not analytics at
all but just hard work”- Werner Vogels, CTO @ Amazon.com
![Page 64: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/64.jpg)
Degrees of Execution Quality
Poor
OK
Good
![Page 65: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/65.jpg)
The Ideal Scenario
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
Quality
![Page 66: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/66.jpg)
Common Case #1“The Garbage Plant”
![Page 67: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/67.jpg)
“Garbage Plant”
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
Poor experiments & improper data collection produce garbage results
Quality
![Page 68: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/68.jpg)
Data is Incorrect
![Page 69: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/69.jpg)
Some Data is Incorrect
![Page 70: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/70.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example (again)
![Page 71: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/71.jpg)
data = add_noise(load_data(), amount=0.01)
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example + Noise
![Page 72: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/72.jpg)
The Very Real Example with 1% Noise
$ python example_with_noise.py
Accuracy: 81.58%
$ python example.py
Accuracy: 94.74%
![Page 73: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/73.jpg)
@ When I Work
2.1
1400
250+
billion / month
/ second
distinct streams
volume
velocity
variety
2 second availability
![Page 74: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/74.jpg)
Basic Toolchain
Ingest Store Consume
![Page 75: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/75.jpg)
How do these Tools Help Overcome...
EntropyFragmented, Inconsistent & Disparate Data
![Page 76: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/76.jpg)
BiasesThumbs on the Scale
How do these Tools Help Overcome...
![Page 77: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/77.jpg)
A Maslow’s Hierarchy of Needs
“food” “shelter” “water”
Ingest Store Consume
![Page 78: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/78.jpg)
A Maslow’s Hierarchy of Needs
Ingest Store Consume
Governance
![Page 79: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/79.jpg)
Strict Governance
Data Catalog
Quarantine
Ingest
Validated Stream
Data Lake
![Page 80: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/80.jpg)
Data Immutability: Write Once
Validated Stream
Data Lake
Read Only
Write Once
![Page 81: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/81.jpg)
Common Case #2“The Silver Bullet”
![Page 82: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/82.jpg)
Quality
“Silver Bullet”
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
Machine Learning will save us
![Page 83: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/83.jpg)
Epsilon Modeling
ε
![Page 84: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/84.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Machine Learning RoI
+$600K
-600 person hours
![Page 85: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/85.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Machine Learning RoI
+$600K
-600 person hours
$ python version-1.py
Accuracy: 97.09%
![Page 86: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/86.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Machine Learning RoI
+$600K-200 person hours
-600 person hours
![Page 87: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/87.jpg)
OutcomeInvest
$0 $0.4M $0.8M $1.2M
Machine Learning RoI
+$600K + ???-200 person hours
-600 person hours
$ python version-2.py
Accuracy: 97.11%
![Page 88: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/88.jpg)
Does the +Δ0.02% Deliver Enough Additional Value?
$ python version-2.py
Accuracy: 97.11%
$ python version-1.py
Accuracy: 97.09%
![Page 89: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/89.jpg)
Netflix Envy
![Page 90: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/90.jpg)
![Page 91: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/91.jpg)
Less Successful Cases
$ python version-1.py
Accuracy: 27.4%
![Page 92: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/92.jpg)
Less Successful Cases
$ python version-2.py
Accuracy: 27.7%
+
$ python version-1.py
Accuracy: 27.4%
![Page 93: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/93.jpg)
Less Successful Cases
+ +
$ python version-3.py
Accuracy: 28.3%
$ python version-2.py
Accuracy: 27.7%
![Page 94: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/94.jpg)
Less Successful Cases
+ + +
$ python version-4.py
Accuracy: 29.0%
$ python version-3.py
Accuracy: 28.3%
![Page 95: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/95.jpg)
Future RoI: Unknown
OutcomeInvest
$0 $0.4M $0.8M $1.2M-$1.2M -$0.8M -$0.4M
?????????? ??????????
![Page 96: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/96.jpg)
Non-Tech Example
Blockbuster Movies
![Page 97: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/97.jpg)
Example: Rogue One
-$0.4B
-$0.8B
-$1.2B
$0
+$0.4B
+$0.8B
+$1.2B+$1.1B
-$520M
![Page 98: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/98.jpg)
Example: Solo
-$0.4B
-$0.8B
-$1.2B
$0
+$0.4B
+$0.8B
+$1.2B
+$400M
-$450M
Earned
Projected
![Page 99: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/99.jpg)
E-Commerce Recommendation Engine
![Page 100: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/100.jpg)
“When you’re fundraising, it’s AI.
When you’re hiring, it’s ML.
When you’re implementing,
it’s logistic regression.”— everyone on Twitter ever
![Page 101: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/101.jpg)
Common Case #3“The Academic Exercise”
![Page 102: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/102.jpg)
Quality
“Academic Exercise”
COLLECT
1
PROCESS
2
RESEARCH
3DEVELOP
4
DEPLOY
5
VALIDATE
6
Subject Matter Experts reign supreme!
![Page 103: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/103.jpg)
Highly Interdisciplinary Field
Machine Learning Skills
Requires expertise in a very wide range of skills
![Page 104: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/104.jpg)
First Hire
Expertise
Extreme
Low
How were they chosen?
Machine Learning Skills
![Page 105: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/105.jpg)
Selection Bias
Expertise
Low
A Data Scientist is someone like me...
Machine Learning Skills
Extreme
![Page 106: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/106.jpg)
A Team Missing Skills
Expertise
Low
Important elements for success are lacking
Machine Learning Skills
Extreme
![Page 107: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/107.jpg)
Built in the LaboratoryWorks in the Laboratory
Can’t Survive in the Wild
![Page 108: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/108.jpg)
Local Development
stopwatch.start()result = model.fit(data)stopwatch.stop()print(stopwatch.elapsed())print(results)
Elapsed: 240 ms
******* RESULTS ******* Data: 200k rows Mean: 12.4Variance: 4.2 Error: 2.3
![Page 109: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/109.jpg)
The Dream of Transparent Scaling
stopwatch.start()result = model.fit(data)stopwatch.stop()print(stopwatch.elapsed())print(results)
Elapsed: 320 ms
******* RESULTS ******* Data: 20B rows Mean: 12.4Variance: 4.2 Error: 2.3
![Page 110: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/110.jpg)
Not a Reality
stopwatch.start()result = model.fit(data)stopwatch.stop()print(stopwatch.elapsed())print(results)
Elapsed: 320 ms
******* RESULTS ******* Data: 20B rows Mean: 12.4Variance: 4.2 Error: 2.3
![Page 111: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/111.jpg)
Common Case #4“Doing Agile”
![Page 112: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/112.jpg)
Quality
“Doing Agile”
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
What looks good in our tickets is...
![Page 113: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/113.jpg)
≠Data Engineering
ApplicationEngineering
![Page 114: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/114.jpg)
Applications can be Engineered“We will be able to deliver value VN in iteration N.”
Iteration 1 Iteration 2 Iteration N
Yields V1 Yields V2 Yields VN
“We will be able to add the automatic logout feature next week.”
![Page 115: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/115.jpg)
Insight cannot be Engineered“We might be able to deliver insight IN in iteration N.”
Iteration 1 Iteration 2 Iteration N
Yields I1? Yields I2? Yields IN?
“We might be able to reduce false positives by 5% next week.”
![Page 116: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/116.jpg)
Insight cannot be Engineered“We weren’t able to deliver insight IN in iteration N.”
Iteration 1 Iteration 2 Iteration N
Yields T1 Yields T2 Yields TN
“We weren’t able to reduce false positives by 5% last week.”
![Page 117: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/117.jpg)
shift into
Task-based Deliverables
Value-based Deliverables
![Page 118: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/118.jpg)
Tasks can be Engineered“We will be able to finish task TN in iteration N.”
Iteration 1 Iteration 2 Iteration N
Yields T1 Yields T2 Yields TN
“We will be able to add a new feature to the model next week.”
![Page 119: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/119.jpg)
Engineer for InsightDon’t Try to Engineer Insight
![Page 120: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/120.jpg)
Engineers Shouldn’t Write ETLA Guide to Building a High Functioning Data Science Department
Jeff MagnssonVP Data Platform
https://multithreaded.stitchfix.com/blog/2016/03/16/engineers-shouldnt-write-etl/
engineers must deploy platforms, services, abstractions, and
frameworks that allow the data scientists to conceive of,
develop, and deploy their ideas ... I like to think of it in terms
of Lego blocks. Engineers design new Lego blocks that data
scientists assemble in creative ways to create new data
science.
![Page 121: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/121.jpg)
Increased Operational Tempo“It’s highly likely we’ll be able to deliver insight IN in iteration N.”
Iteration 1 Iteration 2 Iteration N
Yields I1... Yields I2... Yields IN...
“We’ll likely be able to reduce false positives by 5% next week.”
![Page 122: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/122.jpg)
Common Case #5“Blindly Charging Ahead”
![Page 123: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/123.jpg)
“Blindly Charging Ahead”
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
What effect our we even having...
Quality
![Page 124: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/124.jpg)
“All models are wrong but some are useful.”
- George Box
https://en.wikipedia.org/wiki/George_E._P._Box
![Page 125: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/125.jpg)
ML Always has an Answer
But how wrong is it?
![Page 126: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/126.jpg)
Solar System Orbits
Heliocentric(Sun is the center)
![Page 127: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/127.jpg)
Solar System Orbits
Geocentric(Earth is the center)
![Page 129: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/129.jpg)
Two “Valid” Solutions
![Page 130: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/130.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example (Yet Again)
![Page 131: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/131.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
The Very Real Example (Yet Again)
![Page 132: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/132.jpg)
data = load_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
Modify for a Rare Event Data Set
![Page 133: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/133.jpg)
data = load_rare_event_data()
model = Model()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
Modify for a Rare Event Data Set
![Page 134: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/134.jpg)
data = load_rare_event_data()
model = AlwaysFalseModel()
model.fit(data.training_features, data.training_labels)
predictions = model.predict(data.testing_features)
correct = count_correct(
expected=data.testing_labels,
predicted=predictions
)
print(f'Accuracy: {100 * correct / len(predictions):,.2f}%')
1
2
3
4
5
6
Modify for a Rare Event Data Set
![Page 135: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/135.jpg)
$ python rare_event_example.py
Accuracy: 95.00%
The Rare Event Example
![Page 136: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/136.jpg)
$ python rare_event_example.py
Accuracy: 95.00%
The Rare Event Example
Because True only occurs 5% of the time.
![Page 137: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/137.jpg)
Mocking a Heuristic as a scikit-learn Estimator
Eric NessSr Data Scientist
https://medium.com/when-i-work-data/mocking-a-heuristic-as-a-scikit-learn-estimator-9200bd2fb100
Creating mock models using a heuristic is an excellent way to
remove bottlenecks in the development cycle. [They are also
useful] in establishing the minimum performance necessary
for a model to be valuable. For example, if the model is
trying to predict which customers will leave and which will
stay, then a naive model might predict that all customers will
stay. While it has high accuracy, it’s precision will be poor.
Any viable model will need to beat the naive model’s
performance.
![Page 138: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/138.jpg)
More than a ScoreUnderstanding is Critical
![Page 139: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/139.jpg)
Two “Valid” Solutions
![Page 140: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/140.jpg)
Domain Expertise is CriticalFor big data to mature beyond marketing hype towards truly transformative solutions, it must “grow up” out of the computer science labs that gave birth to it and spend more time on understanding the domain-specific [problems] it is applied to than on the computing algorithms that operationalize them.
- Alev Leetaru
https://www.wired.com/2014/06/how-to-teach-heartless-computers-to-really-get-what-were-feeling/
![Page 141: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/141.jpg)
E-Commerce Recommendation Engine
![Page 142: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/142.jpg)
The Ideal Scenario
COLLECT
1
PROCESS
2
RESEARCH
3
DEVELOP
4
DEPLOY
5
VALIDATE
6
Quality
![Page 143: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/143.jpg)
Self Mockery
Kevin SchirooData Engineer
https://medium.com/when-i-work-data/self-mockery-2f6eabf27b21
One of the biggest reasons that I believe we succeed in
managing all of [our many] projects is our commitment to
our practice, to not only focus on the final results we deliver,
but also on the path we take to deliver them.
![Page 144: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/144.jpg)
So what is the “Joel Test”
for Machine Learning?
![Page 145: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/145.jpg)
![Page 146: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/146.jpg)
.fit().predict()
What I do Know
![Page 147: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/147.jpg)
Machine Learning is a powerful set of tools that require a holistic approach to use effectively.
![Page 148: The Essentials of Effective Machine Learning...Experienced Machine Learning practitioner capable of solving challenging problems with creativity and efficiency. Model Expert in Machine](https://reader036.fdocuments.net/reader036/viewer/2022062602/5ec45ef44e0713237519e64c/html5/thumbnails/148.jpg)
The Essentials of Effective Machine Learning
Scott Ernst● [email protected]● linkedin.com/in/swernst/