All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work
-
Upload
brian-mac-namee -
Category
Data & Analytics
-
view
237 -
download
1
Transcript of All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work
![Page 1: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/1.jpg)
AllModelsAreWrong,ButSomeAreUseful:6LessonsForMakingPredictiveAnalyticsWorkDr.BrianMacNameebrian.macnamee@ucd.ie@brianmacnamee
![Page 2: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/2.jpg)
machinelearning
ar,ficialintelligence
datascience
cogni,vecompu,ng
bigdata
InspiredbyBrendanTierneyh:p://www.oraly,cs.com/2012/06/data-science-is-mul,disciplinary.html
deeplearning
![Page 3: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/3.jpg)
ar#ficialintelligence
datascience
cogni#vecompu#ng
bigdata
deeplearning
InspiredbyBrendanTierneyh:p://www.oraly#cs.com/2012/06/data-science-is-mul#disciplinary.html
machinelearning
![Page 4: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/4.jpg)
![Page 5: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/5.jpg)
![Page 6: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/6.jpg)
if LOAN-SALARY RATIO < 1.5 then OUTCOME=’repay’
else if LOAN-SALARY RATIO > 4 then OUTCOME=’default’
else if AGE < 40 and OCCUPATION =’industrial’ then OUTCOME=’default’
else OUTCOME=’repay’
end if
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 7: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/7.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 8: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/8.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 9: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/9.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 10: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/10.jpg)
Betterdatausuallybeatsbiggermodels
Predictionisalotofthings1
2 Thereisnosuchthingasafreelunch
3 LookforGoldilocks
4
Chooseyourevaluationcarefully5
6 RememberOccam’sRazor
![Page 11: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/11.jpg)
PredictionIsA LotOfThings
1
![Page 12: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/12.jpg)
![Page 13: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/13.jpg)
Predictingthevalueofan
unknownvariableatatimeinthe
future
![Page 14: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/14.jpg)
Forecast
![Page 15: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/15.jpg)
0
27.5
55
82.5
110
July September November January March May
![Page 16: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/16.jpg)
0
27.5
55
82.5
110
July September November January March May
![Page 17: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/17.jpg)
Predictthevalueofanunknownvariableassociatedwithan
object
![Page 18: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/18.jpg)
Label
![Page 19: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/19.jpg)
Image Set
![Page 20: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/20.jpg)
Image Set
ContainingNerves NotContainingNerves
![Page 21: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/21.jpg)
Predictingthepropensityof
somebodytotakeanactionatatime
inthefuture
![Page 22: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/22.jpg)
Rank
![Page 23: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/23.jpg)
Population
![Page 24: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/24.jpg)
Population
LeastLikelyToRespond
MostLikelyToRespond
![Page 25: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/25.jpg)
"Indataanalyticsapredictionisanassignmentofavaluetoanunknownvariable."FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 26: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/26.jpg)
Predictionsmeansalotofdifferentthings,whichmeanswecanapplypredictivemodellingtomanydifferentproblems.
Thinkcarefullyaboutwhattypeofdecisionyouwanttomake(label,rank,orforecast),andthendesignapredictivemodellingsolutiontobesthelpwiththat.
Lesson
![Page 27: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/27.jpg)
27
ThereIsNoSuchThingAsA FreeLunch
2
![Page 30: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/30.jpg)
"Wehavedubbedtheassociated resultsNoFreeLunchtheoremsbecausetheydemonstratethatifanalgorithmperformswellonacertainclassofproblemsthenitnecessarilypaysforthatwithdegradedperformanceonthesetofallremainingproblems."
Wolpert&Macready
"No Free Lunch Theorems for Optimization", David H. Wolpert and William G. Macready, IEEE Transactions On Evolutionary Computation, vol. 1, no. 1, 1997 http://ti.arc.nasa.gov/m/profile/dhw/papers/78.pdf
![Page 31: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/31.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 32: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/32.jpg)
Tree Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 33: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/33.jpg)
Nearest Neighbour Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 34: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/34.jpg)
Linear Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 35: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/35.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 36: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/36.jpg)
Tree Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 37: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/37.jpg)
Nearest Neighbour Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 38: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/38.jpg)
Linear Model
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 39: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/39.jpg)
Thereareahugenumberofdifferentpredictivemodellingalgorithms.Youneedtoexperimentwithlotsofdifferentones.
Lesson
randomforestdecisiontreeistonicregressionneuralnetwork nearest neighbour naive Bayes supportvectormachine logistic regressionBayesiannetworkensemblegradientboostinglinearmodelwinnow
![Page 40: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/40.jpg)
LookForGoldilocks
3
![Page 41: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/41.jpg)
●
●
●
●
●
0 20 40 60 80 100
20000
40000
60000
80000
Age
Income
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 42: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/42.jpg)
●
●
●
●
●
0 20 40 60 80 100
20000
40000
60000
80000
Age
Income
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 43: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/43.jpg)
●
●
●
●
●
0 20 40 60 80 100
20000
40000
60000
80000
Age
Income
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 44: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/44.jpg)
●
●
●
●
●
0 20 40 60 80 100
20000
40000
60000
80000
Age
Income
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 45: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/45.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
![Page 46: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/46.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 47: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/47.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 48: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/48.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 49: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/49.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 50: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/50.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 51: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/51.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 52: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/52.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 53: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/53.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
![Page 54: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/54.jpg)
0 50 100 150 200
0.1
0.2
0.3
0.4
0.5
Training Iteration
Mis
clas
sific
atio
n R
ate
Performance on Training SetPerformance on Validation Set
![Page 55: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/55.jpg)
Alwaystuneyourmodels,butbeverycarefulofoverfitting.Avalidationdatasetiscrucialhere.
Lesson
![Page 56: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/56.jpg)
56
BetterDataUsuallyBeatsBiggerModels
4
![Page 57: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/57.jpg)
DigitalImageProcessin
g,
Gonzalez&W
oods,2002
![Page 58: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/58.jpg)
DigitalImageProcessin
g,
Gonzalez&W
oods,2002
The fitting is obtained by robust linear regression(we use iteratively reweighted least squares) on the½logðfÞ; Iðf;ϕÞ$ scatter plot for f between f0 andf1 (to be specified) cycles per pixels. Robust regres-sion gives consistent estimations which are not influ-enced by the spurious spikes due to pseudoperiodicnoise. Least-squares estimation also gives the stan-dard deviation σ of the residues.
3. Find the localization of upper outliers in the averagepower spectrum as frequency pairs ðξ; ηÞ such that,under the common 3σ rule
logðgjPj2ðξ; ηÞjÞ − ½A − α logðfÞ$σ
> 3: (10)
This results in an outlier map Mpo such that
Mpoðξ; ηÞ ¼ 1 if an outlier is present at ðξ; ηÞ in the
average spectrum of the patches, and ¼ 0 otherwise.Note that a false-positive rate of 1% is expected undera Gaussian distribution. We restrict the outlier detec-tion to frequencies f > f2 (to be specified), since lowfrequencies do not correspond to repetitive patterns.
4. Resize the outlier map of size L × L to size X × Y, giv-ing a map Mo of the probable spurious spikes causedby quasiperiodic noise in the original image spectrum.Multiplying the initial image spectrum by 1 −Mo actsas a notch filter, eliminating the influence of the qua-siperiodic noise.
5. Retrieve an estimation n of the periodic noise compo-nent as the inverse Fourier transform ofMoðξ;ηÞIðξ;ηÞ,and the estimated denoised image i as i − n (i.e., theinverse transform of ½1 −Moðξ; ηÞ$Iðξ; ηÞ).
3.2 Practical ConsiderationsThe implementation details presented below do notplay a crucial role in the good behavior of the algorithm,but are given in order to enable the algorithm to berecreated.
First, since most images have discontinuities betweentheir left/right (respectively top/bottom) borders, their spec-trum shows dominant straight lines along the horizontalaxis (respectively vertical axis). To reduce these boundaryeffects, we multiply the patches p by a two-dimensional
Denoised image
100 200 300 400 500 600
50
100
150
200
250
300
350
400
4500
50
100
150
200
250Method noise
100 200 300 400 500 600
50
100
150
200
250
300
350
400
450−200
−150
−100
−50
0
50
100
150
200
Original image
80 100 120 140 160 180 200 220
320
340
360
380
400
420 0
50
100
150
200
250Denoised image
80 100 120 140 160 180 200 220
320
340
360
380
400
420 0
50
100
150
200
250
(a) (b)
(c) (d)
Fig. 11 Apollo experiment (2). (a) Denoised image. (b) Estimation of the noise. (c) Close-up view ofthe noisy image. (d) Close-up view of the denoised image.
Journal of Electronic Imaging 013003-9 Jan∕Feb 2015 • Vol. 24(1)
Sur and Grédiac: Automated removal of quasiperiodic noise using frequency domain statistics
![Page 59: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/59.jpg)
DigitalImageProcessin
g,
Gonzalez&W
oods,2002
![Page 60: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/60.jpg)
DigitalImageProcessin
g,
Gonzalez&W
oods,2002
![Page 61: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/61.jpg)
DigitalImageProcessin
g,
Gonzalez&W
oods,2002
![Page 62: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/62.jpg)
RawActivity
![Page 63: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/63.jpg)
NormalisedActivity
![Page 64: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/64.jpg)
WakeAlignedActivity
![Page 65: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/65.jpg)
CumulativeWakeAlignedActivity
![Page 66: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/66.jpg)
Activity
![Page 67: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/67.jpg)
Activity Peakactivity(day)
Variationinactivity(day)
Totalactivity(day)
Peakactivity(1sthour)
Variationinactivity(1sthour)
Totalactivity(1sthour)
Areaundercumulativeactivitycurve
…
![Page 68: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/68.jpg)
ChooseAnAlgorithm
GenerateData
TuneModelParameters
![Page 69: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/69.jpg)
ChooseAnAlgorithm
GenerateData
TuneModelParameters
![Page 70: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/70.jpg)
Developingnew,richerfeaturesisoftenabetterwaytoimprovemodelperformancethanusingmoresophisticatedmodellingtechniques.
Lesson
![Page 71: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/71.jpg)
AnAsideOnDeepLearning
![Page 72: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/72.jpg)
Deep Learning
Google Trends: http://www.google.com/trends/
2005 2007 2009 2011 2013 2015
![Page 73: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/73.jpg)
Deep-learningmethodsarerepresentaUon-learningmethodswithmul\plelevelsofrepresenta\on,
obtainedbycomposingsimplebutnon-linearmodulesthateachtransformtherepresenta\onatonelevel
(star\ngwiththerawinput)intoarepresenta\onatahigher,slightlymoreabstractlevel.
[LeCunetal,2014]
Deep Learning Yann LeCun, Yoshua Bengio & Geoffrey Hinton http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html
![Page 74: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/74.jpg)
0 1 2 3 4 5 6 7 8
9
![Page 75: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/75.jpg)
Convolu\onalneuralnetworksseemtobrilliantlyaddresstheselecUvity-invariancedilemmathatis
fundamentaltoalleffortstolearntoclassifyobjects:theyproducerepresenta\onsthatareselec\vetothe
aspectsoftheimagethatareimportantfordiscrimina\on,butthatareinvarianttoirrelevant
aspects
Convolu\onalnetworksholdrecordsforproblemsinimagerecogniUon,speechrecogniUon,andtext
classificaUonamongstotherareas
![Page 76: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/76.jpg)
![Page 77: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/77.jpg)
On Welsh Corgis, Computer Vision, and the Power of Deep Learning, Microsoft Research, 2014 http://research.microsoft.com/en-us/news/features/dnnvision-071414.aspx Rise of the machines, The Economist, 2015 http://www.economist.com/news/briefing/21650526-artificial-intelligence-scares-peopleexcessively-so-rise-machines
![Page 78: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/78.jpg)
HardwareDataAlgorithms
Applica4ons
![Page 79: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/79.jpg)
79
ChooseYourEvaluationCarefully
5
![Page 80: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/80.jpg)
![Page 81: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/81.jpg)
A marketing company working for a charity has developed two different models that predict the likelihood that donors will respond to a mail-shot asking them to make a special extra donation. Two models have been built and an evaluation experiment had been performed. Now we must decide which model to use.
![Page 82: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/82.jpg)
Prediction
TRUE FALSE
TargetTRUE 2355 337
FALSE 329 1714
ClassificationAccuracy:85.93%
Model1
![Page 83: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/83.jpg)
Prediction
TRUE FALSE
TargetTRUE 2198 494
FALSE 471 1572
ClassificationAccuracy:79.62%
Model2
![Page 84: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/84.jpg)
Model1
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 85: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/85.jpg)
Model2
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
![Page 86: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/86.jpg)
Therearemanydifferentperformancemeasuresthatwecanusetoevaluatetheperformanceofamodel.Youneedtopicktheonethatbestmatchesthedecisionsyouaretryingtomake.
Lesson
![Page 87: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/87.jpg)
87
RememberOccam’sRazor
6
![Page 88: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/88.jpg)
![Page 89: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/89.jpg)
![Page 90: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/90.jpg)
![Page 91: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/91.jpg)
![Page 92: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/92.jpg)
Timeline
Followers
Following
Tweets+ Metadata
Profile
![Page 93: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/93.jpg)
Tweets+ Metadata
Profile
![Page 94: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/94.jpg)
Tweets+ Metadata
Profile
![Page 95: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/95.jpg)
http://www.cso.ie/en/releasesandpublications/er/ibn/irishbabiesnames2014/
![Page 96: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/96.jpg)
Alwaysstartwithsimplesolutionsfirst.Onlyaddcomplexityifrequired.
Lesson
Frustrafitperpluraquodpotestfieriperpauciora(Itisfutiletodowithmorethingsthatwhichcanbedonewithfewer)
![Page 97: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/97.jpg)
Betterdatausuallybeatsbiggermodels
Predictionisalotofthings1
2 Thereisnosuchthingasafreelunch
3 LookforGoldilocks
4
Chooseyourevaluationcarefully5
6 RememberOccam’sRazor
![Page 98: All Models Are Wrong, But Some Are Useful: 6 Lessons for Making Predictive Analytics Work](https://reader033.fdocuments.net/reader033/viewer/2022042907/58cf34411a28ab00168b6027/html5/thumbnails/98.jpg)
FundamentalsofMachineLearningforPredictiveDataAnalyticsJohnKelleher,BrianMacNamee,andAoifeD'Arcy www.machinelearningbook.com
ThankYouQuestions?
TrainingCourse:FundamentalsofMachine LearningforPredictiveDataAnalyticsDublin,March21st-23rd www.theanalyticsstore.ie/training/
[email protected]@brianmacnamee