Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate...

33

Transcript of Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate...

Page 1: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”
Page 2: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

PracticalAutomatedMachineLearningonAzure

UsingAzureMachineLearningtoQuicklyBuildAISolutions

DeepakMukunthu,ParasharShah,andWeeHyongTok

Page 3: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

PracticalAutomatedMachineLearningonAzurebyDeepakMukunthu,ParasharShah,andWeeHyongTok

Copyright©2019DeepakMukunthu,ParasharShah,andWeeHyongTok.Allrightsreserved.

PrintedintheUnitedStatesofAmerica.

PublishedbyO’ReillyMedia,Inc.,1005GravensteinHighwayNorth,Sebastopol,CA95472.

O’Reillybooksmaybepurchasedforeducational,business,orsalespromotionaluse.Onlineeditionsarealsoavailableformosttitles(http://oreilly.com).Formoreinformation,contactourcorporate/institutionalsalesdepartment:[email protected].

AcquisitionsEditor:JonathanHassell

DevelopmentEditor:NicoleTaché

ProductionEditor:DeborahBaker

Copyeditor:OctalPublishing,LLC

Proofreader:SharonWilkey

Indexer:JudithMcConville

InteriorDesigner:DavidFutato

CoverDesigner:KarenMontgomery

Illustrator:RebeccaDemarest

September2019:FirstEdition

RevisionHistoryfortheFirstEdition

2019-09-20:FirstRelease

http://oreilly.com
Page 4: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Seehttp://oreilly.com/catalog/errata.csp?isbn=9781492055594forreleasedetails.

TheO’ReillylogoisaregisteredtrademarkofO’ReillyMedia,Inc.PracticalAutomatedMachineLearningonAzure,thecoverimage,andrelatedtradedressaretrademarksofO’ReillyMedia,Inc.

Theviewsexpressedinthisworkarethoseoftheauthors,anddonotrepresentthepublisher’sviews.Whilethepublisherandtheauthorshaveusedgoodfaitheffortstoensurethattheinformationandinstructionscontainedinthisworkareaccurate,thepublisherandtheauthorsdisclaimallresponsibilityforerrorsoromissions,includingwithoutlimitationresponsibilityfordamagesresultingfromtheuseoforrelianceonthiswork.Useoftheinformationandinstructionscontainedinthisworkisatyourownrisk.Ifanycodesamplesorothertechnologythisworkcontainsordescribesissubjecttoopensourcelicensesortheintellectualpropertyrightsofothers,itisyourresponsibilitytoensurethatyourusethereofcomplieswithsuchlicensesand/orrights.

978-1-492-05559-4

[LSI]

http://oreilly.com/catalog/errata.csp?isbn=9781492055594
Page 5: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Dedications

Dedicatedtomywife,kids,andparentsfortheirunconditionallove,encouragementandsupportineverythingIdo.—Deepak

Dedicatedtothewonderfulindividualsinmylife—Juliet,Nathaniel,andJayden.Mygratitudeandloveforthemisinfinite.—WeeHyong

IwouldliketothankmyparentsNitaandMahendraandmysisterVidhifortheirunconditionalloveandencouragementthroughoutmylife.IamthankfultomybuddiesatMicrosoft—Priya,Premal,Vicky,Martha,Savita,Deepti,andSagar—andmybuddiesoutsideofMicrosoft—Kevin,Ritu,Dhaval,Shamit,Priyadarshan,Pradip,andNikhil—fortheirlovingfriendship.—Parashar

Page 6: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Foreword

Ivividlyremembermyfirstundergraduateclassinartificialintelligence(AI).Myfatherhadworkedforyearson“expertsystems,”andIwasatMITtolearnfromthebesthowtoperformthiswizardry.MarvinMinsky,oneofthefoundersofthefield,eventaughtaseriesofguestlecturesthere.Itwasaboutmidwaythroughthesemesterwhenthegreatdisillusionmenthitme:“It’salljustabunchoftricks!”Therewasno“intelligence”tobefound;justabunchofbrittlerulesenginesandcleveruseofmath.Thiswasintheearly’90sandthestartofmyownpersonalAIwinter,whenIdismissedAIasnothavingmuchuse.

Yearslater,whileIwasworkingonadvertisingsystems,Ifinallysawthattherewaspowerinthis“bunchoftricks.”Algorithmsthathadbeenhand-tunedformonthsbytalentedengineerswerebeingbeatenbysimplemodelsprovidedwithlotsofdata.Isawthattheexplosionthatwastocomesimplyneededmoredataandmorecomputationtobeeffective.Overthepast5to10years,theexplosioninbothbigdataandcomputationpowerhasunleashedanindustrythathashadlotsofstartsandstopstoit.

Thistimeisdifferent.WhilethehypeaboutAIisstilltremendouslyhigh,thepotentialapplicationsofpracticalAIhavereallyjustbeguntohitthebusinessworld.TherulesorpeoplemakingpredictionstodaywillbereplacedvirtuallyeveryplacebyAIalgorithms.ThevalueAIcreatesforbusinessesistremendous,frombeingbetterabletovaluetheoilavailableinanoilfieldtobetterpredictingtheinventoryastoreshouldstockofeachnewsneaker.Evenmarginalimprovementsinthesecapabilitiesrepresentbillionsofdollarsofvalueacrossbusinesses.

We’renowinanageofAIimplementation.CompaniesareworkingtofindallthebestplacestodeployAIintheirenterprises.Oneofthebiggestchallengesismatchingthehypetoreality.HalfthecompaniesI’vetalkedtoexpectAItoperformsomekindofmagicforproblemstheyhavenoideahowtosolve.TheotherhalfareunderestimatingthepowerthatAIcanhave.WhattheyneedarepeoplewithenoughbackgroundinAItohelpthemconceiveofwhatispossibleandapplyittotheirbusinessproblems.

Page 7: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

CustomersItalktoarestrugglingtofindenoughpeoplewiththoseskills.Whiletheyhavelotsofdevelopersanddataanalystswhoareskilledandcomfortablemakingpredictionsanddecisionswithdata,theyneeddatascientistswhocanthenbuildthemodelfromthatdata.Thisbookwillhelpfillthatgap.

ItshowshowautomatedMLcanempowerdevelopersanddataanalyststotrainAImodels.IthighlightsanumberofbusinesscaseswhereAIisagreatfittothebusinessproblemandshowexactlyhowtobuildthatmodelandputitintoproduction.Thetechnologyandideasinthisbookhavebeenpressure-testedatscalewithteamsallacrossMicrosoft,includingBing,Office,AzureSecurity,internalIT,andmanymore.It’salsobeenusedbymanyexternalbusinessesusingAzureMachineLearning.

EricBoydMicrosoftCorporateVicePresident,AzureAISeptember2019

Page 8: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Preface

ConventionsUsedinThisBookThefollowingtypographicalconventionsareusedinthisbook:

Italic

Indicatesnewterms,URLs,emailaddresses,filenames,andfileextensions.

Constant width

Usedforprogramlistings,aswellaswithinparagraphstorefertoprogramelementssuchasvariableorfunctionnames,databases,datatypes,environmentvariables,statements,andkeywords.

Constant width bold

Showscommandsorothertextthatshouldbetypedliterallybytheuser.

Constant width italic

Showstextthatshouldbereplacedwithuser-suppliedvaluesorbyvaluesdeterminedbycontext.

TIPThiselementsignifiesatiporsuggestion.

NOTEThiselementsignifiesageneralnote.

WARNINGThiselementindicatesawarningorcaution.

Page 9: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

UsingCodeExamplesSupplementalmaterial(codeexamples,exercises,etc.)isavailablefordownloadathttps://oreil.ly/Practical_Automated_ML_on_Azure.

Thisbookisheretohelpyougetyourjobdone.Ingeneral,ifexamplecodeisofferedwiththisbook,youmayuseitinyourprogramsanddocumentation.Youdonotneedtocontactusforpermissionunlessyou’rereproducingasignificantportionofthecode.Forexample,writingaprogramthatusesseveralchunksofcodefromthisbookdoesnotrequirepermission.SellingordistributingaCD-ROMofexamplesfromO’Reillybooksdoesrequirepermission.Answeringaquestionbycitingthisbookandquotingexamplecodedoesnotrequirepermission.Incorporatingasignificantamountofexamplecodefromthisbookintoyourproduct’sdocumentationdoesrequirepermission.

Weappreciate,butdonotrequire,attribution.Anattributionusuallyincludesthetitle,author,publisher,andISBN.Forexample:“PracticalAutomatedMachineLearningonAzurebyDeepakMukunthu,ParasharShah,andWeeHyongTok(O’Reilly).Copyright2019DeepakMukunthu,ParasharShah,andWeeHyongTok,978-1-492-05559-4.”

Ifyoufeelyouruseofcodeexamplesfallsoutsidefairuseorthepermissiongivenabove,[email protected].

O’ReillyOnlineLearning

NOTEForalmost40years,O’ReillyMediahasprovidedtechnologyandbusinesstraining,knowledge,andinsighttohelpcompaniessucceed.

Ouruniquenetworkofexpertsandinnovatorssharetheirknowledgeandexpertisethroughbooks,articles,conferences,andouronlinelearningplatform.O’Reilly’sonlinelearningplatformgivesyouon-demandaccesstolivetrainingcourses,in-depthlearningpaths,interactivecodingenvironments,andavastcollectionoftextandvideofromO’Reillyand200+otherpublishers.Formore

https://oreil.ly/Practical_Automated_ML_on_Azure
mailto:[email protected]
http://oreilly.com
Page 10: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

information,pleasevisithttp://oreilly.com.

HowtoContactUsPleaseaddresscommentsandquestionsconcerningthisbooktothepublisher:

O’ReillyMedia,Inc.

1005GravensteinHighwayNorth

Sebastopol,CA95472

800-998-9938(intheUnitedStatesorCanada)

707-829-0515(internationalorlocal)

707-829-0104(fax)

Thewebpageforthisbooklistserrata,examples,andadditionalinformation.Youcanaccessthispageathttp://www.oreilly.com/catalog/9781492055594.

Tocommentorasktechnicalquestionsaboutthisbook,[email protected].

Formoreinformationaboutourbooks,courses,conferences,andnews,seeourwebsiteathttp://www.oreilly.com.

FindusonFacebook:http://facebook.com/oreilly

FollowusonTwitter:http://twitter.com/oreillymedia

WatchusonYouTube:http://www.youtube.com/oreillymedia

AcknowledgmentsThisbookwouldn’thavebeenpossiblewithoutgreatcontributionsfromthesefolks–thankyou!

WearethankfultoourcoworkersatMicrosoft(AzureAIproduct,marketing,

http://oreilly.com
http://www.oreilly.com/catalog/9781492055594
mailto:[email protected]
http://www.oreilly.com
http://facebook.com/oreilly
http://twitter.com/oreillymedia
http://www.youtube.com/oreillymedia
Page 11: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

andmanyotherteams)forworkingtogethertodeliverthebestenterprise-readyAzureMachineLearningservice.

NicoloFusi,forsharingdetailsonresearchthatleadtothecreationofAutomatedML(Chapter2).

SharonGillett,fortextinputstoAutomatedMLintroduction(Chapter2).

VanessaMilan,forimagesforAutomatedMLintroduction(Chapter2).

AkcharaMukunthu,forexamplescenariosforMachineLearningtaskdetection(Table2-1inChapter2).

KrishnaAnumalasettyandThomasAbraham,fortechnicalreviewofthebook.

JenStirrup,forfeedbackonthebook.

TheamazingO’Reillyteam(NicoleTache,DeborahBaker,BobRussell,JonathanHassell,BenLorica,andmanymore),forworkingwithusfromconcepttoproductionandgivingustheopportunitytowriteandsharethebookwiththecommunity.

MembersoftheAzureMachineLearningandAzureCATteam,forthesupportiveenvironmentthatenabledtheauthorstowritethebookduringtheiroffhours,andmanyweekendsandholidays.

Page 12: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

PartI.AutomatedMachineLearning

Inthispart,youwilllearnhowAutomatedMachineLearningcanhelpautomatemodeldevelopment.

Page 13: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Chapter1.MachineLearning:OverviewandBestPractices

Howarehumansdifferentfrommachines?Therearequiteafewdifferences,buthere’sanimportantone:humanslearnfromexperience,whereasmachinesfollowinstructionsgiventothem.Whatifmachinescanalsolearnfromexperience?Thatisthecruxofmachinelearning.Formachines,“datafromthepast”isthelogicalequivalentof“experience.”Machinelearningcombinesstatisticsandcomputersciencetoenablemachinestolearnhowtoperformagiventaskwithoutbeingexplicitlyprogrammedtodosoviainstructions.

Machinelearningiswidelyusedtoday,andweinteractwithiteveryday.Hereareafewexamplestoillustrate:

SearchengineslikeBingorGoogle

ProductrecommendationsatonlinestoreslikeAmazonoreBay

PersonalizedvideorecommendationsatNetflixorYouTube

Voice-baseddigitalassistantslikeAlexaorCortana

Spamfiltersforouremailinbox

Creditcardfrauddetection

Whyismachinelearningasatrendemergingsofast?Whyiseveryonesointerestedinitnow?AsshowninFigure1-1,itspopularityarisesfromthreekeytrends:bigdata,better/cheapercompute,andsmarteralgorithms.

Page 14: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-1.Machinelearninggrowth

Inthischapter,weprovideaquickrefresheronmachinelearningbyusingareal-worldexample,discusssomeofthebestpracticesthatdifferentiatesuccessfulmachinelearningprojectsfromtherest,andendwithchallengesaroundproductivityandscale.

MachineLearning:AQuickRefresherWhatdoestheprocessofbuildingamachinelearningmodellooklike?Let’sdigdeeperusingarealscenario:housepriceprediction.Wehavepasthomesalesdata,andthetaskistopredictthesalepriceforagivenhousethatjustcameontothemarketandisn’tcurrentlyinourdataset.

Forsimplicity,let’sassumethatthesizeofthehouse(insquarefeet)isthemostimportantinputattribute(orfeature)thatdetermineshousevalue.AsshowninTable1-1,wehavedatafromfourhouses,A,B,C,D,andweneedtopredictthepriceofhouseX.

Table1-1.Housepricesbasedonsize

House Size(sq.ft) Price($)

A 1300 500,000

B 2000 800,000

C 2500 950,000

D 3200 1,200,000

Page 15: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

X 1800 ?

WebeginbyplottingSizeonthex-axisandPriceonthey-axis,asshowninFigure1-2.

Figure1-2.Plottingpriceversussize

What’sthebestestimateforthepriceofhouseX?

$550,000

$700,000

$1,000,000

Let’sfigureitout.AsshowninFigure1-3,thefourpointsthatweplottedbasedonthedataformanalmoststraightline.Ifwedrawthislinethatbestfitsourdata,wecanfindtherightpointonthelineassociatedwithhouseXonthex-axisandthecorrespondingpointony-axis,whichwillgiveusourpriceestimate.

Page 16: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-3.Creatingastraightlinetofindpriceestimate

Inthiscase,thatstraightlinerepresentsourmodel—anddemonstratesalinearrelationship.Linearregressionisastatisticalapproachformodelingalinearrelationshipbetweeninputvariables(alsocalledfeature,orindependent,variables)andanoutputvariable(alsocalledatarget,ordependent,variable).Mathematically,thislinearrelationshipcanberepresentedasfollows:

where:

yistheoutputvariable;forexample,thehouseprice.

xistheinputvariable;forexample,sizeinsquarefeet.

β0istheintercept(thevalueofywhenx=0).

β1isthecoefficientforxandtheslopeoftheregressionline(“theaverageincreaseinyassociatedwithaone-unitincreaseinx).

ModelParametersβ0andβ1areknownasthemodelparametersofthislinearregressionmodel.

Page 17: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Whenimplementinglinearregression,thealgorithmfindsthelineofbestfitbyusingthemodelparametersβ0andβ1,suchthatitisascloseaspossibletotheactualdatapoints(minimizingthesumofthesquareddistancesbetweeneachactualdatapointandthelinerepresentingmodelpredictions).

Figure1-4showsthisconceptually.Dotsrepresentactualdatapoints,andthelinerepresentsthemodelpredictions.d1tod9representdistancesbetweendatapointsandthecorrespondingmodelprediction,andDisthesumoftheirsquares.Thelineshowninthefigureisthebest-fitregressionlinethatminimizesD.

Figure1-4.Regression

Asyoucansee,modelparametersareanintegralpartofthemodelanddeterminetheoutcome.Theirvaluesarelearnedfromdatathroughthemodeltrainingprocess.

HyperparametersThereisanothersetofparametersknownashyperparameters.Modelhyperparametersareusedduringthemodeltrainingprocesstoestablishthecorrectvaluesofmodelparameters.Theyareexternaltothemodel,andtheirvaluescannotbeestimatedfromdata.Thechoiceofthehyperparameterswill

Page 18: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

affectthedurationofthetrainingandtheaccuracyofthepredictions.Aspartofthemodeltrainingprocess,datascientistsusuallyspecifyhyperparametersbasedonheuristicsorknowledge,andoftentunethehyperparametersmanually.Hyperparametertuningreliesmoreonexperimentalresultsthantheory,andthusthebestmethodtodeterminetheoptimalsettingsistotrymanycombinationsandevaluatetheperformanceofeachmodel.

Simplelinearregressiondoesn’thaveanyhyperparameters.Butvariantsoflinearregression,likeRidgeregressionandLasso,do.Herearesomeexamplesofmodelhyperparametersforvariousmachinelearningalgorithms:

Thekink-nearestneighbors

Thedesireddepthandnumberofleavesinadecisiontree

TheCandsigmainsupportvectormachines(SVMs)

Thelearningrateforaneuralnetworktraining

BestPracticesforMachineLearningProjectsInthissection,weexaminebestpracticesthatmakemachinelearningprojectssuccessful.Thesearepracticaltipsthatmostcompaniesandteamsenduplearningwithexperience.

UnderstandtheDecisionProcessMachinelearning–basedsystemsorprocessesusedatatodrivebusinessdecisions.Hence,itisimportanttounderstandthebusinessproblemthatneedstobesolved,independentoftechnologysolutions—inotherwords,whatdecisionoractionneedstobetakenthatcanbeinformedbydata.Beingclearaboutthedecisionprocessiscritical.Thisstepisalsosometimesreferredtoasmappingabusinessscenario/problemtoadatasciencequestion.

Forourhouse-pricepredictionscenario,thekeybusinessdecisionforahomebuyer,is“ShouldIbuyagivenhouseatthelistedprice?”or“Whatisagoodbidpriceforthishousetomaximizemychanceofwinningthebid?”Thiscouldbemappedtothedatasciencequestion:“Whatisthebestestimateofthehousepricebasedonpastsalesdataofotherhouses?”

http://bit.ly/lasso-proj
Page 19: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Table1-2showsotherreal-worldbusinessscenariosandwhatthisdecisionprocesslookslike.

Table1-2.Understandingadecisionprocess:real-worldscenarios

Businessscenario Keydecision Datasciencequestion

Predictivemaintenance

ShouldIservicethispieceofequipment?

Whatistheprobabilitythisequipmentwillfailwithinthenextxdays?

Energyforecasting

ShouldIbuyorsellenergycontracts?

Whatwillbethelong-/short-termdemandforenergyinaregion?

Customerchurn

WhichcustomersshouldIprioritizetoreducechurn?

Whatistheprobabilityofchurnwithinxdaysforeachcustomer?

Personalizedmarketing WhatproductshouldIofferfirst? Whatistheprobabilitythatcustomerswill

purchaseeachproduct?

Productfeedback

Whichservice/productneedsattention?

Whatisthesocialmediasentimentforeachservice/product?

EstablishPerformanceMetricsAswithanyproject,performancemetricsareimportanttoguideanymachinelearningprojecttowardthepropergoalsandtoensureprogressismade.Afterweunderstandthedecisionprocess,thenextstepistoanswerthesetwokeyquestions:

Howdowemeasureprogresstowardagoalordesiredoutcome?Inotherwords,howdowedefinemetricstoevaluateprogress?

Whatwouldbeconsideredasuccess?Thatis,howdowedefinetargetsforthemetricsdefined?

Forourhouse-pricepredictionexample,weneedametrictomeasurehowcloseourpredictionsaretotheactualprice.Therearequiteafewmetricstochoosefrom.Oneofthemostcommonlyusedmetricsforregressiontasksisroot-mean-squareerror(RMSE).Thisisdefinedasthesquarerootoftheaveragesquareddistancebetweentheactualscoreandthepredictedscore,asshownhere:

Page 20: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Here,y denotesthetruevalueforthei datapoint,andŷ denotesthepredictedvalue.OneintuitivewaytounderstandthisformulaisthatitistheEuclideandistancebetweenthevectorofthetruevaluesandthevectorofthepredictedvalues,averagedbyn,wherenisthenumberofdatapoints.

FocusonTransparencytoGainTrustThereisacommonperceptionthatmachinelearningisablackboxthatjustworksmagically.Itiscriticaltounderstandthatalthoughmodelperformanceasmeasuredbymetricsisimportant,itisevenmoreimportantforustounderstandhowthemodelworks.Withoutthisunderstanding,itisdifficulttotrustthemodelandthereforedifficulttoconvincekeystakeholdersandcustomersofthebusinessvalueofmachinelearningandmachinelearning–basedsystems.

Inheavilyregulatedindustrieslikehealthcareandbanking,whicharerequiredtocomplywithregulation,interpretabilityofmodelsiscritical.Modelinterpretabilityistypicallyrepresentedbyfeatureimportance,whichtellsyouhoweachinputcolumn(orfeature)affectsthemodel’spredictions.Thisallowsdatascientiststoexplainresultingpredictionssothatstakeholderscanseewhichdatapointsaremostimportantinthemodel.

Inourhouse-pricepredictionscenario,ourtrustonthemodelwouldincreaseifthemodel,inadditiontopriceprediction,indicatedkeyinputfeaturesthatcontributedtotheoutput;forexample,housesizeandage.Figure1-5showsfeatureimportanceforourhouse-pricepredictionscenario.Noticethatageandschoolratingarethetopmostfeatures.

jth

j

Page 21: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-5.Featureimportance

EmbraceExperimentationBuildingagoodmachinelearningmodeltakestime.Aswithothersoftwareprojects,thetricktobecomingsuccessfulinmachinelearningprojectsliesinhowfastwetryoutnewhypotheses,learnfromthem,andkeepevolving.AsshowninFigure1-6,thepathtosuccessisn’tusuallyeasyandrequiresalotofpersistence,duediligence,andfailuresontheway.

Page 22: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-6.Successisnoteasy.

Herearekeyaspectsofaculturethatvaluesexperimentation:

Bewillingtolearnfromexperiments(successesorfailures).

Sharethelearningwithpeers.

Promotesuccessfulexperimentstoproduction.

Understandthatfailureisavalidoutcomeofanexperiment.

Quicklymoveontothenexthypothesis.

Refinethenextexperiment.

Don’tOperateinaSiloCustomerstypicallyexperiencemachinelearningmodelsthroughapplications.Figure1-7showshowmachinelearningsystemsaredifferentfromtraditionalsoftwaresystems.Thekeydifferenceisthatmachinelearningsystems,inadditiontocodeworkflow,mustalsoconsiderdataworkflow.

Page 23: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-7.Machinelearningsystemversustraditionalsystems

Afterdatascientistshavebuiltamachinelearningmodelthatissatisfactorytothem,theyhanditofftoanappdeveloperwhointegratesitintothelargerapplicationanddeploysit.Often,anybugsorperformanceissuesgoundiscovereduntiltheapplicationhasalreadybeendeployed.Theresultingfrictionbetweenappdevelopersanddatascientiststoidentifyandfixtherootcausecanbeaslow,frustrating,andexpensiveprocess.

Asmachinelearningentersmorebusiness-criticalapplications,itisincreasinglyclearthatdatascientistsneedtocollaboratecloselywithappdeveloperstobuildanddeploymachinelearning–poweredapplicationsmoreefficiently.Datascientistsarefocusedonthedatasciencelifecycle;namely,dataingestionandpreparation,modelbuilding,anddeployment.Theyarealsointerestedinperiodicallyretrainingandredeployingthemodeltoadjustforfreshlylabeleddata,datadrift,userfeedback,orchangesinmodelinputs.Theappdeveloperisfocusedontheapplicationlifecycle—building,maintaining,andcontinuouslyupdatingthelargerbusinessapplicationthatthemodelispartof.Bothpartiesaremotivatedtomakethebusinessapplicationandmodelworkwelltogethertomeetend-to-endperformance,quality,andreliabilitygoals.

Whatisneededisawaytobridgethedatascienceandapplicationlifecyclesmoreeffectively.Figure1-8showshowthiscollaborationcouldbeenabled.Wewillcoverthisinmoredepthlaterinthebook.

Page 24: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-8.Appdeveloperanddatascientistworkingtogether

AnIterativeandTime-ConsumingProcessInthissection,wedigdeeperintothemachinelearningprocessbyusingourhouse-pricepredictionexample.Westartedwithhousesizeastheonlyinput,andwesawtherelationshipbetweenhousesizeandhousepricetobelinear.Tocreateagoodmodelthatcanpredictpricesmoreaccurately,weneedtoexplore

Page 25: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

goodinputfeatures,selectthebestalgorithm,andtunehyperparametervalues.But,howdoyouknowwhichfeaturesaregood,andwhichalgorithmandhyperparametervalueswilldothebest?Thereisnosilverbullethere;wewillneedtotryoutdifferentcombinationsoffeatures,algorithms,andhyperparametervalues.Let’stakealookateachofthesethreestepsandthenseehowtheyapplytoourhouse-pricepredictionproblem.

FeatureEngineeringFeatureengineeringistheprocessofusingourknowledgeofthedatatocreatefeaturesthatmakemachinelearningalgorithmswork.AsshowninFigure1-9,thisinvolvesfoursteps.

Figure1-9.Featureengineering

First,weacquiredata—collectthedatawithallofthesepossibleinputvariables/featuresandgetittoausablestate.Mostreal-worlddatasetsarenotclean,andneedworktogetthedatatoalevelofqualitybeforeusingit.Thiscaninvolvethingssuchasfixingmissingvalues,removinganomaliesandpossiblyincorrectdata,andensuringthedatadistributionisrepresentative.

Nextyou’llneedtogeneratefeatures:exploregeneratingmorefeaturesfromavailabledata.Thisistypicallyusefulwhendealingwithtextdataortime-seriesdata.Text-relatedfeaturescouldbeassimpleasn-gramsandcountvectorizationorasadvancedassentimentfromreviewtext.Similarly,time-relatedfeaturescouldbeassimpleasmonthandweek-index-of-yearorascomplexastime-basedaggregations.Theseadditionalfeaturesgeneratedcanprovehelpfulinimprovingaccuracyofthemodel.

Withthiscomplete,you’llneedtotransformthedatatomakeitsuitableformachinelearning.Often,machinelearningalgorithmsrequirethatdatabe

Page 26: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

preparedinspecificwaysbeforefittingamachinelearningmodel.Forinstance,manysuchalgorithmscannotoperateoncategoricaldatadirectly,andrequireallinputvariablesandoutputvariablesbenumeric.Acategoricalvariableisavariablethatcantakeononeofalimited,andusuallyfixed,numberofpossiblevalues.Examplesofthesevariablesincludecolor(red,blue,green,etc.),country(UnitedStates,India,China,etc.),andbloodgroup(A,B,O,AB).Categoricalvariablesmustbeconvertedtoanumericalform,whichistypicallydonebyusingintegerencodingorone-hotencodingtechniques.

Thefinalstepisfeatureselection:choosingasubsetoffeaturestotrainthemodelon.Whyisthisnecessary?Whynottrainthemodelwiththefullsetoffeatures?Featureselectionidentifiesandremovestheunneeded,irrelevant,andredundantattributesfromdatathatdon’tcontribute,orcaninfactdecrease,themodel’saccuracy.Theobjectiveoffeatureselectionisthreefold:

Improvemodelaccuracy

Improvemodeltrainingtime/cost

Provideabetterunderstandingoftheunderlyingprocessoffeaturegeneration

NOTEFeatureengineeringstepsarecriticalfortraditionalmachinelearningbutnotsomuchfordeeplearning,becausefeaturesareautomaticallygenerated/inferredthroughthedeeplearningnetwork.

Webeganwithasinglefeature:housesize.Butweknowthatthepriceofahouseisdependentnotonlyonsize,butalsoonothercharacteristics.Whatotherinputfeaturescouldinfluencehouseprice?Althoughsizemightbeoneofthemostimportantinputs,herearefewmoreworthconsidering:

Zipcode

Yearbuilt

Lotsize

https://oreil.ly/4YTwv
Page 27: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Schools

Numberofbedrooms

Numberofbathrooms

Numberofgaragestalls

Amenities

AlgorithmSelectionAfterwehavechosenagoodsetoffeatures,thenextstepistodeterminethecorrectalgorithmforthemodel.Forthedatawehave,asimplelinearregressionmodelmightseemtowork.Butrememberthatwehaveonlyafewdatapoints(fourhouseswithprice)—smallenoughtoberepresentativeandsmallenoughformachinelearning.Also,linearregressionassumesalinearrelationbetweeninputfeaturesandtargetvariable.Aswecollectmoredatapoints,linearregressionmightnotremainmostrelevant,andwewillbemotivatedtoexploreothertechniques(algorithms)dependingontrendsandpatternsindata.

HyperparameterTuningAsdiscussedearlierinthischapter,hyperparametersplayakeyroleinmodelaccuracyandtrainingperformance.Hence,tuningthemisacriticalstepingettingtoagoodmodel.Becausedifferentalgorithmshavedifferentsetsofhyperparameters,thisstepoftuninghyperparametersaddstothecomplexityoftheend-to-endprocess.

TheEnd-to-EndProcessWiththatbasicunderstandingoffeatureengineering,algorithmselection,andhyperparametertuning,let’sgostepbystepthroughourhouse-pricepredictionproblem.

Let’sbeginwithSize,Lotsize,andYearbuiltfeaturesandGradientBoostedtreeswithspecifichyperparametervalues,asshowninFigure1-10.Theresultingmodelis30%accurate.Butwewanttodobetterthanthat.

Page 28: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-10.Machinelearningprocess:step1

Togetunderway,wetrydifferentvaluesofhyperparametersforthesamesetoffeaturesandalgorithm.Ifthatdoesn’timproveaccuracyofthemodeltoasatisfactorylevel,wetrydifferentalgorithms,andifthatdoesn’thelpeither,weaddmorefeatures.Figure1-11showsonesuchintermediatestate,withSchooladdedasafeatureandthek-nearestneighbors(KNN)algorithmused.Theresultingmodelis50%accuratebutstillnotgoodenough,sowecontinuethisprocessandtrydifferentcombinations.

Figure1-11.Machinelearningprocess:intermediatestate

Aftermultipleiterationsoftryingoutdifferentcombinationsoffeatures,algorithms,andhyperparametervalues,weendupwithamodelthatmeetsourcriteria,asshowninFigure1-12.

Page 29: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-12.Machinelearningprocess:bestmodel

Asyoucansee,thisisaniterativeandtime-consumingprocess.Toputthisinperspective:ifthereare10features,thereareatotalof2 (1,024)waystoselectfeatures.Ifwetryfivealgorithms,andassumingeachhasanaverageoffivehyperparameters,wearelookingatatotalof1,024×5×5=25,600iterations!

Figure1-13showsthescikit-learncheatsheetdemonstratingthatchoosingtheproperalgorithmcouldbeacomplexprobleminitself.Nowimagineaddingfeatureengineeringandhyperparametertuningontopofit.Asaresult,ittakesdatascientistsanywherefromacoupleofweekstomonthstoarriveatagoodmodel.

Figure1-13.Scikit-learnalgorithmcheatsheet(source:https://oreil.ly/xUZbU)

10

https://oreil.ly/xUZbU
Page 30: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

GrowingDemandDespitethecomplexityofthemodel-buildingprocess,demandformachinelearninghasskyrocketed.Mostorganizationsacrossallindustriesaretryingtousedataandmachinelearningtogainacompetitiveadvantage—infusingintelligenceintotheirproductsandprocessestodelightcustomersandamplifybusinessimpact.Figure1-14showsthevarietyofreal-worldbusinessproblemsbeingsolvedusingmachinelearning.

Figure1-14.Real-worldbusinessproblemsusingmachinelearning

Asaresult,thereishugedemandformachinelearning–relatedjobs.Figure1-15showsthepercentagegrowthinvariousjobpostingsfrom2015to2018.

Page 31: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-15.Growthinmachinelearning–relatedjobs

AndFigure1-16showstheexpectedrevenuefromenterpriseapplicationsusingmachinelearningandartificialintelligencegrowingastronomically.

Page 32: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

Figure1-16.Machinelearning/artificialintelligencerevenueprojections

ConclusionInthischapter,youlearnedsomeofthebestpracticesthatsuccessfulmachinelearningprojectshaveincommon.Wediscussedthattheprocessofbuildingagoodmachinelearningmodelisiterativeandtime-consuming,resultingindatascientistsrequiringanywherefromacoupleofweekstomonthstobuildagoodmodel.Atthesametime,demandformachinelearningisgrowingrapidlyandisexpectedtoskyrocket.

Tobalancethissupply-versus-demandproblem,thereneedstobeabetterwaytoshortenthetimeittakestobuildmachinelearningmodels.Cansomeofthestepsinthatworkflowbeautomated?Absolutely!AutomatedMachineLearningisoneofthemostimportantskillsthatsuccessfuldatascientistsneedtohaveintheirtoolboxforimprovedproductivity.

Inthefollowingchapterswe’llgodeeperintoAutomatedMachineLearning.We

Page 33: Practical Automated Machine · —Parashar. Foreword I vividly remember my first undergraduate class in artificial intelligence (AI). My father had worked for years on “expert systems,”

willexplorewhatitis,howtogetstarted,andhowitisbeingusedinreal-worldapplicationstoday.


Parashar Smriti Parashar Madhav II - Asiatic Society

Parashar Smriti Parashar Madhav II - Asiatic Society

What is financial inclusion by KARTIK PARASHAR

What is financial inclusion by KARTIK PARASHAR

Read On My Own Recommended Book List READ ON MY OWN · 2019. 6. 20. · A se s Midnight on the Moon Read On My Own Recommended Book List by Mary Pope Osborne Readers can vividly imagine

Read On My Own Recommended Book List READ ON MY OWN · 2019. 6. 20. · A se s Midnight on the Moon Read On My Own Recommended Book List by Mary Pope Osborne Readers can vividly imagine

Jan Bednarik Shaifali Parashar Erhan Gundogdu Mathieu ... · Jan Bednarik Shaifali Parashar Erhan Gundogdu Mathieu Salzmann Pascal Fua CVLab, EPFL, Switzerland ffirstname.lastnameg@epfl.ch

Jan Bednarik Shaifali Parashar Erhan Gundogdu Mathieu ... · Jan Bednarik Shaifali Parashar Erhan Gundogdu Mathieu Salzmann Pascal Fua CVLab, EPFL, Switzerland ffirstname.lastnameg@epfl.ch

garima parashar food poisoning ppt =PR assignment 2

garima parashar food poisoning ppt =PR assignment 2

Bruhat Parashar Hora Sashtra CH-02 ...

Bruhat Parashar Hora Sashtra CH-02 ...

Urban B Sectorcdn.cseindia.org/userfiles/Laghu Parashar Bus sector reforms.pdf · Laghu Parashar 24 th May,2013. Urban India • UrbanIndiahavetoprovidefaster growth to aid Indian

Urban B Sectorcdn.cseindia.org/userfiles/Laghu Parashar Bus sector reforms.pdf · Laghu Parashar 24 th May,2013. Urban India • UrbanIndiahavetoprovidefaster growth to aid Indian

JOURNAL OF GEOLOGY...Tahiti. Another ihncident of my seventh year stands out vividly in my recollections. Just north of Manhattan is Blue Mont, a steep and high bluff, the termination

JOURNAL OF GEOLOGY...Tahiti. Another ihncident of my seventh year stands out vividly in my recollections. Just north of Manhattan is Blue Mont, a steep and high bluff, the termination

Company law, 1956 vivek parashar

Company law, 1956 vivek parashar

Parashar Sanhita...Title Parashar Sanhita Author N. A. Subject RELIGION. THEOLOGY Created Date 4/2/2016 1:46:01 AM

Parashar Sanhita...Title Parashar Sanhita Author N. A. Subject RELIGION. THEOLOGY Created Date 4/2/2016 1:46:01 AM

Fundamental analysis of Ashok Leyland by Kartik parashar

Fundamental analysis of Ashok Leyland by Kartik parashar

Curriculum Vitae of Dr. Manish Parashar

Curriculum Vitae of Dr. Manish Parashar

Autonomic Grid Computing: Concepts, Infrastructure …research.cs.rutgers.edu/...umelbourne-11-06.pdf · CSSE Seminar, University of Melbourne 11/24/2006 Manish Parashar

Autonomic Grid Computing: Concepts, Infrastructure …research.cs.rutgers.edu/...umelbourne-11-06.pdf · CSSE Seminar, University of Melbourne 11/24/2006 Manish Parashar

PFR Parashar 99 - environmentclearance.nic.inenvironmentclearance.nic.in/writereaddata/District/... · Microsoft Word - PFR Parashar 99 Author: Hp Created Date: 10/22/2018 10:12:35

PFR Parashar 99 - environmentclearance.nic.inenvironmentclearance.nic.in/writereaddata/District/... · Microsoft Word - PFR Parashar 99 Author: Hp Created Date: 10/22/2018 10:12:35

Make In India by Kartik Parashar

Make In India by Kartik Parashar

Techniques Department-commerc Prof-Parashar Dave Capital ...

Techniques Department-commerc Prof-Parashar Dave Capital ...

Corporate Governance by Kartik Parashar

Corporate Governance by Kartik Parashar

DIGITAL ENERGY Modern Energy Management Systems Parashar... · Modern Energy Management Systems Manu Parashar, PhD IPCGRID, San Francisco, February 27th, 2019 ... Power restoration

DIGITAL ENERGY Modern Energy Management Systems Parashar... · Modern Energy Management Systems Manu Parashar, PhD IPCGRID, San Francisco, February 27th, 2019 ... Power restoration

Parashar Light Help

Parashar Light Help

Police Aur Hamare Aadhikar Reema Parashar

Police Aur Hamare Aadhikar Reema Parashar