Learn C the Hard Way

AboutThiseBook

ePUBisanopen,industry-standardformatforeBooks.However,supportofePUBanditsmanyfeaturesvariesacrossreadingdevicesandapplications.Useyourdeviceorappsettingstocustomizethepresentationtoyourliking.Settingsthatyoucancustomizeoftenincludefont,fontsize,singleordoublecolumn,landscapeorportraitmode,andfiguresthatyoucanclickortaptoenlarge.Foradditionalinformationaboutthesettingsandfeaturesonyourreadingdeviceorapp,visitthedevicemanufacturer ’sWebsite.Manytitlesincludeprogrammingcodeorconfigurationexamples.Tooptimizethepresentationof

theseelements,viewtheeBookinsingle-column,landscapemodeandadjustthefontsizetothesmallestsetting.Inadditiontopresentingcodeandconfigurationsinthereflowabletextformat,wehaveincludedimagesofthecodethatmimicthepresentationfoundintheprintbook;therefore,wherethereflowableformatmaycompromisethepresentationofthecodelisting,youwillseea“Clickheretoviewcodeimage”link.Clickthelinktoviewtheprint-fidelitycodeimage.Toreturntothepreviouspageviewed,clicktheBackbuttononyourdeviceorapp.

LearnCTheHardWayPracticalExercisesontheComputationalSubjectsYouKeep

Avoiding(LikeC)

ZedA.Shaw

NewYork•Boston•Indianapolis•SanFranciscoToronto•Montreal•London•Munich•Paris•MadridCapetown•Sydney•Tokyo•Singapore•MexicoCity

Manyofthedesignationsusedbymanufacturersandsellerstodistinguishtheirproductsareclaimedastrademarks.Wherethosedesignationsappearinthisbook,andthepublisherwasawareofatrademarkclaim,thedesignationshavebeenprintedwithinitialcapitallettersorinallcapitals.

Theauthorandpublisherhavetakencareinthepreparationofthisbook,butmakenoexpressedorimpliedwarrantyofanykindandassumenoresponsibilityforerrorsoromissions.Noliabilityisassumedforincidentalorconsequentialdamagesinconnectionwithorarisingoutoftheuseoftheinformationorprogramscontainedherein.

Forinformationaboutbuyingthistitleinbulkquantities,orforspecialsalesopportunities(whichmayincludeelectronicversions;customcoverdesigns;andcontentparticulartoyourbusiness,traininggoals,marketingfocus,orbrandinginterests),pleasecontactourcorporatesalesdepartmentatcorpsales@pearsoned.comor(800)382-3419.

Forgovernmentsalesinquiries,[email protected].

ForquestionsaboutsalesoutsidetheU.S.,[email protected].

VisitusontheWeb:informit.com/aw

LibraryofCongressCataloging-in-PublicationData

Shaw,Zed,author.LearnCthehardway:practicalexercisesonthecomputationalsubjectsyoukeepavoiding(likeC)/ZedA.Shaw.pagescmIncludesindex.ISBN978-0-321-88492-3(pbk.:alk.paper)—ISBN0-321-88492-2(pbk.:alk.paper)1.C(Computerprogramlanguage)—Problems,exercises,etc.I.Title.QA76.73.C15S4732016005.13’3—dc232015020858

Copyright©2016ZedA.Shaw

Allrightsreserved.PrintedintheUnitedStatesofAmerica.Thispublicationisprotectedbycopyright,andpermissionmustbeobtainedfromthepublisherpriortoanyprohibitedreproduction,storageinaretrievalsystem,ortransmissioninanyformorbyanymeans,electronic,mechanical,photocopying,recording,orlikewise.Toobtainpermissiontousematerialfromthiswork,pleasesubmitawrittenrequesttoPearsonEducation,Inc.,PermissionsDepartment,200OldTappanRoad,OldTappan,NewJersey07657,oryoumayfaxyourrequestto(201)236-3290.

ISBN-13:978-0-321-88492-3ISBN-10:0-321-88492-2

TextprintedintheUnitedStatesonrecycledpaperatRRDonnelleyinCrawfordsville,Indiana.Firstprinting,August2015

mailto:[email protected]



http://informit.com/aw

Contents

Acknowledgments

ThisBookIsNotReallyaboutCTheUndefinedBehavioristsCIsaPrettyandUglyLanguageWhatYouWillLearnHowtoReadThisBookTheVideosTheCoreCompetenciesReadingandWritingAttentiontoDetailSpottingDifferencesPlanningandDebugging

Exercise0TheSetupLinuxMacOSXWindowsTextEditorDoNotUseanIDE

Exercise1DustOffThatCompilerBreakingItDownWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise2UsingMakefilestoBuildUsingMakeWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise3FormattedPrintingWhatYouShouldSeeExternalResearchHowtoBreakItExtraCredit

Exercise4UsingaDebuggerGDBTricksGDBQuickReferenceLLDBQuickReference

Exercise5MemorizingCOperatorsHowtoMemorizeTheListofOperators

Exercise6MemorizingCSyntaxTheKeywordsSyntaxStructuresAWordofEncouragementAWordofWarning

Exercise7VariablesandTypesWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise8If,Else-If,ElseWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise9While-LoopandBooleanExpressionsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise10SwitchStatementsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise11ArraysandStringsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise12SizesandArraysWhatYouShouldSee

HowtoBreakItExtraCredit

Exercise13For-LoopsandArraysofStringsWhatYouShouldSeeUnderstandingArraysofStringsHowtoBreakItExtraCredit

Exercise14WritingandUsingFunctionsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise15Pointers,DreadedPointersWhatYouShouldSeeExplainingPointersPracticalPointerUsageThePointerLexiconPointersAren’tArraysHowtoBreakItExtraCredit

Exercise16StructsandPointerstoThemWhatYouShouldSeeExplainingStructuresHowtoBreakItExtraCredit

Exercise17HeapandStackMemoryAllocationWhatYouShouldSeeHeapversusStackAllocationHowtoBreakItExtraCredit

Exercise18PointerstoFunctionsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise19Zed’sAwesomeDebugMacrosTheCError-HandlingProblem

TheDebugMacrosUsingdbg.hWhatYouShouldSeeHowtheCPPExpandsMacrosExtraCredit

Exercise20AdvancedDebuggingTechniquesDebugPrintingversusGDBADebuggingStrategyExtraCredit

Exercise21AdvancedDataTypesandFlowControlAvailableDataTypesTypeModifiersTypeQualifiersTypeConversionTypeSizes

AvailableOperatorsMathOperatorsDataOperatorsLogicOperatorsBitOperatorsBooleanOperatorsAssignmentOperators

AvailableControlStructuresExtraCredit

Exercise22TheStack,Scope,andGlobalsex22.handex22.cex22_main.c

WhatYouShouldSeeScope,Stack,andBugsHowtoBreakItExtraCredit

Exercise23MeetDuff’sDeviceWhatYouShouldSeeSolvingthePuzzleWhyBother?

ExtraCredit

Exercise24Input,Output,FilesWhatYouShouldSeeHowtoBreakItTheI/OFunctionsExtraCredit

Exercise25VariableArgumentFunctionsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise26ProjectlogfindThelogfindSpecification

Exercise27CreativeandDefensiveProgrammingTheCreativeProgrammerMind-SetTheDefensiveProgrammerMind-SetTheEightDefensiveProgrammerStrategiesApplyingtheEightStrategiesNeverTrustInputPreventErrorsFailEarlyandOpenlyDocumentAssumptionsPreventionoverDocumentationAutomateEverythingSimplifyandClarifyQuestionAuthority

OrderIsNotImportantExtraCredit

Exercise28IntermediateMakefilesTheBasicProjectStructureMakefile

TheHeaderTheTargetBuildTheUnitTestsTheCleanerTheInstallTheChecker

WhatYouShouldSee

ExtraCredit

Exercise29LibrariesandLinkingDynamicallyLoadingaSharedLibraryWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise30AutomatedTestingWiringUptheTestFrameworkExtraCredit

Exercise31CommonUndefinedBehaviorUB20CommonUBs

Exercise32DoubleLinkedListsWhatAreDataStructuresMakingtheLibraryDoublyLinkedListsDefinitionImplementation

TestsWhatYouShouldSeeHowtoImproveItExtraCredit

Exercise33LinkedListAlgorithmsBubbleandMergeSortTheUnitTestTheImplementationWhatYouShouldSeeHowtoImproveItExtraCredit

Exercise34DynamicArrayAdvantagesandDisadvantagesHowtoImproveItExtraCredit

Exercise35SortingandSearchingRadixSortandBinarySearch

CUnionsTheImplementationRadixMap_findandBinarySearchRadixMap_sortandradix_sort

HowtoImproveItExtraCredit

Exercise36SaferStringsWhyCStringsWereaHorribleIdeaUsingbstrlibLearningtheLibrary

Exercise37HashmapsTheUnitTestHowtoImproveItExtraCredit

Exercise38HashmapAlgorithmsWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise39StringAlgorithmsWhatYouShouldSeeAnalyzingtheResultsExtraCredit

Exercise40BinarySearchTreesHowtoImproveItExtraCredit

Exercise41ProjectdevpkgWhatIsdevpkg?WhatWeWanttoMakeTheDesignTheApachePortableRuntime

ProjectLayoutOtherDependencies

TheMakefileTheSourceFilesTheDBFunctions

TheShellFunctionsTheCommandFunctionsThedevpkgMainFunction

TheFinalChallenge

Exercise42StacksandQueuesWhatYouShouldSeeHowtoImproveItExtraCredit

Exercise43ASimpleStatisticsEngineRollingStandardDeviationandMeanImplementionHowtoUseItExtraCredit

Exercise44RingBufferTheUnitTestWhatYouShouldSeeHowtoImproveItExtraCredit

Exercise45ASimpleTCP/IPClientAugmenttheMakefileThenetclientCodeWhatYouShouldSeeHowtoBreakItExtraCredit

Exercise46TernarySearchTreeAdvantagesandDisadvantagesHowtoImproveItExtraCredit

Exercise47AFastURLRouterWhatYouShouldSeeHowtoImproveItExtraCredit

Exercise48ASimpleNetworkServerTheSpecification

Exercise49AStatisticsServerSpecification

Exercise50RoutingtheStatistics

Exercise51StoringtheStatisticsTheSpecification

Exercise52HackingandImprovingYourServer

NextSteps

Index

Acknowledgments

Iwouldliketothankthreekindsofpeoplewhohelpedmakethisbookwhatitistoday:thehaters,thehelpers,andthepainters.Thehatershelpedmakethisbookstrongerandmoresolidthroughtheirinflexibilityofmind,irrationalheroworshipofoldCgods,andcompletelackofpedagogicalexpertise.Withouttheirshiningexampleofwhatnottobe,Iwouldhaveneverworkedsohardtomakethisbookacompleteintroductiontobecomingabetterprogrammer.ThehelpersareDebraWilliamsCauley,VickiRowland,ElizabethRyan,thewholeteamatAddison-Wesley,andeveryoneonlinewhosentinfixesandsuggestions.Theirworkproducing,fixing,editing,andimprovingthisbookhasformeditintoamoreprofessionalandbetterpieceofwriting.Thepainters,Brian,Arthur,Vesta,andSarah,helpedmefindanewwaytoexpressmyselfandtodistractmefromdeadlinesthatDebandVickiclearlysetformebutthatIkeptmissing.Withoutpaintingandthegiftofarttheseartistsgaveme,Iwouldhavealessmeaningfulandrichlife.Thankyoutoallofyouforhelpingmewritethisbook.Itmaynotbeperfect,becausenobookisperfect,butit’satleastasgoodasIcanpossiblymakeit.

ThisBookIsNotReallyaboutC

Pleasedon’tfeelcheated,butthisbookisnotaboutteachingyouCprogramming.You’lllearntowriteprogramsinC,butthemostimportantlessonyou’llgetfromthisbookisrigorousdefensiveprogramming.Today,toomanyprogrammerssimplyassumethatwhattheywriteworks,butonedayitwillfailcatastrophically.Thisisespeciallytrueifyou’rethekindofpersonwhohaslearnedmostlymodernlanguagesthatsolvemanyproblemsforyou.Byreadingthisbookandfollowingmyexercises,you’lllearnhowtocreatesoftwarethatdefendsitselffrommaliciousactivityanddefects.I’musingCforaveryspecificreason:Cisbroken.Itisfullofdesignchoicesthatmadesenseinthe1970sbutmakezerosensenow.Everythingfromitsunrestricted,wilduseofpointerstoitsseverelybrokenNULterminatedstringsaretoblamefornearlyallofthesecuritydefectsthathitC.It’smybeliefthatCissobrokenthat,whileit’sinwideuse,it’sthemostdifficultlanguagetowritesecurely.IwouldfathomthatAssemblyisactuallyeasiertowritesecurelythanC.Tobehonest,andyou’llfindoutthatI’mveryhonest,Idon’tthinkthatanybodyshouldbewritingnewCcode.Ifthat’sthecase,thenwhyamIteachingyouC?BecauseIwantyoutobecomeabetter,strongerprogrammer,andtherearetworeasonswhyCisanexcellentlanguagetolearnifyouwanttogetbetter.First,C’slackofnearlyeverymodernsafetyfeaturemeansyouhavetobemorevigilantandmoreawareofwhat’sgoingon.Ifyoucanwritesecure,solidCcode,youcanwritesecure,solidcodeinanyprogramminglanguage.Thetechniquesyoulearnwilltranslatetoeverylanguageyouusefromnowon.Second,learningCgivesyoudirectaccesstoamountainoflegacycode,andteachesyouthebasesyntaxofalargenumberofdescendantlanguages.OnceyoulearnC,youcanmoreeasilylearnC++,Java,Objective-C,andJavaScript,andevenotherlanguagesbecomeeasiertolearn.Idon’twanttoscareyouawaybytellingyouthis,becauseIplantomakethisbookincrediblyfun,easy,anddevious.I’llmakeitfuntolearnCbygivingyouprojectsthatyoumightnothavedoneinotherprogramminglanguages.I’llmakethisbookeasybyusingmyprovenpatternofexercisesthathasyoudoingCprogrammingandbuildingyourskillsslowly.I’llmakeitdeviousbyteachingyouhowtobreakandthensecureyourcodesoyouunderstandwhytheseissuesmatter.You’lllearnhowtocausestackoverflows,illegalmemoryaccess,andothercommonflawsthatplagueCprogramssothatyouknowwhatyou’reupagainst.Gettingthroughthisbookwillbechallenging,likeallofmybooks,butwhenyou’redoneyouwillbeafarbetterandmoreconfidentprogrammer.

TheUndefinedBehavioristsBythetimeyou’redonewiththisbook,you’llbeabletodebug,read,andfixalmostanyCprogramyouruninto,andthenwritenew,solidCcodeshouldyouneedto.However,I’mnotreallygoingtoteachyouofficialC.You’lllearnthelanguage,andyou’lllearnhowtouseitwell,butofficialCisn’tverysecure.ThevastmajorityofCprogrammersouttheresimplydon’twritesolidcode,andit’sbecauseofsomethingcalledUndefinedBehavior(UB).UBisapartoftheAmericanNationalStandardsInstitute(ANSI)CstandardthatlistsallofthewaysthataCcompilercandisregardwhatyou’vewritten.There’sactuallyapartofthestandardthatsaysifyouwritecodelikethis,thenallbetsareoffandthecompilerdoesn’thavetodoanythingconsistently.UBoccurswhenaCprogramreadsofftheendofastring,whichisanincrediblycommonprogrammingerrorinC.Forabitofbackground,CdefinesstringsasblocksofmemorythatendinaNULbyte,ora0byte(tosimplifythedefinition).Sincemanystringscomefromoutsidetheprogram,it’scommonforaCprogramtoreceiveastringwithoutthisNULbyte.Whenitdoes,theCprogramattemptstoreadpasttheendofthisstringandintothememoryofthecomputer,causingyourprogramtocrash.EveryotherlanguagedevelopedafterCattemptstopreventthis,butnotC.CdoessolittletopreventUBthateveryCprogrammerseemstothinkitmeanstheydon’thavetodealwithit.TheywritecodefullofpotentialNULbyteoverruns,andwhenyoupointthemouttotheseprogrammers,theysay,“Wellthat’sUB,andIdon’thavetopreventit.”ThisrelianceonC’slargenumberofUBsiswhymostCcodeissohorriblyinsecure.IwriteCcodetotrytoavoidUBbyeitherwritingcodethatdoesn’ttriggerit,orwritingcodethatattemptstopreventit.ThisturnsouttobeanimpossibletaskbecausethereissomuchUBthatitbecomesaGordianknotofinterconnectedpitfallsinyourCcode.Asyougothroughthisbook,I’llpointoutwaysyoucantriggerUB,howtoavoiditifyoucan,andhowtotriggeritinotherpeople’scodeifpossible.However,youshouldkeepinmindthatavoidingthenearlyrandomnatureofUBisalmostimpossible,andyou’lljusthavetodoyourbest.

Warning!You’llfindthathardcoreCfansfrequentlywilltrytobeatyouupaboutUB.There’saclassofCprogrammerswhodon’twriteverymuchCcodebuthavememorizedalloftheUBjustsotheycouldbeatupabeginnerintellectually.Ifyourunintooneoftheseabusiveprogrammers,pleaseignorethem.Often,theyaren’tpracticingCprogrammers,theyarearrogant,abusive,andwillonlyendupaskingyouendlessquestionsinanattempttoprovetheirsuperiorityratherthanhelpingyouwithyourcode.ShouldyoueverneedhelpwithyourCcode,[email protected],andIwillgladlyhelpyou.

CIsaPrettyandUglyLanguageThepresenceofUBthoughisonemorereasonwhylearningCisagoodmoveifyouwanttobeabetterprogrammer.Ifyoucanwritegood,solidCcodeinthewayIteachyou,thenyoucansurviveanylanguage.Onthepositiveside,Cisareallyelegantlanguageinmanyways.Itssyntaxisactuallyincrediblysmallgiventhepowerithas.There’sareasonwhysomanyotherlanguageshavecopieditssyntaxoverthelast45orsoyears.Calsogivesyouquitealotusingverylittletechnology.Whenyou’redonelearningC,you’llhaveanappreciationforasomethingthatisveryelegantandbeautifulbutalsoalittleuglyatthesametime.Cisold,solikeabeautifulmonument,itwilllookfantasticfromabout20feetaway,butwhenyoustepupclose,you’llseeallthecracksandflawsithas.Becauseofthis,I’mgoingtoteachyouthemostrecentversionofCthatIcanmakeworkwithrecent


compilers.It’sapractical,straightforward,simple,yetcompletesubsetofCthatworkswell,workseverywhere,andavoidsmanypitfalls.ThisistheCthatIusetogetrealworkdone,andnottheencyclopedicversionofCthathardcorefanstryandfailtouse.IknowtheCthatIuseissolidbecauseIspenttwodecadeswritingclean,solidCcodethatpoweredlargeoperationswithoutmuchfailureatall.MyCcodehasprobablyprocessedtrillionsoftransactionsbecauseitpoweredtheoperationsofcompanieslikeTwitterandairbnb.Itrarelyfailedorhadsecurityattacksagainstit.InthemanyyearsthatmycodepoweredtheRubyonRailsWebworld,it’srunbeautifullyandevenpreventedsecurityattacks,whileotherWebserversfellrepeatedlytothesimplestofattacks.MystyleofwritingCcodeissolid,butmoreimportantly,mymind-setwhenwritingCisoneeveryprogrammershouldhave.IapproachC,andanyprogramming,withtheideaofpreventingerrorsasbestIcanandassumingthatnothingwillworkright.Otherprogrammers,evensupposedlygoodCprogrammers,tendtowritecodeandassumeeverythingwillwork,butrelyonUBortheoperatingsystemtosavethem,neitherofwhichwillworkasasolution.JustrememberthatifpeopletrytotellyouthatthecodeIteachinthisbookisn’t“realC.”Iftheydon’thavethesametrackrecordasme,maybeyoucanusewhatIteachyoutoshowthemwhytheircodeisn’tverysecure.Doesthatmeanmycodeisperfect?No,notatall.ThisisCcode.WritingperfectCcodeisimpossible,andinfact,writingperfectcodeinanylanguageisimpossible.That’shalfthefunandfrustrationofprogramming.Icouldtakesomeoneelse’scodeandtearitapart,andsomeonecouldtakemycodeandtearitapart.Allcodeisflawed,butthedifferenceisthatItrytoassumemycodeisalwaysflawedandthenpreventtheflaws.Mygifttoyou,shouldyoucompletethisbook,istoteachyouthedefensiveprogrammingmind-setthathasservedmewellfortwodecades,andhashelpedmemakehigh-quality,robustsoftware.

WhatYouWillLearnThepurposeofthisbookistogetyoustrongenoughinCthatyou’llbeabletowriteyourownsoftwarewithitormodifysomeoneelse’sCcode.Afterthisbook,youshouldreadBrianKernighanandDennisRitchie’sTheCProgrammingLanguage,SecondEdition(PrenticeHall,1988),abookbythecreatorsoftheClanguage,alsocalledK&RC.WhatI’llteachyouis

•ThebasicsofCsyntaxandidioms•Compilation,makefiles,linkers•Findingbugsandpreventingthem•Defensivecodingpractices•BreakingCcode•WritingbasicUNIXsystemssoftware

Bythefinalexercise,youwillhavemorethanenoughammunitiontotacklebasicsystemssoftware,libraries,andothersmallerprojects.

HowtoReadThisBookThisbookisintendedforprogrammerswhohavelearnedatleastoneotherprogramminglanguage.IreferyoutomybookLearnPythontheHardWay(Addison-Wesley,2013)ifyouhaven’tlearnedaprogramminglanguageyet.It’smeantforbeginnersandworksverywellasafirstbookonprogramming.Onceyou’vecompletedLearnPythontheHardWay,thenyoucancomebackandstartthisbook.

Forthosewho’vealreadylearnedtocode,thisbookmayseemstrangeatfirst.It’snotlikeotherbookswhereyoureadparagraphafterparagraphofproseandthentypeinabitofcodehereandthere.Instead,therearevideosoflecturesforeachexercise,youcoderightaway,andthenIexplainwhatyoujustdid.Thisworksbetterbecauseit’seasierformetoexplainsomethingyou’vealreadydonethantospeakinanabstractsenseaboutsomethingyouaren’tfamiliarwithatall.Becauseofthisstructure,thereareafewrulesthatyoumustfollowinthisbook:

•Watchthelecturevideofirst,unlesstheexercisesaysotherwise.•Typeinallofthecode.Don’tcopy-paste!•Typeinthecodeexactlyasitappears,eventhecomments.•Getittorunandmakesureitprintsthesameoutput.•Iftherearebugs,fixthem.•DotheExtraCredit,butit’sallrighttoskipanythingyoucan’tfigureout.•Alwaystrytofigureitoutfirstbeforetryingtogethelp.

Ifyoufollowtheserules,doeverythinginthebook,andstillcan’tcodeC,thenyouatleasttried.It’snotforeveryone,butjusttryingwillmakeyouabetterprogrammer.

TheVideosIncludedinthiscoursearevideosforeveryexercise,andinmanycases,morethanonevideoforanexercise.Thesevideosshouldbeconsideredessentialtogetthefullimpactofthebook’seducationalmethod.ThereasonforthisisthatmanyoftheproblemswithwritingCcodeareinteractiveissueswithfailure,debugging,andcommands.Crequiresmuchmoreinteractiontogetthecoderunningandtofixproblems,unlikelanguageslikePythonandRubywherecodejustruns.It’salsomucheasiertoshowyouavideolectureonatopic,suchaspointersormemorymanagement,whereIcandemonstratehowthemachineisactuallyworking.Irecommendthatasyougothroughthecourse,youplantowatchthevideosfirst,andthendotheexercisesunlessdirectedtodootherwise.Insomeoftheexercises,Iuseonevideotopresentaproblemandthenanothertodemonstratethesolution.Inmostoftheotherexercises,Iuseavideotopresentalecture,andthenyoudotheexerciseandcompleteittolearnthetopic.

TheCoreCompetenciesI’mgoingtoguessthatyouhaveexperienceusingalesserlanguage.Oneofthoseusablelanguagesthatletsyougetawaywithsloppythinkingandhalf-bakedhackerylikePythonorRuby.Or,maybeyouusealanguagelikeLISPthatpretendsthecomputerissomepurelyfunctionalfantasylandwithpaddedwallsforlittlebabies.Maybeyou’velearnedProlog,andyouthinktheentireworldshouldjustbeadatabasewhereyouwalkaroundinitlookingforclues.Evenworse,I’mbettingyou’vebeenusinganintegrateddevelopmentenvironment(IDE),soyourbrainisriddledwithmemoryholes,andyoucan’teventypeanentirefunction’snamewithouthittingCTRL-SPACEaftereverythreecharacters.Nomatterwhatyourbackgroundis,youcouldprobablyusesomeimprovementintheseareas:

ReadingandWritingThisisespeciallytrueifyouuseanIDE,butgenerallyIfindprogrammersdotoomuchskimmingandhaveproblemsreadingforcomprehension.They’lljustskimcodethattheyneedtounderstandindetailwithouttakingthetimetounderstandit.Otherlanguagesprovidetoolsthatletprogrammersavoidactuallywritinganycode,sowhenfacedwithalanguagelikeC,theybreakdown.Thesimplestthingtodoisjustunderstandthateveryonehasthisproblem,andyoucanfixitbyforcingyourselftoslowdownandbemeticulousaboutyourreadingandwriting.Atfirst,it’llfeelpainfulandannoying,buttakefrequentbreaks,andtheneventuallyit’llbeeasiertodo.

AttentiontoDetailEveryoneisbadatthis,andit’sthebiggestcauseofbadsoftware.Otherlanguagesletyougetawaywithnotpayingattention,butCdemandsyourfullattentionbecauseit’srightinthemachine,andthemachineisverypicky.WithC,thereisno“kindofsimilar”or“closeenough,”soyouneedtopayattention.Doublecheckyourwork.Assumeeverythingyouwriteiswronguntilyouproveit’sright.

SpottingDifferencesAkeyproblemthatpeoplewhoareusedtootherlanguageshaveisthattheirbrainshavebeentrainedtospotdifferencesinthatlanguage,notinC.Whenyoucomparecodeyou’vewrittentomyexercisecode,youreyeswilljumprightovercharactersyouthinkdon’tmatterorthataren’tfamiliar.I’llbegivingyoustrategiesthatforceyoutoseeyourmistakes,butkeepinmindthatifyourcodeisnotexactlylikethecodeinthisbook,it’swrong.

PlanningandDebuggingIloveother,easierlanguagesbecauseIcanjusthangout.IcantypetheideasIhaveintotheirinterpreterandseeresultsimmediately.They’regreatforjusthackingoutideas,buthaveyounoticedthatifyoukeepdoinghackuntilitworks,eventuallynothingworks?Cisharderonyoubecauseitrequiresyoutofirstplanoutwhatyouwanttocreate.Sure,youcanhackforabit,butyouhavetogetseriousmuchearlierinCthaninotherlanguages.I’llbeteachingyouwaystoplanoutkeypartsofyourprogrambeforeyoustartcoding,andthiswilllikelymakeyouabetterprogrammeratthesametime.Evenjustalittleplanningcansmooththingsoutdowntheroad.LearningCmakesyouabetterprogrammerbecauseyouareforcedtodealwiththeseissuesearlierandmorefrequently.Youcan’tbesloppyaboutwhatyouwriteornothingwillwork.TheadvantageofCisthatit’sasimplelanguagethatyoucanfigureoutonyourown,whichmakesitagreatlanguageforlearningaboutthemachineandgettingstrongerinthesecoreprogrammingskills.

Exercise0.TheSetup

Thetraditionalfirstexercise,Excercise0,iswhereyousetupyourcomputerfortherestofthisbook.Inthisexerciseyou’llinstallpackagesandsoftwaredependingonthetypeofcomputeryouhave.Ifyouhaveproblemsfollowingthisexercise,thensimplywatchtheExercise0videoforyourcomputerandfollowalongwithmysetupinstructions.Thatvideoshoulddemonstratehowtodoeachstepandhelpyousolveanyproblemsthatmightcomeup.

LinuxLinuxismostlikelytheeasiestsystemtoconfigureforCdevelopment.ForDebiansystemsyourunthiscommandfromthecommandline:Clickheretoviewcodeimage

$sudoapt−getinstallbuild−essential

Here’showyouwouldinstallthesamesetuponanRPM-basedLinuxlikeFedora,RedHat,orCentOS7:Clickheretoviewcodeimage

$sudoyumgroupinstalldevelopment−tools

IfyouhaveadifferentvariantofLinux,simplysearchfor“cdevelopmenttools”andyourbrandofLinuxtofindoutwhat’srequired.Onceyouhavethatinstalled,youshouldbeabletotype:

$cc--version

toseewhatcompilerwasinstalled.YouwillmostlikelyhavetheGNUCCompiler(GCC)installedbutdon’tworryifit’sadifferentonefromwhatIuseinthebook.YoucouldalsotryinstallingtheClangCcompilerusingtheClang’sGettingStartedinstructionsforyourversionofLinux,orsearchingonlineifthosedon’twork.

MacOSXOnMacOSX,theinstalliseveneasier.First,you’llneedtoeitherdownloadthelatestXCodefromApple,orfindyourinstallDVDandinstallitfromthere.Thedownloadwillbemassiveandcouldtakeforever,soIrecommendinstallingfromtheDVD.Also,searchonlinefor“installingxcode”forinstructionsonhowtodoit.YoucanalsousetheAppStoretoinstallitjustasyouwouldanyotherapp,andifyoudoitthatwayyou’llreceiveupdatesautomatically.ToconfirmthatyourCcompilerisworking,typethis:

$cc−−version

YoushouldseethatyouareusingaversionoftheClangCCompiler,butifyourXCodeisolderyoumayhaveGCCinstalled.Eitherisfine.

WindowsForMicrosoftWindows,IrecommendyouusetheCygwinsystemtoacquiremanyofthestandardUNIXsoftwaredevelopmenttools.Itshouldbeeasytoinstallanduse,butwatchthevideosforthisexercisetoseehowIdoit.AnalternativetoCygwinistheMinGWsystem;itismoreminimalistbutshouldalsowork.IwillwarnyouthatMicrosoftseemstobephasingoutCsupportintheirdevelopmenttools,soyoumayhaveproblemsusingMicrosoft’scompilerstobuildthecodeinthisbook.AslightlymoreadvancedoptionistouseVirtualBoxtoinstallaLinuxdistributionandrunacompleteLinuxsystemonyourWindowscomputer.ThishastheaddedadvantagethatyoucancompletelydestroythisvirtualmachinewithoutworryingaboutdestroyingyourWindowsconfiguration.It’salsoanopportunitytolearntouseLinux,whichisbothfunandbeneficialtoyourdevelopmentasaprogrammer.Linuxiscurrentlydeployedasthemainoperatingsystemformanydistributedcomputerandcloudinfrastructurecompanies.LearningLinuxwilldefinitelyimproveyourknowledgeofthefutureofcomputing.

TextEditorThechoiceoftexteditorforaprogrammerisatoughone.Forbeginners,IsayjustuseGeditsinceit’ssimpleanditworksforcode.However,itdoesn’tworkincertaininternationalsituations,andifyou’vebeenprogrammingforawhile,chancesareyoualreadyhaveafavoritetexteditor.Withthisinmind,Iwantyoutotryoutafewofthestandardprogrammertexteditorsforyourplatformandthenstickwiththeonethatyoulikebest.Ifyou’vebeenusingGEditandlikeit,thenstickwithit.Ifyouwanttotrysomethingdifferent,thentryitoutrealquickandpickone.Themostimportantthingisdonotgetstucktryingtopicktheperfecteditor.Texteditorsalljustkindofsuckinoddways.Justpickone,stickwithit,andifyoufindsomethingelseyoulike,tryitout.Don’tspenddaysonendconfiguringitandmakingitperfect.Sometexteditorstotryout:

•GEditonLinuxandOSX.•TextWrangleronOSX.•Nano,whichrunsinTerminalandworksnearlyeverywhere.•EmacsandEmacsforOSX;bepreparedtodosomelearning,though.•VimandMacVim.

Thereisprobablyadifferenteditorforeverypersonoutthere,butthesearejustafewofthefreeonesthatIknowwork.Tryoutafewofthese—andmaybesomecommercialones—untilyoufindonethatyoulike.

DoNotUseanIDE

Warning!Avoidusinganintegrateddevelopmentenvironment(IDE)whileyouarelearningalanguage.Theyarehelpfulwhenyouneedtogetthingsdone,buttheirhelptendsalsotopreventyoufromreallylearningthelanguage.Inmyexperience,thestrongerprogrammersdon’tuseanIDEandalsohavenoproblemproducingcodeatthesamespeedasIDEusers.IalsofindthatthecodeproducedwithanIDEisoflowerquality.Ihavenoideawhythatisthecase,butifyouwantdeep,solidskillsinaprogramminglanguage,IhighlyrecommendthatyouavoidIDEswhileyou’relearning.Knowinghowtouseaprofessionalprogrammer’stexteditorisalsoausefulskillinyour

professionallife.Whenyou’redependentonanIDE,youhavetowaitforanewIDEbeforeyoucanlearnthenewerprogramminglanguages.Thisaddsacosttoyourcareer:Itpreventsyoufromgettingaheadofshiftsinlanguagepopularity.Withagenerictexteditor,youcancodeinanylanguage,anytimeyoulike,withoutwaitingforanyonetoaddittoanIDE.Agenerictexteditormeansfreedomtoexploreonyourownandmanageyourcareerasyouseefit.

Exercise1.DustOffThatCompiler

Afteryouhaveeverythinginstalled,youneedtoconfirmthatyourcompilerworks.TheeasiestwaytodothatistowriteaCprogram.Sinceyoushouldalreadyknowatleastoneprogramminglanguage,Ibelieveyoucanstartwithasmallbutextensiveexample.

ex1.c

Clickheretoviewcodeimage

1#include<stdio.h>23/*Thisisacomment.*/4intmain(intargc,char*argv[])5{6intdistance=100;78//thisisalsoacomment9printf("Youare%dmilesaway.\n",distance);1011return0;12}

Ifyouhaveproblemsgettingthecodeupandrunning,watchthevideoforthisexercisetoseemedoitfirst.

BreakingItDownThereareafewfeaturesoftheClanguageinthiscodethatyoumightormightnothavefiguredoutwhileyouweretypingit.I’llbreakthisdown,linebyline,quickly,andthenwecandoexercisestounderstandeachpartbetter.Don’tworryifyoudon’tunderstandeverythinginthisbreakdown.IamsimplygivingyouaquickdiveintoCandpromiseyouwilllearnalloftheseconceptslaterinthebook.Here’saline-by-linedescriptionofthecode:

ex1.c:1Aninclude,anditisthewaytoimportthecontentsofonefileintothissourcefile.Chasaconventionofusing.hextensionsforheaderfiles,whichcontainlistsoffunctionstouseinyourprogram.

ex1.c:3Thisisamultilinecomment,andyoucouldputasmanylinesoftextbetweentheopening/*andclosing*/charactersasyouwant.

ex1.c:4Amorecomplexversionofthemainfunctionyou’vebeenusingsofar.HowCprogramsworkisthattheoperatingsystemloadsyourprogram,andthenitrunsthefunctionnamedmain.Forthefunctiontobetotallycompleteitneedstoreturnanintandtaketwoparameters:anintfortheargumentcountandanarrayofchar*stringsforthearguments.Didthatjustflyoveryourhead?Don’tworry,we’llcoverthissoon.

ex1.c:5Tostartthebodyofanyfunction,youwritea{characterthatindicatesthebeginningofablock.InPython,youjustdida:andindented.Inotherlanguages,youmighthaveabeginordowordtostart.

ex1.c:6Avariabledeclarationandassignmentatthesametime.Thisishowyoucreateavariable,

withthesyntaxtypename=value;.InC,statements(exceptforlogic)endina;(semicolon)character.

ex1.c:8Anotherkindofcomment.ItworkslikeinPythonorRuby,wherethecommentstartsatthe//andgoesuntiltheendoftheline.

ex1.c:9Acalltoyouroldfriendprintf.Likeinmanylanguages,functioncallsworkwiththesyntaxname(arg1,arg2);andcanhavenoargumentsoranynumberofthem.Theprintffunctionisactuallykindofweirdinthatitcantakemultiplearguments.You’llseethatlater.

ex1.c:11Areturnfromthemainfunctionthatgivestheoperatingsystem(OS)yourexitvalue.YoumaynotbefamiliarwithhowUNIXsoftwareusesreturncodes,sowe’llcoverthataswell.

ex1.c:12Finally,weendthemainfunctionwithaclosingbrace}character,andthat’stheendoftheprogram.

There’salotofinformationinthisbreakdown,sostudyitlinebylineandmakesureyouatleasthaveagraspofwhat’sgoingon.Youwon’tknoweverything,butyoucanprobablyguessbeforewecontinue.

WhatYouShouldSeeYoucanputthisintoanex1.candthenrunthecommandsshownhereinthissampleshelloutput.Ifyou’renotsurehowthisworks,watchthevideothatgoeswiththisexercisetoseemedoit.

Exercise1Session


$makeex1cc-Wall-gex1.c-oex1

$./ex1Youare100milesaway.

$

ThefirstcommandmakeisatoolthatknowshowtobuildCprograms(andmanyothers).Whenyourunitandgiveitex1youaretellingmaketolookfortheex1.cfile,runthecompilertobuildit,andleavetheresultsinafilenamedex1.Thisex1fileisanexecutablethatyoucanrunwith./ex1,whichoutputsyourresults.

HowtoBreakItInthisbook,I’mgoingtohaveasmallsectionforeachprogramteachingyouhowtobreaktheprogramifit’spossible.I’llhaveyoudooddthingstotheprograms,runtheminweirdways,orchangecodesothatyoucanseecrashesandcompilererrors.Forthisprogram,simplytryremovingthingsatrandomandstillgetittocompile.Justmakeaguessatwhatyoucanremove,recompileit,andthenseewhatyougetforanerror.

ExtraCredit•Opentheex1fileinyourtexteditorandchangeordeleterandomparts.Tryrunningitandseewhathappens.

•Printoutfivemorelinesoftextorsomethingmorecomplexthan“helloworld.”•Runman3printfandreadaboutthisfunctionandmanyothers.•Foreachline,writeoutthesymbolsyoudon’tunderstandandseeifyoucanguesswhattheymean.Writealittlechartonpaperwithyourguesssoyoucancheckitlatertoseeifyougotitright.

Exercise2.UsingMakefilestoBuild

We’regoingtouseaprogramcalledmaketosimplifybuildingyourexercisecode.Themakeprogramhasbeenaroundforaverylongtime,andbecauseofthisitknowshowtobuildquiteafewtypesofsoftware.Inthisexercise,I’llteachyoujustenoughMakefilesyntaxtocontinuewiththecourse,andthenanexerciselaterwillteachyoumorecompleteMakefileusage.

UsingMakeHowmakeworksisyoudeclaredependencies,andthendescribehowtobuildthemorrelyontheprogram’sinternalknowledgeofhowtobuildmostcommonsoftware.Ithasdecadesofknowledgeaboutbuildingawidevarietyoffilesfromotherfiles.Inthelastexercise,youdidthisalreadyusingcommands:

$makeex1#orthisonetoo$CFLAGS="-Wall"makeex1

Inthefirstcommand,you’retellingmake,“Iwantafilenamedex1tobecreated.”Theprogramthenasksanddoesthefollowing:

1.Doesthefileex1existalready?2.No.Okay,isthereanotherfilethatstartswithex1?3.Yes,it’scalledex1.c.DoIknowhowtobuild.cfiles?4.Yes,Irunthiscommandccex1.c-oex1tobuildthem.5.Ishallmakeyouoneex1byusingcctobuilditfromex1.c.

Thesecondcommandinthelistingaboveisawaytopassmodifierstothemakecommand.Ifyou’renotfamiliarwithhowtheUNIXshellworks,youcancreatetheseenvironmentvariablesthatwillgetpickedupbyprogramsyourun.SometimesyoudothiswithacommandlikeexportCFLAGS="-Wall"dependingontheshellyouuse.Youcan,however,alsojustputthembeforethecommandyouwanttorun,andthatenvironmentvariablewillbesetonlywhilethatcommandruns.Inthisexample,IdidCFLAGS="-Wall"makeex1sothatitwouldaddthecommandlineoption-Walltothecccommandthatmakenormallyruns.Thatcommandlineoptiontellsthecompilercctoreportallwarnings(which,inasicktwistoffate,isn’tactuallyallthewarningspossible).Youcanactuallygetprettyfarwithjustusingmakeinthatway,butlet’sgetintomakingaMakefilesoyoucanunderstandmakealittlebetter.Tostartoff,createafilewithjustthefollowinginit.

ex2.1.mak

CFLAGS=-Wall-g

clean:rm-fex1

SavethisfileasMakefileinyourcurrentdirectory.Theprogramautomaticallyassumesthere’sa

filecalledMakefileandwilljustrunit.

Warning!MakesureyouareonlyenteringTABcharacters,notmixturesofTABandspaces.

ThisMakefileisshowingyousomenewstuffwithmake.First,wesetCFLAGSinthefilesoweneverhavetosetitagain,aswellasaddingthe-gflagtogetdebugging.Then,wehaveasectionnamedcleanthattellsmakehowtocleanupourlittleproject.Makesureit’sinthesamedirectoryasyourex1.cfile,andthenrunthesecommands:

$makeclean$makeex1

WhatYouShouldSeeIfthatworked,thenyoushouldseethis:

Exercise2Session


$makecleanrm-fex1


ex1.c:Infunction'main':

ex1.c:3:warning:implicitdeclarationoffunction'puts'

$

HereyoucanseethatI’mrunningmakeclean,whichtellsmaketorunourcleantarget.GolookattheMakefileagainandyou’llseethatunderthiscommand,IindentandthenputintheshellcommandsIwantmaketorunforme.Youcouldputasmanycommandsasyouwantedinthere,soit’sagreatautomationtool.

Warning!Ifyoufixedex1.ctohave#include<stdio.h>,thenyouroutputwon’thavethewarning(whichshouldreallybeanerror)aboutputs.IhavetheerrorherebecauseIdidn’tfixit.

Noticethateventhoughwedon’tmentionex1intheMakefile,makestillknowshowtobuilditanduseourspecialsettings.

HowtoBreakItThatshouldbeenoughtogetyoustarted,butfirstlet’sbreakthisMakefileinaparticularwaysoyoucanseewhathappens.Takethelinerm-fex1andremovetheindent(moveitallthewayleft)soyoucanseewhathappens.Rerunmakeclean,andyoushouldgetsomethinglikethis:Clickheretoviewcodeimage

$makeclean

Makefile:4:***missingseparator.Stop.

Alwaysremembertoindent,andifyougetweirderrorslikethis,doublecheckthatyou’reconsistentlyusingtabcharactersbecausesomemakevariantsareverypicky.

ExtraCredit•Createanall:ex1targetthatwillbuildex1withjustthecommandmake.•Readmanmaketofindoutmoreinformationonhowtorunit.•Readmancctofindoutmoreinformationonwhattheflags-Walland-gdo.•ResearchMakefilesonlineandseeifyoucanimprovethisone.•FindaMakefileinanotherCprojectandtrytounderstandwhatit’sdoing.

Exercise3.FormattedPrinting

KeepthatMakefilearoundsinceit’llhelpyouspoterrors,andwe’llbeaddingtoitwhenweneedtoautomatemorethings.ManyprogramminglanguagesusetheCwayofformattingoutput,solet’stryit:

ex3.c


1#include<stdio.h>23intmain()4{5intage=10;6intheight=72;78printf("Iam%dyearsold.\n",age);9printf("Iam%dinchestall.\n",height);1011return0;12}

Onceyou’vefinishedthat,dotheusualmakeex3tobuildandrunit.Makesureyoufixallwarnings.Thisexercisehasawholelotgoingoninasmallamountofcode,solet’sbreakitdown:

•Firstwe’reincludinganotherheaderfilecalledstdio.h.Thistellsthecompilerthatyou’regoingtousethestandardInput/Outputfunctions.Oneofthoseisprintf.•Thenwe’reusingavariablenamedageandsettingitto10.•Nextwe’reusingavariableheightandsettingitto72.•Thenwe’readdingtheprintffunctiontoprinttheageandheightofthetallest10-year-oldontheplanet.•Inprintf,you’llnoticewe’reincludingaformatstring,asseeninmanyotherlanguages.•Afterthisformatstring,we’reputtinginthevariablesthatshouldbe“replaced”intotheformatstringbyprintf.

Theresultisgivingprintfsomevariablesandit’sconstructinganewstringandthenprintingittotheterminal.

WhatYouShouldSeeWhenyoudothewholebuild,youshouldseesomethinglikethis:

Exercise3Session



$./ex3

Iam10yearsold.

Iam72inchestall.

$

PrettysoonI’mgoingtostoptellingyoutorunmakeandwhatthebuildlookslike,sopleasemakesureyou’regettingthisrightandthatit’sworking.

ExternalResearchIntheExtraCreditsectionofeachexercise,youmayhaveyougofindinformationonyourownandfigurethingsout.Thisisanimportantpartofbeingaself-sufficientprogrammer.Ifyou’reconstantlyrunningtoasksomeoneaquestionbeforetryingtofigurethingsoutyourself,thenyou’llneverlearnhowtosolveproblemsindependently.You’llneverbuildconfidenceinyourskillsandwillalwaysneedsomeoneelsearoundtodoyourwork.Thewaytobreakthishabitistoforceyourselftotrytoansweryourownquestionfirst,andthenconfirmthatyouranswerisright.Youdothisbytryingtobreakthings,experimentingwithyouranswer,anddoingyourownresearch.Forthisexercise,Iwantyoutogoonlineandfindoutalloftheprintfescapecodesandformatsequences.Escapecodesare\nor\tthatletyouprintanewlineortab,respectively.Formatsequencesarethe%sor%dthatletyouprintastringorinteger.Findthemall,learnhowtomodifythem,andseewhatkindof“precisions”andwidthsyoucando.Fromnowon,thesekindsoftaskswillbeintheExtraCreditsections,andyoushoulddothem.

HowtoBreakItTryafewofthesewaystobreakthisprogram,whichmayormaynotcauseittocrashonyourcomputer:

•Taketheagevariableoutofthefirstprintfcall,thenrecompile.Youshouldgetacoupleofwarnings.•Runthisnewprogramanditwilleithercrashorprintoutareallycrazyage.•Puttheprintfbackthewayitwas,andthendon’tsetagetoaninitialvaluebychangingthatlinetointage;,andthenrebuilditandrunitagain.

Exercise3.badSession


#editex3.ctobreakprintf$makeex3cc-Wall-gex3.c-oex3


ex3.c:8:warning:toofewargumentsforformat

ex3.c:5:warning:unusedvariable'age'

$./ex3Iam-919092456yearsold.

Iam72inchestall.

#editex3.cagaintofixprintf,butdon'tinitage$makeex3cc-Wall-gex3.c-oex3


ex3.c:8:warning:'age'isuseduninitializedinthisfunction

$./ex3

Iam0yearsold.

Iam72inchestall.

$

ExtraCredit•Findasmanyotherwaystobreakex3.casyoucan.•Runman3printfandreadabouttheother%formatcharactersyoucanuse.Theseshouldlookfamiliarifyouusedtheminotherlanguages(theycomefromprintf).•Addex3tothealllistinyourMakefile.Usethistomakecleanallandbuildallofyourexercisesthusfar.•Addex3tothecleanlistinyourMakefileaswell.Usemakecleantoremoveitwhenyouneedto.

Exercise4.UsingaDebugger

Thisisavideo-focusedexercisewhereIshowyouhowtousethedebuggerthatcomeswithyourcomputertodebugyourprograms,detecterrors,andevendebugprocessesthatarecurrentlyrunning.Pleasewatchtheaccompanyingvideotolearnmoreaboutthistopic.

GDBTricksHere’salistofsimpletricksyoucandowithGNUDebugger(GDB):

gdb--argsNormally,gdbtakesargumentsyougiveitandassumestheyareforitself.Using--argspassesthemtotheprogram.

threadapplyallbtDumpabacktraceforallthreads.It’sveryuseful.gdb--batch--exr--exbt--exq--argsRuntheprogramsothatifitbombs,yougetabacktrace.

GDBQuickReferenceThevideoisgoodforlearninghowtouseadebugger,butyou’llneedtoreferbacktothecommandsasyouwork.HereisaquickreferencetotheGDBcommandsthatIusedinthevideosoyoucanusethemlaterinthebook:

run[args]Startyourprogramwith[args].break[file:]functionSetabreakpointat[file:]function.Youcanalsouseb.backtraceDumpabacktraceofthecurrentcallingstack.Shorthandisbt.printexprPrintthevalueofexpr.Shorthandisp.continueContinuerunningtheprogram.Shorthandisc.nextNextline,butstepoverfunctioncalls.Shorthandisn.stepNextline,butstepintofunctioncalls.Shorthandiss.quitExitGDB.helpListthetypesofcommands.Youcanthengethelpontheclassofcommandaswellasthecommand.

cd,pwd,makeThisisjustlikerunningthesecommandsinyourshell.shellQuicklystartashellsoyoucandootherthings.clearClearabreakpoint.infobreak,infowatchShowinformationaboutbreakpointsandwatchpoints.attachpidAttachtoarunningprocesssoyoucandebugit.detachDetachfromtheprocess.listListoutthenexttensourcelines.Adda-tolisttheprevioustenlines.

LLDBQuickReferenceInOSX,younolongerhaveGDBandinsteadmustuseasimilarprogramcalledLLDBDebugger(LLDB).Thecommandsarealmostthesame,buthere’saquickreferenceforLLDB:

run[args]Startyourprogramwith[args].

breakpointset--name[file:]functionSetabreakpointat[file:]function.Youcanalsouseb,whichiswayeasier.

threadbacktraceDumpabacktraceofthecurrentcallingstack.Shorthandisbt.printexprPrintthevalueofexpr.Shorthandisp.continueContinuerunningtheprogram.Shorthandisc.nextNextline,butstepoverfunctioncalls.Shorthandisn.stepNextline,butstepintofunctioncalls.Shorthandiss.quitExitLLDB.helpListthetypesofcommands.Youcanthengethelpontheclassofcommandaswellasthecommanditself.

cd,pwd,makejustlikerunningthesecommandsinyourshell.shellQuicklystartashellsoyoucandootherthings.clearClearabreakpoint.infobreak,infowatchShowinformationaboutbreakpointsandwatchpoints.attach-ppidAttachtoarunningprocesssoyoucandebugit.detachDetachfromtheprocess.listListoutthenexttensourcelines.Adda-tolisttheprevioustensources.

YoucanalsosearchonlineforquickreferencecardsandtutorialsforbothGDBandLLDB.

Exercise5.MemorizingCOperators

Whenyoulearnedyourfirstprogramminglanguage,itmostlikelyinvolvedgoingthroughabook,typingincodeyoudidn’tquiteunderstand,andthentryingtofigureouthowitworked.That’showIwrotemostofmyotherbooks,andthatworksverywellforbeginners.Inthebeginning,therearecomplextopicsyouneedtounderstandbeforeyoucangraspwhatallthesymbolsandwordsmean,soit’saneasywaytolearn.However,onceyoualreadyknowoneprogramminglanguage,thismethodoffumblingaroundlearningthesyntaxbyosmosisisn’tthemostefficientwaytolearnalanguage.Itworks,butthereisamuchfasterwaytobuildbothyourskillsinalanguageandyourconfidenceinusingit.Thismethodoflearningaprogramminglanguagemightseemlikemagic,butyou’llhavetotrustmethatitworkssurprisinglywell.HowIwantyoutolearnCistofirstmemorizeallthebasicsymbolsandsyntax,thenapplythemthroughaseriesofexercises.Thismethodisverysimilartohowyoumightlearnhumanlanguagesbymemorizingwordsandgrammar,andthenapplyingwhatyoumemorizeinconversations.Withjustasimpleamountofmemorizationeffortinthebeginning,youcangainfoundationalknowledgeandhaveaneasiertimereadingandwritingCcode.

Warning!Somepeoplearedeadagainstmemorization.Usually,theyclaimitmakesyouuncreativeandboring.I’mproofthatmemorizingthingsdoesn’tmakeyouuncreativeandboring.Ipaint,playandbuildguitars,sing,code,writebooks,andImemorizelotsofthings.Thisbeliefisentirelyunfoundedanddetrimentaltoefficientlearning.Pleaseignoreanyonetellingyouthis.

HowtoMemorizeThebestwaytomemorizesomethingisafairlysimpleprocess:

1.Createasetofflashcardsthathaveasymbolononesideandthedescriptionontheother.YoucouldalsouseaprogramcalledAnkitodothisonyourcomputer.IprefercreatingmyownbecauseithelpsmememorizethemasImakethem.

2.Randomizetheflashcardsandstartgoingthroughthemononeside.Tryyourbesttoremembertheothersideofthecardwithoutlooking.

3.Ifyoucan’trecalltheothersideofthecard,thenlookatitandrepeattheanswertoyourself,thenputthatcardintoaseparatepile.

4.Onceyougothroughallthecardsyou’llhavetwopiles:onepileofcardsyourecalledquickly,andanotheryoufailedtorecall.Pickupthefailpileanddrillyourselfononlythosecards.

5.Attheveryendofthesession,whichisusually15–30minutes,you’llhaveasetofcardsyoujustcan’trecall.Takethosecardswithyouwhereveryougo,andwhenyouhavefreetime,practicememorizingthem.

Therearemanyothertrickstomemorizingthings,butI’vefoundthatthisisthebestwaytobuildinstantrecallonthingsyouneedtobeabletouseimmediately.Thesymbols,keywords,andsyntaxofCarethingsyouneedinstantrecallon,sothismethodisthebestoneforthistask.Alsorememberthatyouneedtodobothsidesofthecards.Youshouldbeabletoreadthedescriptionandknowwhatsymbolmatchesit,aswellasknowingthedescriptionforasymbol.

Finally,youdon’thavetostopwhileyou’rememorizingtheseoperators.Thebestapproachistocombinethiswithexercisesinthisbooksoyoucanapplywhatyou’vememorized.Seethenextexerciseformoreonthis.

TheListofOperatorsThefirstoperatorsarethearithmeticoperators,whichareverysimilartoalmosteveryotherprogramminglanguage.Whenyouwritethecards,thedescriptionsideshouldsaythatit’sanarithmeticoperator,andwhatitdoes.

Relationaloperatorstestvaluesforequality,andagain,theyareverycommoninprogramminglanguages.

Logicaloperatorsperformlogictests,andyoushouldalreadyknowwhatthesedo.Theonlyoddoneisthelogicalternary,whichyou’lllearnlaterinthisbook.

Bitwiseoperatorsdosomethingyoulikelywon’texperienceofteninmoderncode.Theyalterthebitsthatmakeupbytesandotherdatatypesinvariousways.Iwon’tcoverthisinmybook,buttheyareveryhandywhenworkingwithcertaintypesoflower-levelsystems.

Assignmentoperatorssimplyassignexpressionstovariables,butCcombinesalargenumberofotheroperatorswithassignment.SowhenIsayand-equal,Imeanthebitwiseoperators,notthelogicaloperators.

I’mcallingthesedataoperatorsbuttheyreallydealwithaspectsofpointers,memberaccess,andvariouselementsofdatastructuresinC.

Finally,thereareafewmiscellaneoussymbolsthatareeitherfrequentlyusedfordifferentroles(like,),ordon’tfitintoanyofthepreviouscategoriesforvariousreasons.

Studyyourflashcardswhileyoucontinuewiththebook.Ifyouspent15–30minutesadaybeforestudying,andanother15–30minutesbeforebed,youcouldmostlikelymemorizealloftheseinafewweeks.

Exercise6.MemorizingCSyntax

Afterlearningtheoperators,it’stimetomemorizethekeywordsandbasicsyntaxstructuresyou’llbeusing.TrustmewhenItellyouthatthesmallamountoftimespentmemorizingthesethingswillpayhugedividendslaterasyougothroughthebook.AsImentionedinExercise5,youdon’thavetostopreadingthebookwhileyoumemorizethesethings.Youcanandshoulddoboth.Useyourflashcardsasawarmupbeforecodingthatday.Takethemoutanddrillonthemfor15–30minutes,thensitdownanddosomemoreexercisesinthebook.Asyougothroughthebook,trytousethecodeyou’retypingasmoreofawaytopracticewhatyou’rememorizing.Onetrickistobuildapileofflashcardscontainingoperatorsandkeywordsthatyoudon’timmediatelyrecognizewhileyou’recoding.Afteryou’redonefortheday,practicethoseflashcardsforanother15–30minutes.Keepthisupandyou’lllearnCmuchfasterandmoresolidlythanyouwouldifyoujuststumbledaroundtypingcodeuntilyoumemorizeditsecondhand.

TheKeywordsThekeywordsofalanguagearewordsthataugmentthesymbolssothatthelanguagereadswell.TherearesomelanguageslikeAPLthatdon’treallyhavekeywords.ThereareotherlanguageslikeForthandLISPthatarealmostnothingbutkeywords.InthemiddlearelanguageslikeC,Python,Ruby,andmanymorethatmixsetsofkeywordswithsymbolstocreatethebasisofthelanguage.

Warning!Thetechnicaltermforprocessingthesymbolsandkeywordsofaprogramminglanguageislexicalanalysis.Thewordforoneofthesesymbolsorkeywordsisalexeme.

SyntaxStructuresIsuggestyoumemorizethosekeywords,aswellasmemorizingthesyntaxstructures.AsyntaxstructureisapatternofsymbolsthatmakeupaCprogramcodeform,suchastheformofanif-statementorawhile-loop.Youshouldfindmostofthesefamiliar,sinceyoualreadyknowonelanguage.TheonlytroubleisthenlearninghowCdoesit.Here’showyoureadthese:

1.AnythinginALLCAPSismeantasareplacementspotorhole.2.Seeing[ALLCAPS]meansthatpartisoptional.3.Thebestwaytotestyourmemoryofsyntaxstructuresistoopenatexteditor,andwhereyouseeswitch-statement,trytowritethecodeformaftersayingwhatitdoes.

Anif-statementisyourbasiclogicbranchingcontrol:if(TEST){CODE;}elseif(TEST){CODE;}else{CODE;}

Aswitch-statementislikeanif-statementbutworksonsimpleintegerconstants:switch(OPERAND){caseCONSTANT:CODE;break;default:CODE;}

Awhile-loopisyourmostbasicloop:

while(TEST){CODE;}

Youcanalsousecontinuetocauseittoloop.Callthisformwhile-continue-loopfornow:while(TEST){if(OTHER_TEST){continue;}CODE;}

Youcanalsousebreaktoexitaloop.Callthisformwhile-break-loop:while(TEST){if(OTHER_TEST){break;}CODE;}

Thedo-while-loopisaninvertedversionofawhile-loopthatrunsthecodethenteststoseeifitshouldrunagain:

do{CODE;}while(TEST);

Itcanalsohavecontinueandbreakinsidetocontrolhowitoperates.Thefor-loopdoesacontrolledcountedloopthrougha(hopefully)fixednumberofiterationsusingacounter:

for(INIT;TEST;POST){CODE;}

Anenumcreatesasetofintegerconstants:Clickheretoviewcodeimage

enum{CONST1,CONST2,CONST3}NAME;

Agotowilljumptoalabel,andisonlyusedinafewusefulsituationslikeerrordetectionandexiting:

if(ERROR_TEST){gotofail;}

fail:CODE;

Afunctionisdefinedthisway:TYPENAME(ARG1,ARG2,..){CODE;returnVALUE;}

Thatmaybehardtoremember,sotrythisexampletoseewhat’smeantbyTYPE,NAME,ARGandVALUE:

intname(arg1,arg2){CODE;return0;}

Atypedefdefinesanewtype:Clickheretoviewcodeimage

typedefDEFINITIONIDENTIFIER;

Amoreconcreteformofthisis:Clickheretoviewcodeimage

typedefunsignedcharbyte;

Don’tletthespacesfoolyou;theDEFINITIONisunsignedcharandtheIDENTIFIERisbyteinthatexample.Astructisapackagingofmanybasedatatypesintoasingleconcept,whichareusedheavilyinC:

structNAME{ELEMENTS;}[VARIABLE_NAME];

The[VARIABLE_NAME]isoptional,andIprefernottouseitexceptinafewsmallcases.Thisiscommonlycombinedwithtypedeflikethis:Clickheretoviewcodeimage

typedefstruct[STRUCT_NAME]{ELEMENTS;}IDENTIFIER;

Finally,unioncreatessomethinglikeastruct,buttheelementswilloverlapinmemory.Thisisstrangetounderstand,sosimplymemorizetheformfornow:

unionNAME{ELEMENTS;}[VARIABLE_NAME];

AWordofEncouragementOnceyou’vecreatedflashcardsforeachofthese,drillonthemintheusualwaybystartingwiththenameside,andthenreadingthedescriptionandformontheotherside.Inthevideoforthisexercise,IshowyouhowtouseAnkitodothisefficiently,butyoucanreplicatetheexperiencewithsimpleindexcards,too.I’venoticedsomefearordiscomfortinstudentswhoareaskedtomemorizesomethinglikethis.I’mnotexactlysurewhy,butIencourageyoutodoitanyway.Lookatthisasanopportunitytoimproveyourmemorizationandlearningskills.Themoreyoudoit,thebetteratityougetandtheeasieritgets.It’snormaltofeeldiscomfortandfrustration.Don’ttakeitpersonally.Youmightspend15minutesandsimplyhatedoingitandfeellikeatotalfailure.Thisisnormal,anditdoesn’tmeanyouactuallyareafailure.Perseverancewillgetyoupasttheinitialfrustration,andthislittleexercisewillteachyoutwothings:

1.Youcanusememorizationasaself-evaluationofyourcompetence.Nothingtellsyouhowwellyoureallyknowasubjectlikeamemorytestofitsconcepts.

2.Thewaytoconquerdifficultyisalittlepieceatatime.Programmingisagreatwaytolearnthisbecauseit’ssoeasytobreakdownintosmallpartsandfocusonwhat’slacking.Takethisasanopportunitytobuildyourconfidenceintacklinglargetasksinsmallpieces.

AWordofWarningI’lladdafinalwordofwarningaboutmemorization.Memorizingalargequantityoffactsdoesn’tautomaticallymakeyougoodatapplyingthosefacts.YoucanmemorizetheentireANSICstandardsdocumentandstillbeaterribleprogrammer.I’veencounteredmanysupposedCexpertswhoknoweverysquareinchofstandardCgrammarbutstillwriteterrible,buggy,weirdcode,ordon’tcodeatall.Neverconfuseanabilitytoregurgitatememorizedfactswithabilitytoactuallydosomethingwell.Todothatyouneedtoapplythesefactsindifferentsituationsuntilyouknowhowtousethem.That’swhattherestofthisbookwillhelpyoudo.

Exercise7.VariablesandTypes

YoushouldbegettingagraspofhowasimpleCprogramisstructured,solet’sdothenextsimplestthingandmakesomevariablesofdifferenttypes:

ex7.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5intdistance=100;6floatpower=2.345f;7doublesuper_power=56789.4532;8charinitial='A';9charfirst_name[]="Zed";10charlast_name[]="Shaw";1112printf("Youare%dmilesaway.\n",distance);13printf("Youhave%flevelsofpower.\n",power);14printf("Youhave%fawesomesuperpowers.\n",super_power);15printf("Ihaveaninitial%c.\n",initial);16printf("Ihaveafirstname%s.\n",first_name);17printf("Ihavealastname%s.\n",last_name);18printf("Mywholenameis%s%c.%s.\n",19first_name,initial,last_name);2021intbugs=100;22doublebug_rate=1.2;2324printf("Youhave%dbugsattheimaginaryrateof%f.\n",25bugs,bug_rate);2627longuniverse_of_defects=1L*1024L*1024L*1024L;28printf("Theentireuniversehas%ldbugs.\n",universe_of_defects);2930doubleexpected_bugs=bugs*bug_rate;31printf("Youareexpectedtohave%fbugs.\n",expected_bugs);3233doublepart_of_universe=expected_bugs/universe_of_defects;34printf("Thatisonlya%eportionoftheuniverse.\n",35part_of_universe);3637//thismakesnosense,justademoofsomethingweird38charnul_byte='\0';39intcare_percentage=bugs*nul_byte;40printf("Whichmeansyoushouldcare%d%%.\n",care_percentage);4142return0;43}

Inthisprogram,we’redeclaringvariablesofdifferenttypesandthenprintingthemusingdifferentprintfformatstrings.Icanbreakitdownasfollows:

ex7.c:1-4TheusualstartofaCprogram.ex7.c:5-6Declareanintanddoubleforsomefakebugdata.

ex7.c:8-9Printoutthosetwo,sonothingnewhere.ex7.c:11Declareahugenumberusinganewtype,long,forstoringbignumbers.ex7.c:12-13Printoutthatnumberusing%ldthataddsamodifiertotheusual%d.Addingl(theletter)tellstheprogramtoprintthenumberasalongdecimal.

ex7.c:15-17Thisisjustmoremathandprinting.ex7.c:19-21Craftadepictionofyourbugratecomparedtothebugsintheuniverse,whichisacompletelyinaccuratecalculation.It’ssosmallthatwehavetouse%etoprintitinscientificnotation.

ex7.c:24Makeacharacter,withaspecialsyntax'\0'thatcreatesanulbytecharacter.Thisiseffectivelythenumber0.

ex7.c:25Multiplybugsbythischaracter,whichproduces0,asinhowmuchyoushouldcare.Thisdemonstratesanuglyhackyoumightseesometimes.

ex7.c:26-27Printthatout,andnoticewe’veused%%(twopercentsigns)sothatwecanprinta%(percent)character.

ex7.c:28-30Theendofthemainfunction.Thissourcefiledemonstrateshowsomemathworkswithdifferenttypesofvariables.Attheendoftheprogram,italsodemonstratessomethingyouseeinCbutnotinmanyotherlanguages.ToC,acharacterisjustaninteger.It’sareallysmallinteger,butthat’sallitis.Thismeansyoucandomathonthem,andalotofsoftwaredoesjustthat—forgoodorbad.ThislastbitisyourfirstglanceathowCgivesyoudirectaccesstothemachine.We’llbeexploringthatmoreinlaterexercises.

WhatYouShouldSeeAsusual,here’swhatyoushouldseefortheoutput:

Exercise7Session



$./ex7Youhave100bugsattheimaginaryrateof1.200000.

Theentireuniversehas1073741824bugs.

Youareexpectedtohave120.000000bugs.

Thatisonlya1.117587e-07portionoftheuniverse.

Whichmeansyoushouldcare0%.

$

HowtoBreakItAgain,gothroughthisandtrytobreaktheprintfbypassinginthewrongarguments.Seewhathappensifyoutrytoprintoutthenul_bytevariablealongwith%sversus%c.Whenyoubreakit,runitunderthedebuggertoseewhatitsaysaboutwhatyoudid.

ExtraCredit•Makethenumberyouassigntouniverse_of_defectsvarioussizesuntilyougetawarningfromthecompiler.•Whatdothesereallyhugenumbersactuallyprintout?•Changelongtounsignedlongandtrytofindthenumberthatmakesittoobig.•Gosearchonlinetofindoutwhatunsigneddoes.•Trytoexplaintoyourself(beforeIdointhenextexercise)whyyoucanmultiplyacharandanint.

Exercise8.If,Else-If,Else

InC,therereallyisn’taBooleantype.Instead,anyintegerthat’s0isfalseorotherwiseit’strue.Inthelastexercise,theexpressionargc>1actuallyresultedin1or0,notanexplicitTrueorFalselikeinPython.ThisisanotherexampleofCbeingclosertohowacomputerworks,becausetoacomputer,truthvaluesarejustintegers.However,Cdoeshaveatypicalif-statementthatusesthisnumericideaoftrueandfalsetodobranching.It’sfairlysimilartowhatyouwoulddoinPythonandRuby,asyoucanseeinthisexercise:

ex8.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5inti=0;67if(argc==1){8printf("Youonlyhaveoneargument.Yousuck.\n");9}elseif(argc>1&&argc<4){10printf("Here'syourarguments:\n");1112for(i=0;i<argc;i++){13printf("%s",argv[i]);14}15printf("\n");16}else{17printf("Youhavetoomanyarguments.Yousuck.\n");18}1920return0;21}

Theformatfortheif-statementisthis:

if(TEST){CODE;}elseif(TEST){CODE;}else{CODE;}

ThisislikemostotherlanguagesexceptforsomespecificCdifferences:•Asmentionedbefore,theTESTpartsarefalseiftheyevaluateto0,orotherwisetrue.•YouhavetoputparenthesesaroundtheTESTelements,whilesomeotherlanguagesletyouskipthat.•Youdon’tneedthe{}bracestoenclosethecode,butitisverybadformtonotusethem.Thebracesmakeitclearwhereonebranchofcodebeginsandends.Ifyoudon’tincludethemthenobnoxiouserrorscomeup.

Otherthanthat,thecodeworksthewayitdoesinmostotherlanguages.Youdon’tneedtohaveeitherelseiforelseparts.

WhatYouShouldSeeThisoneisprettysimpletorunandtryout:

Exercise8Session



$./ex8Youonlyhaveoneargument.Yousuck.

$./ex8oneHere'syourarguments:

./ex8one

$./ex8onetwoHere'syourarguments:

./ex8onetwo

$./ex8onetwothreeYouhavetoomanyarguments.Yousuck.

$

HowtoBreakItThisoneisn’teasytobreakbecauseit’ssosimple,buttrymessingupthetestsintheif-statement:

•Removetheelseattheend,andtheprogramwon’tcatchtheedgecase.•Changethe&&toa||soyougetanorinsteadofanandtestandseehowthatworks.

ExtraCredit•Youwerebrieflyintroducedto&&,whichdoesanandcomparison,sogoresearchonlinethedifferentBooleanoperators.•Writeafewmoretestcasesforthisprogramtoseewhatyoucancomeupwith.•Isthefirsttestreallysayingtherightthing?Toyou,thefirstargumentisn’tthesamefirstargumentauserentered.Fixit.

Exercise9.While-LoopandBooleanExpressions

ThefirstloopingconstructI’llshowyouisthewhile-loop,andit’sthesimplest,usefulloopyoucouldpossiblyuseinC.Here’sthisexercise’scodefordiscussion:

ex9.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5inti=0;6while(i<25){7printf("%d",i);8i++;9}1011return0;12}

Fromthiscode,andfromyourmemorizationofthebasicsyntax,youcanseethatawhile-loopissimplythis:

while(TEST){CODE;}

ItsimplyrunstheCODEaslongasTESTistrue(1).Sotoreplicatehowthefor-loopworks,weneedtodoourowninitializingandincrementingofi.Rememberthati++incrementsiwiththepost-incrementoperator.Referbacktoyourlistoftokensifyoudidn’trecognizethat.

WhatYouShouldSeeTheoutputisbasicallythesame,soIjustdiditalittledifferentlysothatyoucanseeitrunanotherway.

Exercise9Session



$./ex9arg0:./ex9

state0:California

state1:Oregon

state2:Washington

state3:Texas

$$./ex9testitarg0:./ex9

arg1:test

arg2:it

state0:California

state1:Oregon

state2:Washington

state3:Texas

$

HowtoBreakItThereareseveralwaystogetawhile-loopwrong,soIdon’trecommendyouuseitunlessyoumust.Hereareafeweasywaystobreakit:

•Forgettoinitializethefirstinti;.Dependingonwhatistartswith,theloopmightnotrunatall,orrunforanextremelylongtime.•Forgettoinitializethesecondloop’sisothatitretainsthevaluefromtheendofthefirstloop.Nowyoursecondloopmightormightnotrun.•Forgettodoai++incrementattheendoftheloopandyou’llgetaforeverloop,oneofthedreadedproblemscommoninthefirstdecadeortwoofprogramming.

ExtraCredit•Maketheloopcountbackwardbyusingi--tostartat25andgoto0.•Writeafewmorecomplexwhile-loopsusingwhatyouknowsofar.

Exercise10.SwitchStatements

Inotherlanguages,likeRuby,youhaveaswitch-statementthatcantakeanyexpression.Somelanguages,likePython,don’thaveaswitch-statementbecauseanif-statementwithBooleanexpressionsisaboutthesamething.Fortheselanguages,switch-statementsaremorelikealternativestoif-statementsandworkthesameinternally.InC,theswitch-statementisactuallyquitedifferentandisreallyajumptable.InsteadofrandomBooleanexpressions,youcanonlyputexpressionsthatresultinintegers.Theseintegersareusedtocalculatejumpsfromthetopoftheswitchtothepartthatmatchesthatvalue.Here’ssomecodetohelpyouunderstandthisconceptofjumptables:

ex10.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5inti=0;67//gothrougheachstringinargv8//whyamIskippingargv[0]?9for(i=1;i<argc;i++){10printf("arg%d:%s\n",i,argv[i]);11}1213//let'smakeourownarrayofstrings14char*states[]={15"California","Oregon",16"Washington","Texas"17};1819intnum_states=4;2021for(i=0;i<num_states;i++){22printf("state%d:%s\n",i,states[i]);23}2425return0;26}

Inthisprogram,wetakeasinglecommandlineargumentandprintoutallvowelsinanincrediblytediouswaytodemonstrateaswitch-statement.Here’showtheswitch-statementworks:

•Thecompilermarkstheplaceintheprogramwheretheswitch-statementstarts.Let’scallthislocationY.•Itthenevaluatestheexpressioninswitch(letter)tocomeupwithanumber.Inthiscase,thenumberwillbetherawASCIIcodeoftheletterinargv[1].•Thecompileralsotranslateseachofthecaseblockslikecase'A':intoalocationintheprogramthat’sthatfaraway.Sothecodeundercase'A'isatY+Aintheprogram.•ItthendoesthemathtofigureoutwhereY+letterislocatedintheswitch-statement,and

ifit’stoofar,thenitadjustsittoY+default.•Onceitknowsthelocation,theprogramjumpstothatspotinthecode,andthencontinuesrunning.Thisiswhyyouhavebreakonsomeofthecaseblocksbutnotonothers.•If'a'isentered,thenitjumpstocase'a'.There’snobreak,soit“fallsthrough”totheonerightunderit,case'A',whichhascodeandabreak.•Finally,itrunsthiscode,hitsthebreak,andthenexitsoutoftheswitch-statemententirely.

Thisisadeepdiveintohowtheswitch-statementworks,butinpracticeyoujusthavetorememberafewsimplerules:

•Alwaysincludeadefault:branchsothatyoucatchanymissinginputs.•Don’tallowfallthroughunlessyoureallywantit.It’salsoagoodideatoadda//fallthroughcommentsopeopleknowit’sonpurpose.•Alwayswritethecaseandthebreakbeforeyouwritethecodethatgoesinit.•Trytouseif-statementsinsteadifyoucan.

WhatYouShouldSeeHere’sanexampleofmeplayingwiththis,andalsodemonstratingvariouswaystopassintheargument:

Exercise10Session



$./ex10ERROR:Youneedoneargument.

$$./ex10Zed0:Zisnotavowel

1:'E'

2:disnotavowel

$$./ex10ZedShawERROR:Youneedoneargument.

$$./ex10"ZedShaw"0:Zisnotavowel

1:'E'

2:disnotavowel

3:isnotavowel

4:Sisnotavowel

5:hisnotavowel

6:'A'

7:wisnotavowel

$

Rememberthatthere’sanif-statementatthetopthatexitswithareturn1;whenyoudon’tprovideenougharguments.Areturnthat’snot0indicatestotheOSthattheprogramhadanerror.Youcantestforanyvaluethat’sgreaterthan0inscriptsandotherprogramstofigureoutwhathappened.

HowtoBreakItIt’sincrediblyeasytobreakaswitch-statement.Herearejustafewwaysyoucanmessoneoftheseup:

•Forgetabreak,andit’llruntwoormoreblocksofcodeyoudon’twantittorun.•Forgetadefault,andit’llsilentlyignorevaluesyouforgot.•Accidentallyputavariableintotheswitchthatevaluatestosomethingunexpected,likeanint,whichbecomesweirdvalues.•Useuninitializedvaluesintheswitch.

Youcanalsobreakthisprograminafewotherways.Seeifyoucanbustityourself.

ExtraCredit•Writeanotherprogramthatusesmathonthelettertoconvertittolowercase,andthenremovealloftheextraneousuppercaselettersintheswitch.•Usethe','(comma)toinitializeletterinthefor-loop.•Makeithandlealloftheargumentsyoupassitwithyetanotherfor-loop.•Convertthisswitch-statementtoanif-statement.Whichdoyoulikebetter?•Inthecasefor'Y'Ihavethebreakoutsideoftheif-statement.What’stheimpactofthis,andwhathappensifyoumoveitinsideoftheif-statement.Provetoyourselfthatyou’reright.

Exercise11.ArraysandStrings

ThisexerciseshowsyouthatCstoresitsstringssimplyasanarrayofbytes,terminatedwiththe'\0'(nul)byte.Youprobablycluedintothisinthelastexercisesincewediditmanually.Here’showwedoitinanotherwaytomakeitevenclearerbycomparingittoanarrayofnumbers:

ex11.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5intnumbers[4]={0};6charname[4]={'a'};78//first,printthemoutraw9printf("numbers:%d%d%d%d\n",10numbers[0],numbers[1],numbers[2],numbers[3]);1112printf("nameeach:%c%c%c%c\n",13name[0],name[1],name[2],name[3]);1415printf("name:%s\n",name);1617//setupthenumbers18numbers[0]=1;19numbers[1]=2;20numbers[2]=3;21numbers[3]=4;2223//setupthename24name[0]='Z';25name[1]='e';26name[2]='d';27name[3]='\0';2829//thenprintthemoutinitialized30printf("numbers:%d%d%d%d\n",31numbers[0],numbers[1],numbers[2],numbers[3]);3233printf("nameeach:%c%c%c%c\n",34name[0],name[1],name[2],name[3]);3536//printthenamelikeastring37printf("name:%s\n",name);3839//anotherwaytousename40char*another="Zed";4142printf("another:%s\n",another);4344printf("anothereach:%c%c%c%c\n",45another[0],another[1],another[2],another[3]);4647return0;48}

Inthiscode,wesetupsomearraysthetediousway,byassigningavaluetoeachelement.Innumbers,wearesettingupnumbers;butinname,we’reactuallybuildingastringmanually.

WhatYouShouldSeeWhenyourunthiscode,youshouldfirstseethearraysprintedwiththeircontentsinitializedto0(zero),theninitsinitializedform.

Exercise11Session



$./ex11numbers:0000

nameeach:a

name:a

numbers:1234

nameeach:Zed

name:Zed

another:Zed

anothereach:Zed

$

You’llnoticesomeinterestingthingsaboutthisprogram:•Ididn’thavetogiveallfourelementsofthearraystoinitializethem.ThisisashortcutinC.Ifyousetjustoneelement,it’llfillintherestwith0.•Wheneachelementofnumbersisprinted,theyallcomeoutas0.•Wheneachelementofnameisprinted,onlythefirstelement'a'showsupbecausethe'\0'characterisspecialandwon’tdisplay.•Thenthefirsttimeweprintname,itonlyprintsthelettera.Thisisbecausethearraywillbefilledwith0afterthefirst'a'intheinitializer,sothestringiscorrectlyterminatedbya'\0'character.•Wethensetupthearrayswithatedious,manualassignmenttoeachthingandprintthemoutagain.Lookathowtheychanged.Nowthenumbersareset,butdoyouseehowthenamestringprintsmynamecorrectly?•Therearealsotwosyntaxesfordoingastring:charname[4]={'a'}online6versuschar*another="name"online44.Thefirstoneislesscommonandthesecondiswhatyoushoulduseforstringliteralslikethis.

NoticethatI’musingthesamesyntaxandstyleofcodetointeractwithbothanarrayofintegersandanarrayofcharacters,butprintfthinksthatthenameisjustastring.Again,thisisbecausetheClanguagedoesn’tdifferentiatebetweenastringandanarrayofcharacters.Finally,whenyoumakestringliteralsyoushouldtypicallyusethechar*another="Literal"syntax.Thisworksouttobethesamething,butit’smoreidiomaticandeasiertowrite.

HowtoBreakItThesourceofalmostallbugsinCcomefromforgettingtohaveenoughspace,orforgettingtoputa'\0'attheendofastring.Infact,it’ssocommonandhardtogetrightthatthemajorityofgoodCcodejustdoesn’tuseC-stylestrings.Inlaterexercises,we’llactuallylearnhowtoavoidCstringscompletely.Inthisprogram,thekeytobreakingitistoforgettoputthe'\0'characterattheendofthestrings.Thereareafewwaystodothis:

•Getridoftheinitializersthatsetupname.•Accidentallysetname[3]='A';sothatthere’snoterminator.•Settheinitializerto{'a','a','a','a'}sothattherearetoomany'a'charactersandnospaceforthe'\0'terminator.

Trytocomeupwithsomeotherwaystobreakthis,andrunalloftheseunderthedebuggersoyoucanseeexactlywhat’sgoingonandwhattheerrorsarecalled.Sometimesyou’llmakethesemistakesandevenadebuggercan’tfindthem.Trymovingwhereyoudeclarethevariablestoseeifyougetanerror.ThisispartofthevoodooofC:Sometimesjustwherethevariableislocatedchangesthebug.

ExtraCredit•Assignthecharactersintonumbers,andthenuseprintftoprintthemonecharacteratatime.Whatkindofcompilerwarningsdoyouget?•Dotheinverseforname,tryingtotreatitlikeanarrayofintandprintitoutoneintatatime.Whatdoesthedebuggerthinkofthat?•Inhowmanyotherwayscanyouprintthisout?•Ifanarrayofcharactersis4byteslong,andanintegeris4byteslong,thencanyoutreatthewholenamearraylikeit’sjustaninteger?Howmightyouaccomplishthiscrazyhack?•Takeoutapieceofpaperanddraweachofthesearraysasarowofboxes.Thendotheoperationsyoujustdidonpapertoseeifyougetthemright.•Convertnametobeinthestyleofanotherandseeifthecodekeepsworking.

Exercise12.SizesandArrays

Inthelastexercise,youdidmathbutwith,a'\0'(nul)character.Thismayseemoddifyou’recomingfromotherlanguages,sincetheytrytotreatstringsandbytearraysasdifferentbeasts.Ctreatsstringsasjustarraysofbytes,andit’sonlythedifferentprintingfunctionsthatrecognizeadifference.BeforeIcanreallyexplainthesignificanceofthis,Ihavetointroduceacouplemoreconcepts:sizeofandarrays.Here’sthecodewe’llbetalkingabout:

ex12.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5intareas[]={10,12,13,14,20};6charname[]="Zed";7charfull_name[]={8'Z','e','d',9'','A','.','',10'S','h','a','w','\0'11};1213//WARNING:Onsomesystemsyoumayhavetochangethe14//%ldinthiscodetoa%usinceitwilluseunsignedints15printf("Thesizeofanint:%ld\n",sizeof(int));16printf("Thesizeofareas(int[]):%ld\n",sizeof(areas));17printf("Thenumberofintsinareas:%ld\n",18sizeof(areas)/sizeof(int));19printf("Thefirstareais%d,the2nd%d.\n",areas[0],areas[1]);2021printf("Thesizeofachar:%ld\n",sizeof(char));22printf("Thesizeofname(char[]):%ld\n",sizeof(name));23printf("Thenumberofchars:%ld\n",sizeof(name)/sizeof(char));2425printf("Thesizeoffull_name(char[]):%ld\n",sizeof(full_name));26printf("Thenumberofchars:%ld\n",27sizeof(full_name)/sizeof(char));2829printf("name=\"%s\"andfull_name=\"%s\"\n",name,full_name);3031return0;32}

Inthiscode,wecreateafewarrayswithdifferentdatatypesinthem.BecausearraysofdataaresocentraltohowCworks,thereareahugenumberofwaystocreatethem.Fornow,justusethesyntaxtypename[]={initializer};andwe’llexploremorelater.Whatthissyntaxmeansis,“Iwantanarrayoftypethatisinitializedto{..}.”WhenCseesthis,itknowsto:

•Lookatthetype,andinthisfirstcase,it’sint.•Lookatthe[]andseethatthere’snolengthgiven.•Lookattheinitializer{10,12,13,14,20}andfigureoutthatyouwantthosefiveintegersinyourarray.

•Createapieceofmemoryinthecomputerthatcanhold5integersoneafteranother.•Takethenameyouwant,areas,andassignitthislocation.

Inthecaseofareas,it’screatinganarrayoffiveintegersthatcontainthosenumbers.Whenitgetstocharname[]="Zed";it’sdoingthesamething,exceptit’screatinganarrayofthreecharactersandassigningthattoname.Thefinalarraywemakeisfull_name,butweusetheannoyingsyntaxofspellingitoutonecharacteratatime.ToC,nameandfull_nameareidenticalmethodsofcreatingachararray.Intherestofthefile,we’reusingakeywordcalledsizeoftoaskChowbigthingsareinbytes.Cisallaboutthesizeandlocationofpiecesofmemory,andwhatyoudowiththem.Tohelpyoukeepthisstraight,itgivesyousizeofsothatyoucanaskhowbigsomethingisbeforeyouworkwithit.Thisiswherestuffgetstricky,solet’srunthiscodefirstandthenexplainitlater.

WhatYouShouldSee

Exercise12Session



$./ex12Thesizeofanint:4

Thesizeofareas(int[]):20

Thenumberofintsinareas:5

Thefirstareais10,the2nd12.

Thesizeofachar:1

Thesizeofname(char[]):4

Thenumberofchars:4

Thesizeoffull_name(char[]):12

Thenumberofchars:12

name="Zed"andfull_name="ZedA.Shaw"

$

NowyouseetheoutputofthesedifferentprintfcallsandstarttogetaglimpseofwhatCisdoing.Youroutputcouldactuallybetotallydifferentfrommine,sinceyourcomputermighthavedifferentsizeintegers.I’llgothroughmyoutput:

5Mycomputerthinksanintis4bytesinsize.Yourcomputermightuseadifferentsizeifit’sa32-bitversus64-bitCPU.

6Theareasarrayhasfiveintegersinit,soitmakessensethatmycomputerrequires20bytestostoreit.

7Ifwedividethesizeofareasbythesizeofanint,thenwegetfiveelements.Lookingatthecode,thismatcheswhatweputintheinitializer.

8Wethendidanarrayaccesstogetareas[0]andareas[1],whichmeansCiszeroindexedlikePythonandRuby.

9-11Werepeatthisforthenamearray,butdoyounoticesomethingoddaboutthesizeofthearray?Itsaysit’s4byteslong,butweonlytyped“Zed”forthreecharacters.Where’sthefourthonecomingfrom?

12-13Wedothesamethingwithfull_name,andnownoticeitgetsthiscorrect.

13Finally,wejustprintoutthenameandfull_nametoprovethattheyactuallyare“strings”accordingtoprintf.

Makesureyoucangothroughandseehowtheseoutputlinesmatchwhatwascreated.We’llbebuildingonthis,andexploringmoreaboutarraysandstoragenext.

HowtoBreakItBreakingthisprogramisfairlyeasy.Trysomeofthese:

•Getridofthe'\0'attheendoffull_nameandrerunit.Runitunderthedebuggertoo.Now,movethedefinitionoffull_nametothetopofmainbeforeareas.Tryrunningitunderthedebuggerafewtimesandseeifyougetsomenewerrors.Insomecases,youmightstillgetluckyandnotcatchanyerrors.•Changeitsothatinsteadofareas[0]youtrytoprintareas[10].Seewhatthedebuggerthinksofthat.•Tryotherwaystobreakitlikethis,doingittonameandfull_name,too.

ExtraCredit•Tryassigningtoelementsintheareasarraywithareas[0]=100;andsimilar.•Tryassigningtoelementsofnameandfull_name.•Trysettingoneelementofareastoacharacterfromname.•SearchonlineforthedifferentsizesusedforintegersondifferentCPUs.

Exercise13.For-LoopsandArraysofStrings

Youcanmakeanarrayofvarioustypeswiththeideathatastringandanarrayofbytesarethesamething.Thenextstepistodoanarraythathasstringsinit.We’llalsointroduceyourfirstloopingconstruct,thefor-loop,tohelpprintoutthisnewdatastructure.Thefunpartofthisisthatthere’sbeenanarrayofstringshidinginyourprogramsforawhilenow:thechar*argv[]inthemainfunctionarguments.Here’scodethatwillprintoutanycommandlineargumentsyoupassit:

ex13.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5if(argc!=2){6printf("ERROR:Youneedoneargument.\n");7//thisishowyouabortaprogram8return1;9}1011inti=0;12for(i=0;argv[1][i]!='\0';i++){13charletter=argv[1][i];1415switch(letter){16case'a':17case'A':18printf("%d:'A'\n",i);19break;2021case'e':22case'E':23printf("%d:'E'\n",i);24break;2526case'i':27case'I':28printf("%d:'I'\n",i);29break;3031case'o':32case'O':33printf("%d:'O'\n",i);34break;3536case'u':37case'U':38printf("%d:'U'\n",i);39break;4041case'y':42case'Y':43if(i>2){44//it'sonlysometimesY45printf("%d:'Y'\n",i);

46}47break;4849default:50printf("%d:%cisnotavowel\n",i,letter);51}52}5354return0;55}

Theformatofafor-loopisthis:Clickheretoviewcodeimage

for(INITIALIZER;TEST;INCREMENTER){CODE;}

Here’showthefor-loopworks:•TheINITIALIZERiscodethat’sruntosetuptheloop,whichinthiscaseisi=0.•Next,theTESTBooleanexpressionischecked.Ifit’sfalse(0),thenCODEisskipped,doingnothing.•TheCODErunsanddoeswhateveritdoes.•AftertheCODEruns,theINCREMENTERpartisrun,usuallyincrementingsomething,suchasini++.•Anditcontinuesagainwithstep2untiltheTESTisfalse(0).

Thisfor-loopisgoingthroughthecommandlineargumentsusingargcandargvlikethis:•TheOSpasseseachcommandlineargumentasastringintheargvarray.Theprogram’sname(./ex10)isat0,withtherestcomingafterit.•TheOSalsosetsargctothenumberofargumentsintheargvarray,soyoucanprocessthemwithoutgoingpasttheend.Rememberthatifyougiveoneargument,theprogram’snameisthefirst,soargcis2.•Thefor-loopsetsupwithi=1intheinitializer.•Itthenteststhatiislessthanargcwiththetesti<argc.Since$1<2$,it’llpass.•Itthenrunsthecodethatjustprintsouttheiandusesitoindexintoargv.•Theincrementeristhenrunusingthei++syntax,whichisahandywayofwritingi=i+1.•Thisthenrepeatsuntili<argcisfinallyfalse(0),theloopexits,andtheprogramcontinueson.

WhatYouShouldSeeToplaywiththisprogram,then,youhavetorunittwoways.Thefirstwayistopassinsomecommandlineargumentssothatargcandargvgetset.Thesecondistorunitwithnoargumentssoyoucanseethatthefirstfor-loopdoesn’trunifi<argcisfalse.

Exercise13Session



$./ex13iamabunchofargumentsarg1:i

arg2:am

arg3:a

arg4:bunch

arg5:of

arg6:arguments

state0:California

state1:Oregon

state2:Washington

state3:Texas

$$./ex13state0:California

state1:Oregon

state2:Washington

state3:Texas

$

UnderstandingArraysofStringsInCyoumakeanarrayofstringsbycombiningthechar*str="blah"syntaxwiththecharstr[]={'b','l','a','h'}syntaxtoconstructatwo-dimensionalarray.Thesyntaxchar*states[]={...}online14isthistwo-dimensionalcombination,eachstringbeingoneelement,andeachcharacterinthestringbeinganother.Confusing?Theconceptofmultipledimensionsissomethingmostpeopleneverthinkabout,sowhatyoushoulddoisbuildthisarrayofstringsonpaper:

•Makeagridwiththeindexofeachstringontheleft.•Thenputtheindexofeachcharacteronthetop.•Thenfillinthesquaresinthemiddlewithwhatsinglecharactergoesineachsquare.•Onceyouhavethegrid,tracethroughthecodeusingthisgridofpaper.

Anotherwaytofigurethisisoutistobuildthesamestructureinaprogramminglanguageyouaremorefamiliarwith,likePythonorRuby.

HowtoBreakIt•Takeyourfavoriteotherlanguageanduseittorunthisprogram,butincludeasmanycommandlineargumentsaspossible.Seeifyoucanbustitbygivingitwaytoomanyarguments.•Initializeito0andseewhatthatdoes.Doyouhavetoadjustargcaswell,ordoesitjustwork?Whydoes0-basedindexingworkhere?•Setnum_stateswrongsothatit’sahighervalueandseewhatitdoes.

ExtraCredit•Figureoutwhatkindofcodeyoucanputintothepartsofafor-loop.•Lookuphowtousethecommacharacter(,)toseparatemultiplestatementsinthepartsofthefor-loop,butbetweenthesemicoloncharacters(;).

•ReadaboutwhataNULLisandtrytouseitinoneoftheelementsfromthestatesarraytoseewhatit’llprint.•Seeifyoucanassignanelementfromthestatesarraytotheargvarraybeforeprintingboth.Trytheinverse.

Exercise14.WritingandUsingFunctions

Upuntilnow,we’vejustusedfunctionsthatarepartofthestdio.hheaderfile.Inthisexercise,you’llwritesomefunctionsandusesomeotherfunctions.

ex14.c


1#include<stdio.h>2#include<ctype.h>34//forwarddeclarations5intcan_print_it(charch);6voidprint_letters(chararg[]);78voidprint_arguments(intargc,char*argv[])9{10inti=0;1112for(i=0;i<argc;i++){13print_letters(argv[i]);14}15}1617voidprint_letters(chararg[])18{19inti=0;2021for(i=0;arg[i]!='\0';i++){22charch=arg[i];2324if(can_print_it(ch)){25printf("'%c'==%d",ch,ch);26}27}2829printf("\n");30}3132intcan_print_it(charch)33{34returnisalpha(ch)||isblank(ch);35}3637intmain(intargc,char*argv[])38{39print_arguments(argc,argv);40return0;41}

Inthisexamplewe’recreatingfunctionstoprintoutthecharactersandASCIIcodesforanythatarealphaorblanks.Here’sthebreakdown:

ex14.c:2Includeanewheaderfile,sowecangainaccesstoisalphaandisblank.ex14.c:5-6TellCthatyou’llbeusingsomefunctionslaterinyourprogramwithoutactuallyhavingtodefinethem.Thisisaforwarddeclarationanditsolvesthechicken-and-eggproblemofneedingtouseafunctionbeforeyou’vedefinedit.

ex14.c:8-15Definetheprint_argumentsfunction,whichknowshowtoprintthesamearrayofstringsthatmaintypicallygets.

ex14.c:17-30Definethenextfunction,print_letters,whichiscalledbyprint_argumentsandknowshowtoprinteachofthecharactersandtheircodes.

ex14.c:32-35Definecan_print_it,whichsimplyreturnsthetruthvalue(0or1)ofisalpha(ch)||isblank(ch)backtoitscaller,print_letters.

ex14.c:38-42Finally,mainsimplycallsprint_argumentstomakethewholechainoffunctionsgo.

Ishouldn’thavetodescribewhat’sineachfunction,becausethey’reallthingsyou’verunintobefore.Whatyoushouldbeabletosee,though,isthatI’vesimplydefinedfunctionsthesamewayyou’vebeendefiningmain.TheonlydifferenceisyouhavetohelpCoutbytellingitaheadoftimeifyou’regoingtousefunctionsithasn’tencounteredyetinthefile.That’swhattheforwarddeclarationsdo.

WhatYouShouldSeeToplaywiththisprogram,youjustfeeditdifferentcommandlinearguments,whichgetpassedthroughyourfunctions.Here’smeplayingwithittodemonstrate:

Exercise14Session



$./ex14'e'==101'x'==120

$./ex14hithisiscool'e'==101'x'==120

'h'==104'i'==105

't'==116'h'==104'i'==105's'==115

'i'==105's'==115

'c'==99'o'==111'o'==111'l'==108

$./ex14"Igo3spaces"'e'==101'x'==120

'I'==73''==32'g'==103'o'==111''==32''==32\

's'==115'p'==112'a'==97'c'==99'e'==101's'==115$

Theisalphaandisblankdoalltheworkoffiguringoutifthegivencharacterisaletterorablank.WhenIdothelastrun,itprintseverythingbutthe3charactersincethat’sadigit.

HowtoBreakItTherearetwodifferentkindsofbreakinginthisprogram:

•Removetheforwarddeclarationstoconfusethecompilerandcauseittocomplainaboutcan_print_itandprint_letters.•Whenyoucallprint_argumentsinsidemain,tryadding1toargcsothatitgoespastthe

endoftheargvarray.

ExtraCredit•Reworkthesefunctionssothatyouhavefewerfunctions.Forexample,doyoureallyneedcan_print_it?•Haveprint_argumentsfigureouthowlongeachargumentstringisbyusingthestrlenfunction,andthenpassthatlengthtoprint_letters.Then,rewriteprint_letterssoitonlyprocessesthisfixedlengthanddoesn’trelyonthe'\0'terminator.You’llneedthe#include<string.h>forthis.•Usemantolookupinformationonisalphaandisblank.Useothersimilarfunctionstoprintoutonlydigitsorothercharacters.•Goreadabouthowotherpeopleliketoformattheirfunctions.NeverusetheK&Rsyntax(it’santiquatedandconfusing)butunderstandwhatit’sdoingincaseyourunintosomeonewholikesit.

Exercise15.Pointers,DreadedPointers

PointersarefamousmysticalcreaturesinC.I’llattempttodemystifythembyteachingyouthevocabularytodealwiththem.Theyactuallyaren’tthatcomplex,butthey’refrequentlyabusedinweirdwaysthatmakethemhardtouse.Ifyouavoidthestupidwaystousepointers,thenthey’refairlyeasy.Todemonstratepointersinawaythatwecantalkaboutthem,I’vewrittenafrivolousprogramthatprintsagroupofpeople’sagesinthreedifferentways.

ex15.c


1#include<stdio.h>23intmain(intargc,char*argv[])4{5//createtwoarrayswecareabout6intages[]={23,43,12,89,2};7char*names[]={8"Alan","Frank",9"Mary","John","Lisa"10};1112//safelygetthesizeofages13intcount=sizeof(ages)/sizeof(int);14inti=0;1516//firstwayusingindexing17for(i=0;i<count;i++){18printf("%shas%dyearsalive.\n",names[i],ages[i]);19}2021printf("---\n");2223//setupthepointerstothestartofthearrays24int*cur_age=ages;25char**cur_name=names;2627//secondwayusingpointers28for(i=0;i<count;i++){29printf("%sis%dyearsold.\n",30*(cur_name+i),*(cur_age+i));31}3233printf("---\n");3435//thirdway,pointersarejustarrays36for(i=0;i<count;i++){37printf("%sis%dyearsoldagain.\n",cur_name[i],cur_age[i]);38}3940printf("---\n");4142//fourthwaywithpointersinastupidcomplexway43for(cur_name=names,cur_age=ages;44(cur_age-ages)<count;cur_name++,cur_age++){45printf("%slived%dyearssofar.\n",*cur_name,*cur_age);

46}4748return0;49}

Beforeexplaininghowpointerswork,let’sbreakthisprogramdownlinebylinesoyougetanideaofwhat’sgoingon.Asyougothroughthisdetaileddescription,trytoanswerthequestionsforyourselfonapieceofpaper,thenseeifwhatyouguessedmatchesmydescriptionofpointerslater.

ex15.c:6-10Createtwoarrays:agesstoringsomeintdata,andnamesstoringanarrayofstrings.

ex15.c:12-13Thesearesomevariablesforourfor-loopslater.ex15.c:16-19Thisisjustloopingthroughthetwoarraysandprintinghowoldeachpersonis.Thisisusingitoindexintothearray.

ex15.c:24Createapointerthatpointsatages.Noticetheuseofint*tocreateapointertointegertypeofpointer.That’ssimilartochar*,whichisapointertochar,andastringisanarrayofchars.Seeingthesimilarityyet?

ex15.c:25Createapointerthatpointsatnames.Achar*isalreadyapointertochar,sothat’sjustastring.However,youneedtwolevelssincenamesistwo-dimensional,whichthenmeansyouneedchar**forapointerto(apointertochar)type.Studythatandtrytoexplainittoyourself,too.

ex15.c:28-31Loopthroughagesandnamesbutusethepointersplusanoffsetofiinstead.Writing*(cur_name+i)isthesameaswritingname[i],andyoureaditas“thevalueof(pointercur_nameplusi).”

ex15.c:35-39Thisshowshowthesyntaxtoaccessanelementofanarrayisthesameforapointerandanarray.

ex15.c:44-50Thisisanotheradmittedlyinsaneloopthatdoesthesamethingastheothertwo,butinsteaditusesvariouspointerarithmeticmethods:ex15.c:44Initializeourfor-loopbysettingcur_nameandcur_agetothebeginningofthenamesandagesarrays.

ex15.c:45Thetestportionofthefor-loopthencomparesthedistanceofthepointercur_agefromthestartofages.Whydoesthatwork?

ex15.c:46Theincrementpartofthefor-loopthenincrementsbothcur_nameandcur_agesothattheypointatthenextelementofthenameandagearrays.

ex15.c:48-49Thepointerscur_nameandcur_agearenowpointingatoneelementofthearraysthattheyworkon,andwecanprintthemoutusingjust*cur_nameand*cur_age,whichmeans“thevalueofwherevercur_nameispointing.”

Thisseeminglysimpleprogramhasalargeamountofinformation,andmygoalistogetyoutoattempttofigurepointersoutforyourselfbeforeIexplainthem.Don’tcontinueuntilyou’vewrittendownwhatyouthinkapointerdoes.

WhatYouShouldSeeAfteryourunthisprogram,trytotracebackeachlineprintedouttothelineinthecodethatproducedit.Ifyouhaveto,altertheprintfcallstomakesureyou’vegottherightlinenumber.

Exercise15Session



$./ex15Alanhas23yearsalive.

Frankhas43yearsalive.

Maryhas12yearsalive.

Johnhas89yearsalive.

Lisahas2yearsalive.

---

Alanis23yearsold.

Frankis43yearsold.

Maryis12yearsold.

Johnis89yearsold.

Lisais2yearsold.

---

Alanis23yearsoldagain.

Frankis43yearsoldagain.

Maryis12yearsoldagain.

Johnis89yearsoldagain.

Lisais2yearsoldagain.

---

Alanlived23yearssofar.

Franklived43yearssofar.

Marylived12yearssofar.

Johnlived89yearssofar.

Lisalived2yearssofar.

$

ExplainingPointersWhenyoutypesomethinglikeages[i],you’reindexingintothearrayages,andyou’reusingthenumberthat’sheldinitodoit.Ifiissettozerothenit’sthesameastypingages[0].We’vebeencallingthisnumberianindexsinceit’salocationinsideagesthatwewant.Itcouldalsobecalledanaddress,whichisawayofsaying“Iwanttheintegerinagesthat’sataddressi.”Ifiisanindex,thenwhat’sages?ToC,agesisalocationinthecomputer ’smemorywherealloftheseintegersstart.It’salsoanaddress,andtheCcompilerwillreplaceagesanywhereyoutypeitwiththeaddressoftheveryfirstintegerinages.Anotherwaytothinkofagesisthatit’sthe“addressofthefirstintegerinages.”Buthere’sthetrick:agesisanaddressinsidetheentirecomputer.It’snotlikeithat’sjustanaddressinsideages.Theagesarraynameisactuallyanaddressinthecomputer.Thatleadstoacertainrealization:Cthinksyourwholecomputerisonemassivearrayofbytes.Obviously,thisisn’tveryuseful,butthenwhatCdoesislayerontopofthismassivearrayofbytestheconceptoftypesandsizesofthosetypes.Youalreadysawhowthisworkedinpreviousexercises,butnowyoustarttogetanideaofhowCisdoingthefollowingwithyourarrays:

•Creatingablockofmemoryinsideyourcomputer•Pointingthenameagesatthebeginningofthatblock•Indexingintotheblockbytakingthebaseaddressofagesandgettingtheelementthat’siawayfromthere•Convertingthataddressatages+iintoavalidintoftherightsize,suchthattheindexworks

toreturnwhatyouwant:theintatindexiIfyoucantakeabaseaddress,likeages,andaddtoitwithanotheraddresslikeitoproduceanewaddress,thencanyoujustmakesomethingthatpointsrightatthislocationallthetime?Yes,andthatthingiscalledapointer.Thisiswhatthepointerscur_ageandcur_namearedoing:Theyarevariablespointingatthelocationwhereagesandnamesliveinyourcomputer ’smemory.Theexampleprogramisthenmovingthemaroundordoingmathonthemtogetvaluesoutofthememory.Inoneinstance,theyjustadditocur_age,whichisthesameaswhattheprogramdoeswitharray[i].Inthelastfor-loop,though,thesetwopointersarebeingmovedontheirown,withoutitohelpout.Inthatloop,thepointersaretreatedlikeacombinationofarrayandintegeroffsetrolledintoone.Apointerissimplyanaddresspointingsomewhereinsidethecomputer ’smemorywithatypespecifiersothatyougettherightsizeofdatawithit.It’skindoflikeacombinationofagesandirolledintoonedatatype.Cknowswherepointersarepointing,knowsthedatatypetheypointat,thesizeofthosetypes,andhowtogetthedataforyou.Justlikewithi,youcanincrement,decrement,subtract,oraddtothem.But,justlikeages,youcanalsogetvaluesout,putnewvaluesin,anduseallofthearrayoperations.Thepurposeofapointeristoletyoumanuallyindexdataintoblocksormemorywhenanarraywon’tdoitright.Inalmostallothercases,youactuallywanttouseanarray.But,therearetimeswhenyouhavetoworkwitharawblockofmemoryandthat’swhereapointercomesin.Apointergivesyouraw,directaccesstoablockofmemorysoyoucanworkwithit.Thefinalthingtograspatthisstageisthatyoucanuseeithersyntaxformostarrayorpointeroperations.Youcantakeapointertosomething,butusethearraysyntaxtoaccessit.Youcantakeanarrayanddopointerarithmeticwithit.

PracticalPointerUsageThereareprimarilyfourusefulthingsyoucandowithpointersinCcode:

•AsktheOSforachunkofmemoryanduseapointertoworkwithit.Thisincludesstringsandsomethingyouhaven’tseenyet,structs.•Passlargeblocksofmemory(likelargestructs)tofunctionswithapointer,soyoudon’thavetopassthewholethingtothem.•Taketheaddressofafunction,soyoucanuseitasadynamiccallback.•Scancomplexchunksofmemory,convertingbytesoffofanetworksocketintodatastructuresorparsingfiles.

Fornearlyeverythingelse,youmightseepeopleusepointerswhentheyshouldbeusingarrays.IntheearlydaysofCprogramming,peopleusedpointerstospeeduptheirprograms,becausethecompilerswerereallybadatoptimizingarrayusage.Thesedays,thesyntaxtoaccessanarrayversusapointeraretranslatedintothesamemachinecodeandoptimizedinthesameway,soit’snotasnecessary.Instead,youshouldgowitharrayswheneveryoucan,andthenonlyusepointersasaperformanceoptimizationifyouabsolutelyhaveto.

ThePointerLexiconI’mnowgoingtogiveyoualittlelexicontouseforreadingandwritingpointers.Wheneveryourunintoacomplexpointerstatement,justrefertothisandbreakitdownbitbybit(orjustdon’tuseitsinceit’sprobablynotgoodcode.)

type*ptrApointeroftypenamedptr*ptrThevalueofwhateverptrispointedat*(ptr+i)Thevalueof(whateverptrispointedatplusi)&thingTheaddressofthingtype*ptr=&thingApointeroftypenamedptrsettotheaddressofthingptr++Incrementwhereptrpoints

We’llbeusingthissimplelexicontobreakdownallofthepointersweusefromnowoninthebook.

PointersAren’tArraysNomatterwhat,youshouldneverthinkthatpointersandarraysarethesamething.Theyaren’tthesamething,eventhoughCletsyouworkwiththeminmanyofthesameways.Forexample,ifyoudosizeof(cur_age)inthecodeabove,youwouldgetthesizeofthepointer,notthesizeofwhatitpointsat.Ifyouwantthesizeofthefullarray,youhavetousethearray’sname,age,asIdidonline12.Todo:Expandonthissomemorewithwhatdoesn’tworkthesameonpointersandarrays.

HowtoBreakItYoucanbreakthisprogrambysimplypointingthepointersatthewrongthings:

•Trytomakecur_agepointatnames.You’llneedtouseaCcasttoforceit,sogolookthatupandtrytofigureitout.•Inthefinalfor-loop,trygettingthemathwronginweirdways.•Tryrewritingtheloopssothattheystartattheendofthearraysandgotothebeginning.Thisisharderthanitlooks.

ExtraCredit•Rewriteallofthearraysinthisprogramaspointers.•Rewriteallofthepointersasarrays.•Gobacktosomeoftheotherprogramsthatusearraysandtrytousepointersinstead.•Processcommandlineargumentsusingjustpointers,similartohowyoudidnamesinthisone.•Playwithcombinationsofgettingthevalueofandtheaddressofthings.•Addanotherfor-loopattheendthatprintsouttheaddressesthatthesepointersareusing.You’llneedthe%pformatforprintf.•Rewritethisprogramtouseafunctionforeachofthewaysyou’reprintingoutthings.Trytopasspointerstothesefunctionssothattheyworkonthedata.Rememberyoucandeclareafunctiontoacceptapointer,butjustuseitlikeanarray.

•Changethefor-loopstowhile-loopsandseewhatworksbetterforwhichkindofpointerusage.

Exercise16.StructsAndPointerstoThem

Inthisexercise,you’lllearnhowtomakeastruct,pointapointeratit,anduseittomakesenseofinternalmemorystructures.We’llalsoapplytheknowledgeofpointersfromthelastexercise,andthengetyouconstructingthesestructuresfromrawmemoryusingmalloc.Asusual,here’stheprogramwe’lltalkabout,sotypeitinandmakeitwork.

ex16.c


1#include<stdio.h>2#include<assert.h>3#include<stdlib.h>4#include<string.h>56structPerson{7char*name;8intage;9intheight;10intweight;11};1213structPerson*Person_create(char*name,intage,intheight,14intweight)15{16structPerson*who=malloc(sizeof(structPerson));17assert(who!=NULL);1819who->name=strdup(name);20who->age=age;21who->height=height;22who->weight=weight;2324returnwho;25}2627voidPerson_destroy(structPerson*who)28{29assert(who!=NULL);3031free(who->name);32free(who);33}3435voidPerson_print(structPerson*who)36{37printf("Name:%s\n",who->name);38printf("\tAge:%d\n",who->age);39printf("\tHeight:%d\n",who->height);40printf("\tWeight:%d\n",who->weight);41}4243intmain(intargc,char*argv[])44{45//maketwopeoplestructures46structPerson*joe=Person_create("JoeAlex",32,64,140);47

48structPerson*frank=Person_create("FrankBlank",20,72,180);4950//printthemoutandwheretheyareinmemory51printf("Joeisatmemorylocation%p:\n",joe);52Person_print(joe);5354printf("Frankisatmemorylocation%p:\n",frank);55Person_print(frank);5657//makeeveryoneage20yearsandprintthemagain58joe->age+=20;59joe->height-=2;60joe->weight+=40;61Person_print(joe);6263frank->age+=20;64frank->weight+=20;65Person_print(frank);6667//destroythembothsowecleanup68Person_destroy(joe);69Person_destroy(frank);7071return0;72}

Todescribethisprogram,I’mgoingtouseadifferentapproachthanbefore.I’mnotgoingtogiveyoualine-by-linebreakdownoftheprogram,I’mgoingtomakeyouwriteit.I’mgivingyouaguideoftheprogrambasedonthepartsitcontains,andyourjobiswriteoutwhateachlinedoes.

includesIincludesomenewheaderfilesheretogainaccesstosomenewfunctions.Whatdoeseachgiveyou?

structPersonThisiswhereI’mcreatingastructurethathasfourelementstodescribeaperson.Thefinalresultisanewcompoundtypethatletsmereferencetheseelementsallasoneoreachpiecebyname.It’ssimilartoarowofadatabasetableoraclassinanobject-orientedprogramming(OOP)language.

functionPerson_createIneedawaytocreatethesestructures,soI’vemadeafunctiontodothat.Herearetheimportantthings:•IusemallocformemoryallocatetoasktheOStogivemeapieceofrawmemory.•Ipasstomallocthesizeof(structPerson),whichcalculatesthetotalsizeofthestructure,givenallofthefieldsinsideit.•IuseasserttomakesurethatIhaveavalidpieceofmemorybackfrommalloc.There’saspecialconstantcalledNULLthatyouusetomean“unsetorinvalidpointer.”Thisassertisbasicallycheckingthatmallocdidn’treturnaNULLinvalidpointer.•IinitializeeachfieldofstructPersonusingthex->ysyntax,tosaywhatpartofthestructureIwanttoset.•Iusethestrdupfunctiontoduplicatethestringforthename,justtomakesurethatthisstructureactuallyownsit.Thestrdupactuallyislikemalloc,anditalsocopiestheoriginalstringintothememoryitcreates.

functionPerson_destroyIfIhaveacreatefunction,thenIalwaysneedadestroyfunction,andthisiswhatdestroysPersonstructures.IagainuseasserttomakesureI’mnotgettingbadinput.ThenIusethefunctionfreetoreturnthememoryIgotwithmallocandstrdup.

Ifyoudon’tdothis,yougetamemoryleak.functionPerson_printIthenneedawaytoprintoutpeople,whichisallthisfunctiondoes.Itusesthesamex->ysyntaxtogetthefieldfromthestructuretoprintit.

functionmainInthemainfunction,IuseallofthepreviousfunctionsandthestructPersontodothefollowing:•Createtwopeople,joeandfrank.•Printthemout,butnoticeI’musingthe%pformatsoyoucanseewheretheprogramhasactuallyputyourstructureinmemory.•Agebothofthemby20yearswithchangestotheirbodies,too.•Printeachoneafteragingthem.•Finally,destroythestructuressowecancleanupcorrectly.

Gothroughthisdescriptioncarefully,anddothefollowing:•Lookupeveryfunctionandheaderfileyoudon’tknow.Rememberthatyoucanusuallydoman2functionorman3function,andit’lltellyouaboutit.Youcanalsosearchonlinefortheinformation.•WriteacommentaboveeachandeverysinglelinethatsayswhatthelinedoesinEnglish.•Tracethrougheachfunctioncallandvariablesoyouknowwhereitcomesfromintheprogram.•Lookupanysymbolsyoudon’tunderstand.

WhatYouShouldSeeAfteryouaugmenttheprogramwithyourdescriptioncomments,makesureitreallyrunsandproducesthisoutput:

Exercise16Session



$./ex16Joeisatmemorylocation0xeba010:

Name:JoeAlex

Age:32Height:64Weight:140Frankisatmemorylocation0xeba050:

Name:FrankBlank

Age:20Height:72Weight:180Name:JoeAlex

Age:52Height:62Weight:180Name:FrankBlank

Age:40Height:72

Weight:200

ExplainingStructuresIfyou’vedonethework,thenstructuresshouldbemakingsense,butletmeexplainthemexplicitlyjusttomakesureyou’veunderstoodit.AstructureinCisacollectionofotherdatatypes(variables)thatarestoredinoneblockofmemorywhereyoucanaccesseachvariableindependentlybyname.Theyaresimilartoarecordinadatabasetable,oraverysimplisticclassinanOOPlanguage.Wecanbreakonedownthisway:

•Intheabovecode,wemakeastructthathasfieldsforaperson:name,age,weight,andheight.•Eachofthosefieldshasatype,likeint.•Cthenpacksthosetogethersothattheycanallbecontainedinonesinglestruct.•ThestructPersonisnowacompounddatatype,whichmeansyoucanrefertostructPersonusingthesamekindsofexpressionsyouwouldforotherdatatypes.•Thisletsyoupassthewholecohesivegroupingtootherfunctions,asyoudidwithPerson_print.•Youcanthenaccesstheindividualpartsofastructbytheirnamesusingx->yifyou’redealingwithapointer.•There’salsoawaytomakeastructthatdoesn’tneedapointer,andyouusethex.y(period)syntaxtoworkwithit.We’lldothisintheExtraCreditsection.

Ifyoudidn’thavestruct,you’dneedtofigureoutthesize,packing,andlocationofpiecesofmemorywithcontentslikethis.Infact,inmostearlyAssemblercode(andevensomenow),thisiswhatyouwoulddo.InC,youcanletithandlethememorystructuringofthesecompounddatatypesandthenfocusonwhatyoudowiththem.

HowtoBreakItThewaysinwhichtobreakthisprograminvolvehowyouusethepointersandthemallocsystem:

•TrypassingNULLtoPerson_destroyseewhatitdoes.Ifitdoesn’tabort,thenyoumustnothavethe-goptioninyourMakefile'sCFLAGS.•ForgettocallPerson_destroyattheend,andthenrunitunderthedebuggertoseeitreportthatyouforgottofreethememory.Figureouttheoptionsyouneedtopasstothedebuggertogetittoprinthowyouleakedthismemory.•Forgettofreewho->nameinPerson_destroyandcomparetheoutput.Again,usetherightoptionstoseehowthedebuggertellsyouexactlywhereyoumessedup.•Thistime,passNULLtoPerson_printandseewhatthedebuggerthinksofthat.You’llfigureoutthatNULLisaquickwaytocrashyourprogram.

ExtraCreditInthispartoftheexercise,Iwantyoutoattemptsomethingdifficultfortheextracredit:Convertthisprogramtonotusepointersandmalloc.Thiswillbehard,soyou’llwanttoresearchthefollowing:

•Howtocreateastructonthestack,justlikeyou’remakinganyothervariable.

•Howtoinitializeitusingthex.y(period)characterinsteadofthex->ysyntax.•Howtopassastructuretootherfunctionswithoutusingapointer.

Exercise17.HeapandStackMemoryAllocation

Inthisexercise,you’regoingtomakeabigleapindifficultyandcreateanentiresmallprogramtomanageadatabase.Thisdatabaseisn’tveryefficientanddoesn’tstoreverymuch,butitdoesdemonstratemostofwhatyou’velearnedsofar.Italsointroducesmemoryallocationmoreformally,andgetsyoustartedworkingwithfiles.WeusesomefileI/Ofunctions,butIwon’tbeexplainingthemtoowellsothatyoucantrytofigurethemoutfirst.Asusual,typethiswholeprograminandgetitworking,thenwe’lldiscussit.

ex17.c


1#include<stdio.h>2#include<assert.h>3#include<stdlib.h>4#include<errno.h>5#include<string.h>67#defineMAX_DATA5128#defineMAX_ROWS100910structAddress{11intid;12intset;13charname[MAX_DATA];14charemail[MAX_DATA];15};1617structDatabase{18structAddressrows[MAX_ROWS];19};2021structConnection{22FILE*file;23structDatabase*db;24};2526voiddie(constchar*message)27{28if(errno){29perror(message);30}else{31printf("ERROR:%s\n",message);32}3334exit(1);35}3637voidAddress_print(structAddress*addr)38{39printf("%d%s%s\n",addr->id,addr->name,addr->email);40}4142voidDatabase_load(structConnection*conn)43{44intrc=fread(conn->db,sizeof(structDatabase),1,conn->file);45if(rc!=1)

46die("Failedtoloaddatabase.");47}4849structConnection*Database_open(constchar*filename,charmode)50{51structConnection*conn=malloc(sizeof(structConnection));52if(!conn)53die("Memoryerror");5455conn->db=malloc(sizeof(structDatabase));56if(!conn->db)57die("Memoryerror");5859if(mode=='c'){60conn->file=fopen(filename,"w");61}else{62conn->file=fopen(filename,"r+");6364if(conn->file){65Database_load(conn);66}67}6869if(!conn->file)70die("Failedtoopenthefile");7172returnconn;73}7475voidDatabase_close(structConnection*conn)76{77if(conn){78if(conn->file)79fclose(conn->file);80if(conn->db)81free(conn->db);82free(conn);83}84}8586voidDatabase_write(structConnection*conn)87{88rewind(conn->file);8990intrc=fwrite(conn->db,sizeof(structDatabase),1,conn->file);91if(rc!=1)92die("Failedtowritedatabase.");9394rc=fflush(conn->file);95if(rc==-1)96die("Cannotflushdatabase.");97}9899voidDatabase_create(structConnection*conn)100{101inti=0;102103for(i=0;i<MAX_ROWS;i++){104//makeaprototypetoinitializeit105structAddressaddr={.id=i,.set=0};106//thenjustassignit107conn->db->rows[i]=addr;108}109}

110111voidDatabase_set(structConnection*conn,intid,constchar*name,112constchar*email)113{114structAddress*addr=&conn->db->rows[id];115if(addr->set)116die("Alreadyset,deleteitfirst");117118addr->set=1;119//WARNING:bug,readthe"HowToBreakIt"andfixthis120char*res=strncpy(addr->name,name,MAX_DATA);121//demonstratethestrncpybug122if(!res)123die("Namecopyfailed");124125res=strncpy(addr->email,email,MAX_DATA);126if(!res)127die("Emailcopyfailed");128}129130voidDatabase_get(structConnection*conn,intid)131{132structAddress*addr=&conn->db->rows[id];133134if(addr->set){135Address_print(addr);136}else{137die("IDisnotset");138}139}140141voidDatabase_delete(structConnection*conn,intid)142{143structAddressaddr={.id=id,.set=0};144conn->db->rows[id]=addr;145}146147voidDatabase_list(structConnection*conn)148{149inti=0;150structDatabase*db=conn->db;151152for(i=0;i<MAX_ROWS;i++){153structAddress*cur=&db->rows[i];154155if(cur->set){156Address_print(cur);157}158}159}160161intmain(intargc,char*argv[])162{163if(argc<3)164die("USAGE:ex17<dbfile><action>[actionparams]");165166char*filename=argv[1];167charaction=argv[2][0];168structConnection*conn=Database_open(filename,action);169intid=0;170171if(argc>3)id=atoi(argv[3]);172if(id>=MAX_ROWS)die("There'snotthatmanyrecords.");

173174switch(action){175case'c':176Database_create(conn);177Database_write(conn);178break;179180case'g':181if(argc!=4)182die("Needanidtoget");183184Database_get(conn,id);185break;186187case's':188if(argc!=6)189die("Needid,name,emailtoset");190191Database_set(conn,id,argv[4],argv[5]);192Database_write(conn);193break;194195case'd':196if(argc!=4)197die("Needidtodelete");198199Database_delete(conn,id);200Database_write(conn);201break;202203case'l':204Database_list(conn);205break;206default:207die("Invalidaction:c=create,g=get,s=set,d=del,l=list");208}209210Database_close(conn);211212return0;213}

Inthisprogram,we’reusingasetofstructures,orstructs,tocreateasimpledatabaseforanaddressbook.Therearesomethingsyou’veneverseen,soyoushouldgothroughitlinebyline,explainwhateachlinedoes,andlookupanyfunctionsthatyoudon’trecognize.Thereareafewkeythingsthatyoushouldpayattentionto,aswell:

#defineforconstantsWeuseanotherpartoftheCpreprocessor(CPP)tocreateconstantsettingsofMAX_DATAandMAX_ROWS.I’llcovermoreofwhattheCPPdoeslater,butthisisawaytocreateaconstantthatwillworkreliably.Thereareotherways,buttheydon’tapplyincertainsituations.

FixedsizedstructsTheAddressstructthenusestheseconstantstocreateapieceofdatathatisfixedinsize,makingitlessefficientbuteasiertostoreandread.TheDatabasestructisthenalsoafixedsizebecauseit’safixedlengtharrayofAddressstructs.Thatletsyouwritethewholethingtodiskinonemovelater.

diefunctiontoabortwithanerrorInasmallprogramlikethis,youcanmakeasinglefunctionthatkillstheprogramwithanerrorifthere’sanythingwrong.Icallthisdie,andit’susedtoexittheprogramwithanerrorafteranyfailedfunctioncallsorbadinputs.

errnoandperror()forerrorreportingWhenyouhaveanerrorreturnfromafunction,itwillusuallysetanexternalvariablecallederrnotosayexactlywhathappened.Thesearejustnumbers,soyoucanuseperrortoprinttheerrormessage.

FILEfunctionsI’musingallnewfunctionslikefopen,fread,fclose,andrewindtoworkwithfiles.EachofthesefunctionsworksonaFILEstructthat’sjustlikeyourotherstructs,butit’sdefinedbytheCstandardlibrary.

nestedstructpointersThere’sausefornestedstructuresandgettingtheaddressofarrayelementsthatyoushouldstudy.Specifically,codelike&conn->db->rows[i]thatreads“gettheielementofrows,whichisindb,whichisinconn,thengettheaddressof(&)it.”

copyingstructprototypesBestshowninDatabase_delete,youcanseeI’musingatemporarylocalAddress,initializingitsidandsetfields,andthensimplycopyingitintotherowsarraybyassigningittotheelementIwant.Thistrickmakessurethatallfieldsexceptsetandidareinitializedtozerosandit’sactuallyeasiertowrite.Incidentally,youshouldn’tbeusingmemcpytodothesekindsofstructcopyingoperations.ModernCallowsyoutosimplyassignonestructtoanotherandit’llhandlethecopyingforyou.

processingcomplexargumentsI’mdoingsomemorecomplexargumentparsing,butthisisn’treallythebestwaytodoit.We’llgetintoabetteroptionforparsinglaterinthebook.

convertingstringstointsIusetheatoifunctiontotakethestringfortheidonthecommandlineandconvertittotheintidvariable.Readuponthisandsimilarfunctions.

allocatinglargedataontheheapThewholepointofthisprogramisthatI’musingmalloctoasktheOSforalargeamountofmemorywhenIcreatetheDatabase.We’llcoverthisinmoredetaillater.

NULLis0,soBooleanworksInmanyofthechecks,I’mtestingthatapointerisnotNULLbysimplydoingif(!ptr)die("fail!"),becauseNULLwillevaluatetofalse.Youcouldbeexplicitandsayif(ptr==NULL)die("fail!"),aswell.Insomeraresystems,NULLwillbestoredinthecomputer(represented)assomethingnot0,buttheCstandardsaysyoushouldstillbeabletowritecodeasifithasa0value.FromnowonwhenIsay“NULLis0,”Imeanitsvalueforanyonewhoisoverlypedantic.

WhatYouShouldSeeYoushouldspendasmuchtimeasyoucantestingthatitworks,andrunningitwithadebuggertoconfirmthatyou’vegotallofthememoryusageright.Here’sasessionofmetestingitnormally,andthenusingthedebuggertochecktheoperations:

Exercise17Session



$./ex17db.datc$./[email protected]$./[email protected]$./[email protected]$$./ex17db.datl

[email protected]

[email protected]

[email protected]

$./ex17db.datd3$./[email protected]

[email protected]

$./[email protected]

HeapversusStackAllocationYoukidshaveitgreatthesedays.YouplaywithyourRubyorPythonandjustmakeobjectsandvariableswithoutanycareforwheretheylive.Youdon’tcareifit’sonthestack,andwhataboutontheheap?Fuggedaboutit.Youdon’tevenknow,andyouknowwhat,chancesareyourlanguageofchoicedoesn’tevenputthevariablesonstackatall.It’sallheap,andyoudon’tevenknowifitis.Cisdifferentbecauseit’susingtherealCPU’sactualmachinerytodoitswork,andthatinvolvesachunkofRAMcalledthestackandanothercalledtheheap.What’sthedifference?Italldependsonwhereyougetthestorage.Theheapiseasiertoexplainsinceit’sjustalltheremainingmemoryinyourcomputer,andyouaccessitwiththefunctionmalloctogetmore.Eachtimeyoucallmalloc,theOSusesinternalfunctionstoregisterthatpieceofmemorytoyou,andthenreturnsapointertoit.Whenyou’redonewithit,youusefreetoreturnittotheOSsothatitcanbeusedbyotherprograms.Failingtodothiswillcauseyourprogramtoleakmemory,butValgrindwillhelpyoutracktheseleaksdown.Thestackisaspecialregionofmemorythatstorestemporaryvariables,whicheachfunctioncreatesaslocalstothatfunction.Howitworksisthateachargumenttoafunctionispushedontothestackandthenusedinsidethefunction.It’sreallyastackdatastructure,sothelastthinginisthefirstthingout.Thisalsohappenswithalllocalvariableslikecharactionandintidinmain.Theadvantageofusingastackforthisissimplythatwhenthefunctionexits,theCcompilerpopsthesevariablesoffofthestacktocleanup.Thisissimpleandpreventsmemoryleaksifthevariableisonthestack.Theeasiestwaytokeepthisstraightiswiththismantra:Ifyoudidn’tgetitfrommalloc,orafunctionthatgotitfrommalloc,thenit’sonthestack.Therearethreeprimaryproblemswithstacksandheapstowatchoutfor:

•Ifyougetablockofmemoryfrommalloc,andhavethatpointeronthestack,thenwhenthefunctionexitsthepointerwillgetpoppedoffandlost.•Ifyouputtoomuchdataonthestack(likelargestructsandarrays),thenyoucancauseastackoverflowandtheprogramwillabort.Inthiscase,usetheheapwithmalloc.•Ifyoutakeapointertosomethingonthestack,andthenpassorreturnitfromyourfunction,thenthefunctionreceivingitwillsegmentationfault(segfault),becausetheactualdatawillgetpoppedoffanddisappear.You’llbepointingatdeadspace.

ThisiswhyIcreatedaDatabase_openthatallocatesmemoryordies,andthenaDatabase_closethatfreeseverything.Ifyoucreateacreatefunctionthatmakesthewholethingornothing,andthenadestroyfunctionthatsafelycleansupeverything,thenit’seasiertokeepitallstraight.Finally,whenaprogramexits,theOSwillcleanupalloftheresourcesforyou,butsometimesnotimmediately.Acommonidiom(andoneIuseinthisexercise)istojustabortandlettheOScleanup

http://[email protected]






onerror.

HowtoBreakItThisprogramhasalotofplaceswhereyoucanbreakit,sotrysomeofthesebutalsocomeupwithyourown:

•Theclassicwayistoremovesomeofthesafetycheckssothatyoucanpassinarbitrarydata.Forexample,removethecheckonline160thatpreventsyoufrompassinginanyrecordnumber.•Youcanalsotrycorruptingthedatafile.Openitinanyeditorandchangerandombytes,andthencloseit.•Youcouldalsofindwaystopassbadargumentstotheprogramwhenit’srun.Forexample,gettingthefileandactionbackwardwillmakeitcreateafilenamedaftertheaction,andthendoanactionbasedonthefirstcharacter.•There’sabuginthisprogrambecausestrncpyispoorlydesigned.Goreadaboutstrncpyandtrytofindoutwhathappenswhenthenameoraddressyougiveisgreaterthan512bytes.Fixthisbysimplyforcingthelastcharacterto'\0'sothatit’salwayssetnomatterwhat(whichiswhatstrncpyshoulddo).•IntheExtraCreditsection,Ihaveyouaugmenttheprogramtocreatearbitrarysizedatabases.Trytoseewhatthebiggestdatabaseisbeforeyoucausetheprogramtodieduetolackofmemoryfrommalloc.

ExtraCredit•Thediefunctionneedstobeaugmentedtoletyoupasstheconnvariable,soitcancloseitandcleanup.•ChangethecodetoacceptparametersforMAX_DATAandMAX_ROWS,storethemintheDatabasestruct,andwritethattothefile,thuscreatingadatabasethatcanbearbitrarilysized.•Addmoreoperationsyoucandowiththedatabase,likefind.•ReadabouthowCdoesit’sstructpacking,andthentrytoseewhyyourfileisthesizeitis.Seeifyoucancalculateanewsizeafteraddingmorefields.•AddsomemorefieldstoAddressandmakethemsearchable.•Writeashellscriptthatwilldoyourtestingautomaticallyforyoubyrunningcommandsintherightorder.Hint:Useset-eatthetopofabashtomakeitabortthewholescriptifanycommandhasanerror.•Tryreworkingtheprogramtouseasingleglobalforthedatabaseconnection.Howdoesthisnewversionoftheprogramcomparetotheotherone?•Goresearchstackdatastructureandwriteoneinyourfavoritelanguage,thentrytodoitinC.

Exercise18.PointerstoFunctions

FunctionsinCareactuallyjustpointerstoaspotintheprogramwheresomecodeexists.Justlikeyou’vebeencreatingpointerstostructs,strings,andarrays,youcanpointapointeratafunction,too.Themainuseforthisistopasscallbackstootherfunctions,ortosimulateclassesandobjects.Inthisexercise,we’lldosomecallbacks,andinthenextexercise,we’llmakeasimpleobjectsystem.Theformatofafunctionpointerlookslikethis:Clickheretoviewcodeimage

int(*POINTER_NAME)(inta,intb)

Awaytorememberhowtowriteoneistodothis:•Writeanormalfunctiondeclaration:intcallme(inta,intb)•Wrapthefunctionnamewiththepointersyntax:int(*callme)(inta,intb)•Changethenametothepointername:int(*compare_cb)(inta,intb)

Thekeythingtorememberisthatwhenyou’redonewiththis,thevariablenameforthepointeriscalledcompare_cbandyouuseitjustlikeit’safunction.Thisissimilartohowpointerstoarrayscanbeusedjustlikethearraystheypointto.Pointerstofunctionscanbeusedlikethefunctionstheypointtobutwithadifferentname.Clickheretoviewcodeimage

int(*tester)(inta,intb)=sorted_order;printf("TEST:%dissameas%d\n",tester(2,3),sorted_order(2,3));

Thiswillworkevenifthefunctionpointerreturnsapointertosomething:•Writeit:char*make_coolness(intawesome_levels)•Wrapit:char*(*make_coolness)(intawesome_levels)•Renameit:char*(*coolness_cb)(intawesome_levels)

Thenextproblemtosolvewithusingfunctionpointersisthatit’shardtogivethemasparameterstoafunction,suchaswhenyouwanttopassthefunctioncallbacktoanotherfunction.Thesolutionistousetypedef,whichisaCkeywordformakingnewnamesforother,morecomplextypes.Theonlythingyouneedtodoisputtypedefbeforethesamefunctionpointersyntax,andthenafterthatyoucanusethenamelikeit’satype.Idemonstratethisinthefollowingexercisecode:

ex18.c


1#include<stdio.h>2#include<stdlib.h>3#include<errno.h>4#include<string.h>56/**Ouroldfrienddiefromex17.*/7voiddie(constchar*message)8{9if(errno){10perror(message);

11}else{12printf("ERROR:%s\n",message);13}1415exit(1);16}1718//atypedefcreatesafaketype,inthis19//caseforafunctionpointer20typedefint(*compare_cb)(inta,intb);2122/**23*Aclassicbubblesortfunctionthatusesthe24*compare_cbtodothesorting.25*/26int*bubble_sort(int*numbers,intcount,compare_cbcmp)27{28inttemp=0;29inti=0;30intj=0;31int*target=malloc(count*sizeof(int));3233if(!target)34die("Memoryerror.");3536memcpy(target,numbers,count*sizeof(int));3738for(i=0;i<count;i++){39for(j=0;j<count-1;j++){40if(cmp(target[j],target[j+1])>0){41temp=target[j+1];42target[j+1]=target[j];43target[j]=temp;44}45}46}4748returntarget;49}5051intsorted_order(inta,intb)52{53returna-b;54}5556intreverse_order(inta,intb)57{58returnb-a;59}6061intstrange_order(inta,intb)62{63if(a==0||b==0){64return0;65}else{66returna%b;67}68}6970/**71*Usedtotestthatwearesortingthingscorrectly72*bydoingthesortandprintingitout.73*/

74voidtest_sorting(int*numbers,intcount,compare_cbcmp)75{76inti=0;77int*sorted=bubble_sort(numbers,count,cmp);7879if(!sorted)80die("Failedtosortasrequested.");8182for(i=0;i<count;i++){83printf("%d",sorted[i]);84}85printf("\n");8687free(sorted);88}8990intmain(intargc,char*argv[])91{92if(argc<2)die("USAGE:ex1843156");9394intcount=argc-1;95inti=0;96char**inputs=argv+1;9798int*numbers=malloc(count*sizeof(int));99if(!numbers)die("Memoryerror.");100101for(i=0;i<count;i++){102numbers[i]=atoi(inputs[i]);103}104105test_sorting(numbers,count,sorted_order);106test_sorting(numbers,count,reverse_order);107test_sorting(numbers,count,strange_order);108109free(numbers);110111return0;112}

Inthisprogram,you’recreatingadynamicsortingalgorithmthatcansortanarrayofintegersusingacomparisoncallback.Here’sthebreakdownofthisprogram,soyoucanclearlyunderstandit:

ex18.c:1-6Theusualincludesthatareneededforallofthefunctionsthatwecall.ex18.c:7-17ThisisthediefunctionfromthepreviousexercisethatI’llusetodoerrorchecking.ex18.c:21Thisiswherethetypedefisused,andlaterIusecompare_cblikeit’satypesimilartointorcharinbubble_sortandtest_sorting.

ex18.c:27-49Abubblesortimplementation,whichisaveryinefficientwaytosortsomeintegers.Here’sabreakdown:ex18.c:27Iusethetypedefforcompare_cbasthelastparametercmp.Thisisnowafunctionthatwillreturnacomparisonbetweentwointegersforsorting.

ex18.c:29-34Theusualcreationofvariablesonthestack,followedbyanewarrayofintegersontheheapusingmalloc.Makesureyouunderstandwhatcount*sizeof(int)isdoing.

ex18.c:38Theouterloopofthebubblesort.ex18.c:39Theinnerloopofthebubblesort.

ex18.c:40NowIcallthecmpcallbackjustlikeit’sanormalfunction,butinsteadofbeingthenameofsomethingthatwedefined,it’sjustapointertoit.Thisletsthecallerpassinanythingitwantsaslongasitmatchesthesignatureofthecompare_cbtypedef.

ex18.c:41-43Theactualswappingoperationwhereabubblesortneedstodowhatitdoes.ex18.c:48Finally,thisreturnsthenewlycreatedandsortedresultarraytarget.

ex18.c:51-68Threedifferentversionsofthecompare_cbfunctiontype,whichneedstohavethesamedefinitionasthetypedefthatwecreated.TheCcompilerwillcomplaintoyouifyougetthiswrongandsaythetypesdon’tmatch.

ex18.c:74-87Thisisatesterforthebubble_sortfunction.YoucanseenowhowI’malsousingcompare_cbtopasstobubble_sort,demonstratinghowthesecanbepassedaroundlikeanyotherpointers.

ex18.c:90-103Asimplemainfunctionthatsetsupanarraybasedonintegerstopassonthecommandline,andthenitcallsthetest_sortingfunction.

ex18.c:105-107Finally,yougettoseehowthecompare_cbfunctionpointertypedefisused.Isimplycalltest_sortingbutgiveitthenameofsorted_order,reverse_order,andstrange_orderasthefunctiontouse.TheCcompilerthenfindstheaddressofthosefunctions,andmakesitapointerfortest_sortingtouse.Ifyoulookattest_sorting,you’llseethatitthenpasseseachofthesetobubble_sort,butitactuallyhasnoideawhattheydo.Thecompileronlyknowsthattheymatchthecompare_cbprototypeandshouldwork.

ex18.c:109Lastthingwedoisfreeupthearrayofnumbersthatwemade.

WhatYouShouldSeeRunningthisprogramissimple,butyoushouldtrydifferentcombinationsofnumbers,orevenothercharacters,toseewhatitdoes.

Exercise18Session



$./ex1841732080123478

8743210

3427108

$

HowtoBreakItI’mgoingtohaveyoudosomethingkindofweirdtobreakthis.Thesefunctionpointersarelikeeveryotherpointer,sotheypointatblocksofmemory.Chasthisabilitytotakeonepointerandconvertittoanothersoyoucanprocessthedataindifferentways.It’susuallynotnecessary,buttoshowyouhowtohackyourcomputer,Iwantyoutoaddthisattheendoftest_sorting:Clickheretoviewcodeimage

unsignedchar*data=(unsignedchar*)cmp;

for(i=0;i<25;i++){printf("%02x:",data[i]);}

printf("\n");

Thisloopissortoflikeconvertingyourfunctiontoastring,andthenprintingoutitscontents.Thiswon’tbreakyourprogramunlesstheCPUandOSyou’reonhasaproblemwithyoudoingthis.Whatyou’llseeafteritprintsthesortedarrayisastringofhexadecimalnumbers,likethis:Clickheretoviewcodeimage

55:48:89:e5:89:7d:fc:89:75:f8:8b:55:fc:8b:45:

Thatshouldbetherawassemblerbytecodeofthefunctionitself,andyoushouldseethattheystartthesamebutthenhavedifferentendings.It’salsopossiblethatthisloopisn’tgettingallofthefunction,orit’sgettingtoomuchandstompingonanotherpieceoftheprogram.Withoutmoreanalysisyouwon’tknow.

ExtraCredit•Getahexeditorandopenupex18,andthenfindthesequenceofhexdigitsthatstartafunctiontoseeifyoucanfindthefunctionintherawprogram.•Findotherrandomthingsinyourhexeditorandchangethem.Rerunyourprogramandseewhathappens.Stringsyoufindaretheeasiestthingstochange.•Passinthewrongfunctionforthecompare_cbandseewhattheCcompilercomplainsabout.•PassinNULLandwatchyourprogramseriouslybiteit.Then,runthedebuggerandseewhatthatreports.•Writeanothersortingalgorithm,thenchangetest_sortingsothatittakesbothanarbitrarysortfunctionandthesortfunction’scallbackcomparison.Useittotestbothofyouralgorithms.

Exercise19.Zed’sAwesomeDebugMacros

There’sareoccurringprobleminCthatwe’vebeendancingaround,butI’mgoingtosolveitinthisexerciseusingasetofmacrosIdeveloped.Youcanthankmelaterwhenyourealizehowinsanelyawesomethesemacrosare.Rightnow,youdon’tknowhowawesometheyare,soyou’lljusthavetousethem,andthenyoucanwalkuptomeonedayandsay,“Zed,thosedebugmacroswerethebomb.Ioweyoumyfirstbornchildbecauseyousavedmeadecadeofheartacheandpreventedmefromkillingmyselfmorethanonce.Thankyou,goodsir,here’samilliondollarsandtheoriginalSnakeheadTelecasterprototypesignedbyLeoFender.”Yes,theyarethatawesome.

TheCError-HandlingProblemHandlingerrorsisadifficultactivityinalmosteveryprogramminglanguage.Thereareentireprogramminglanguagesthattryashardastheycantoavoideventheconceptofanerror.Otherlanguagesinventcomplexcontrolstructureslikeexceptionstopasserrorconditionsaround.Theproblemexistsmostlybecauseprogrammersassumeerrorsdon’thappen,andthisoptimisminfectsthetypesoflanguagestheyuseandcreate.Ctacklestheproblembyreturningerrorcodesandsettingaglobalerrnovaluethatyoucheck.Thismakesforcomplexcodethatsimplyexiststocheckifsomethingyoudidhadanerror.AsyouwritemoreandmoreCcode,you’llwritecodewiththispattern:

•Callafunction.•Checkifthereturnvalueisanerror(anditmustlookthatupeachtime,too).•Then,cleanupalltheresourcescreatedsofar.•Lastly,printoutanerrormessagethathopefullyhelps.

Thismeansforeveryfunctioncall(andyes,everyfunction),youarepotentiallywritingthreeorfourmorelinesjusttomakesureitworked.Thatdoesn’tincludetheproblemofcleaningupallofthejunkyou’vebuilttothatpoint.Ifyouhavetendifferentstructures,threefiles,andadatabaseconnection,you’dhave14morelineswhenyougetanerror.Inthepast,thiswasn’taproblembecauseCprogramsdidwhatyou’vebeendoingwhentherewasanerror:die.NopointinbotheringwithcleanupwhentheOSwilldoitforyou.Today,though,manyCprogramsneedtorunforweeks,months,oryears,andhandleerrorsfrommanydifferentsourcesgracefully.Youcan’tjusthaveyourWebserverdieattheslightesttouch,andyoudefinitelycan’thavealibrarythatyou’vewrittennuketheprogramit’susedin.That’sjustrude.Otherlanguagessolvethisproblemwithexceptions,butthosehaveproblemsinC(andinotherlanguages,too).InC,youonlyhaveonereturnvalue,butexceptionsmakeupanentirestack-basedreturnsystemwitharbitraryvalues.TryingtomarshalexceptionsupthestackinCisdifficult,andnootherlibrarieswillunderstandit.

TheDebugMacrosThesolutionI’vebeenusingforyearsisasmallsetofdebugmacrosthatimplementsabasicdebugginganderror-handlingsystemforC.Thissystemiseasytounderstand,workswitheverylibrary,andmakesCcodemoresolidandclearer.Itdoesthisbyadoptingtheconventionthatwheneverthere’sanerror,yourfunctionwilljumptoan

error:partofthefunctionthatknowshowtocleanupeverythingandreturnanerrorcode.Youcanuseamacrocalledchecktocheckreturncodes,printanerrormessage,andthenjumptothecleanupsection.Youcancombinethatwithasetofloggingfunctionsforprintingoutusefuldebugmessages.I’llnowshowyoutheentirecontentsofthemostawesomesetofbrillianceyou’veeverseen.

dbg.h


#ifndef__dbg_h__

#define__dbg_h__

#include<stdio.h>

#include<errno.h>

#include<string.h>

#ifdefNDEBUG

#definedebug(M,...)

#else

#definedebug(M,...)fprintf(stderr,"DEBUG%s:%d:"M"\n",\

__FILE__,__LINE__,##__VA_ARGS__)#endif

#defineclean_errno()(errno==0?"None":strerror(errno))

#definelog_err(M,...)fprintf(stderr,\

"[ERROR](%s:%d:errno:%s)"M"\n",__FILE__,__LINE__,\clean_errno(),##__VA_ARGS__)

#definelog_warn(M,...)fprintf(stderr,\

"[WARN](%s:%d:errno:%s)"M"\n",\__FILE__,__LINE__,clean_errno(),##__VA_ARGS__)

#definelog_info(M,...)fprintf(stderr,"[INFO](%s:%d)"M"\n",\

__FILE__,__LINE__,##__VA_ARGS__)

#definecheck(A,M,...)if(!(A)){\

log_err(M,##__VA_ARGS__);errno=0;gotoerror;}

#definesentinel(M,...){log_err(M,##__VA_ARGS__);\

errno=0;gotoerror;}

#definecheck_mem(A)check((A),"Outofmemory.")

#definecheck_debug(A,M,...)if(!(A)){debug(M,##__VA_ARGS__);\

errno=0;gotoerror;}

#endif

Yes,that’sit,andhere’sabreakdownofeveryline:dbg.h:1-2Theusualdefenseagainstaccidentallyincludingthefiletwice,whichyousawinthelastexercise.

dbg.h:4-6Includesforthefunctionsthatthesemacrosneed.dbg.h:8Thestartofa#ifdefthatletsyourecompileyourprogramsothatallofthedebuglogmessagesareremoved.

dbg.h:9IfyoucompilewithNDEBUGdefined,then“nodebug”messageswillremain.Youcanseeinthiscasethe#definedebug()isjustreplacedwithnothing(therightsideisempty).

dbg.h:10Thematching#elsefortheabove#ifdef.dbg.h:11Thealternative#definedebugthattranslatesanyuseofdebug("format",arg1,arg2)intoanfprintfcalltostderr.ManyCprogrammersdon’tknowthis,butyoucancreatemacrosthatactuallyworklikeprintfandtakevariablearguments.SomeCcompilers(actuallyCPP)don’tsupportthis,buttheonesthatmatterdo.Themagichereistheuseof##__VA_ARGS__thatsays“putwhatevertheyhadforextraarguments(...)here.”Alsonoticetheuseof__FILE__and__LINE__togetthecurrentfile:lineforthedebugmessage.Veryhelpful.

dbg.h:12Theendofthe#ifdef.dbg.h:14Theclean_errnomacrothat’susedintheotherstogetasafe,readableversionoferrno.Thatstrangesyntaxinthemiddleisaternaryoperatorandyou’lllearnwhatitdoeslater.

dbg.h:16-20Thelog_err,log_warn,andlog_info,macrosforloggingmessagesthataremeantfortheenduser.Theyworklikedebugbutcan’tbecompiledout.

dbg.h:22Thebestmacroever,check,willmakesuretheconditionAistrue,andifnot,itlogstheerrorM(withvariableargumentsforlog_err),andthenjumpstothefunction’serror:forcleanup.

dbg.h:24Thesecondbestmacroever,sentinel,isplacedinanypartofafunctionthatshouldn’trun,andifitdoes,itprintsanerrormessageandthenjumpstotheerror:label.Youputthisinif-statementsandswitch-statementstocatchconditionsthatshouldn’thappen,likethedefault:.

dbg.h:26Ashorthandmacrocalledcheck_memthatmakessureapointerisvalid,andifitisn’t,itreportsitasanerrorwith“Outofmemory.”

dbg.h:28Analternativemacro,check_debug,whichstillchecksandhandlesanerror,butiftheerroriscommon,thenitdoesn’tbotherreportingit.Inthisone,itwillusedebuginsteadoflog_errtoreportthemessage.SowhenyoudefineNDEBUG,thecheckstillhappens,andtheerrorjumpgoesoff,butthemessageisn’tprinted.

Usingdbg.hHere’sanexampleofusingallofdbg.hinasmallprogram.Thisdoesn’tactuallydoanythingbutdemonstratehowtouseeachmacro.However,we’llbeusingthesemacrosinalloftheprogramswewritefromnowon,sobesuretounderstandhowtousethem.

ex19.c


1#include"dbg.h"2#include<stdlib.h>3#include<stdio.h>45voidtest_debug()

6{7//noticeyoudon'tneedthe\n8debug("IhaveBrownHair.");910//passinginargumentslikeprintf11debug("Iam%dyearsold.",37);12}1314voidtest_log_err()15{16log_err("Ibelieveeverythingisbroken.");17log_err("Thereare%dproblemsin%s.",0,"space");18}1920voidtest_log_warn()21{22log_warn("Youcansafelyignorethis.");23log_warn("Maybeconsiderlookingat:%s.","/etc/passwd");24}2526voidtest_log_info()27{28log_info("WellIdidsomethingmundane.");29log_info("Ithappened%ftimestoday.",1.3f);30}3132inttest_check(char*file_name)33{34FILE*input=NULL;35char*block=NULL;3637block=malloc(100);38check_mem(block);//shouldwork3940input=fopen(file_name,"r");41check(input,"Failedtoopen%s.",file_name);4243free(block);44fclose(input);45return0;4647error:48if(block)free(block);49if(input)fclose(input);50return-1;51}5253inttest_sentinel(intcode)54{55char*temp=malloc(100);56check_mem(temp);5758switch(code){59case1:60log_info("Itworked.");61break;62default:63sentinel("Ishouldn'trun.");64}6566free(temp);67return0;6869error:

70if(temp)71free(temp);72return-1;73}7475inttest_check_mem()76{77char*test=NULL;78check_mem(test);7980free(test);81return1;8283error:84return-1;85}8687inttest_check_debug()88{89inti=0;90check_debug(i!=0,"Oops,Iwas0.");9192return0;93error:94return-1;95}9697intmain(intargc,char*argv[])98{99check(argc==2,"Needanargument.");100101test_debug();102test_log_err();103test_log_warn();104test_log_info();105106check(test_check("ex19.c")==0,"failedwithex19.c");107check(test_check(argv[1])==-1,"failedwithargv");108check(test_sentinel(1)==0,"test_sentinelfailed.");109check(test_sentinel(100)==-1,"test_sentinelfailed.");110check(test_check_mem()==-1,"test_check_memfailed.");111check(test_check_debug()==-1,"test_check_debugfailed.");112113return0;114115error:116return1;117}

Payattentiontohowcheckisused,andwhenit’sfalse,itjumpstotheerror:labeltodoacleanup.Thewaytoreadthoselinesis,“checkthatAistrue,andifnot,sayMandjumpout.”

WhatYouShouldSeeWhenyourunthis,giveitsomebogusfirstparametertoseethis:

Exercise19Session


$makeex19

cc-Wall-g-DNDEBUGex19.c-oex19

$./ex19test[ERROR](ex19.c:16:errno:None)Ibelieveeverythingisbroken.

[ERROR](ex19.c:17:errno:None)Thereare0problemsinspace.

[WARN](ex19.c:22:errno:None)Youcansafelyignorethis.

[WARN](ex19.c:23:errno:None)Maybeconsiderlookingat:/etc/passwd.

[INFO](ex19.c:28)WellIdidsomethingmundane.

[INFO](ex19.c:29)Ithappened1.300000timestoday.

[ERROR](ex19.c:38:errno:Nosuchfileordirectory)Failedtoopentest.

[INFO](ex19.c:57)Itworked.

[ERROR](ex19.c:60:errno:None)Ishouldn'trun.

[ERROR](ex19.c:74:errno:None)Outofmemory.

Seehowitreportstheexactlinenumberwherethecheckfailed?That’sgoingtosaveyouhoursofdebugginglater.Also,seehowitprintstheerrormessageforyouwhenerrnoisset?Again,thatwillsaveyouhoursofdebugging.

HowtheCPPExpandsMacrosIt’snowtimeforyoutogetashortintroductiontotheCPPsothatyouknowhowthesemacrosactuallywork.Todothis,I’mgoingtobreakdownthemostcomplexmacrofromdbg.h,andhaveyouruncppsoyoucanseewhatit’sactuallydoing.ImaginethatIhaveafunctioncalleddosomething()thatreturnsthetypical0forsuccessand-1foranerror.EverytimeIcalldosomething,Ihavetocheckforthiserrorcode,soI’dwritecodelikethis:Clickheretoviewcodeimage

intrc=dosomething();

if(rc!=0){fprintf(stderr,"Therewasanerror:%s\n",strerror());gotoerror;}

WhatIwanttousetheCPPforistoencapsulatethisif-statementintoamorereadableandmemorablelineofcode.Iwantwhatyou’vebeendoingindbg.hwiththecheckmacro:Clickheretoviewcodeimage

intrc=dosomething();check(rc==0,"Therewasanerror.");

Thisismuchclearerandexplainsexactlywhat’sgoingon:Checkthatthefunctionworked,andifnot,reportanerror.Todothis,weneedsomespecialCPPtricksthatmaketheCPPusefulasacodegenerationtool.Takealookatthecheckandlog_errmacrosagain:Clickheretoviewcodeimage

#definelog_err(M,...)fprintf(stderr,\"[ERROR](%s:%d:errno:%s)"M"\n",__FILE__,__LINE__,\clean_errno(),##__VA_ARGS__)#definecheck(A,M,...)if(!(A)){\log_err(M,##__VA_ARGS__);errno=0;gotoerror;}

Thefirstmacro,log_err,issimpler.Itsimplyreplacesitselfwithacalltofprintftostderr.Theonlytrickypartofthismacroistheuseof...inthedefinitionlog_err(M,...).Whatthisdoesisletyoupassvariableargumentstothemacro,soyoucanpassintheargumentsthatshouldgo

tofprintf.Howdotheygetinjectedintothefprintfcall?Lookattheendforthe##__VA_ARGS__,whichistellingtheCPPtotaketheargsenteredwherethe...is,andinjectthematthatpartofthefprintfcall.Youcanthendothingslikethis:Clickheretoviewcodeimage

log_err("Age:%d,name:%s",age,name);

Theargumentsage,namearethe...partofthedefinition,andthosegetinjectedintothefprintfoutput:Clickheretoviewcodeimage

fprintf(stderr,"[ERROR](%s:%d:errno:%s)Age%d:name%d\n",__FILE__,__LINE__,clean_errno(),age,name);

Seetheage,nameattheend?That’show...and##__VA_ARGS__worktogether,whichwillworkinmacrosthatcallothervariableargumentmacros.Lookatthecheckmacronowandseethatitcallslog_err,butcheckisalsousingthe...and##__VA_ARGS__todothecall.That’showyoucanpassfullprintfstyleformatstringstocheck,whichgotolog_err,andthenmakebothworklikeprintf.Thenextthingtostudyishowcheckcraftstheif-statementfortheerrorchecking.Ifwestripoutthelog_errusage,weseethis:Clickheretoviewcodeimage

if(!(A)){errno=0;gotoerror;}

Whichmeans:IfAisfalse,thenclearerrnoandgototheerrorlabel.Thecheckmacroisbeingreplacedwiththeif-statement,soifwemanuallyexpandoutthemacrocheck(rc==0,"Therewasanerror."),wegetthis:Clickheretoviewcodeimage

if(!(rc==0)){log_err("Therewasanerror.");errno=0;gotoerror;}

WhatyoushouldbegettingfromthistripthroughthesetwomacrosisthattheCPPreplacesmacroswiththeexpandedversionoftheirdefinition,anditwilldothisrecursively,expandingallofthemacrosinmacros.TheCPP,then,isjustarecursivetemplatingsystem,asImentionedbefore.Itspowercomesfromitsabilitytogeneratewholeblocksofparameterizedcode,thusbecomingahandycodegenerationtool.Thatleavesonequestion:Whynotjustuseafunctionlikedie?Thereasonisthatyouwantfile:linenumbersandthegotooperationforanerrorhandlingexit.Ifyoudidthisinsideafunction,youwouldn’tgetalinenumberwheretheerroractuallyhappened,andthegotowouldbemuchmorecomplicated.Anotherreasonisthatyoustillhavetowritetherawif-statement,whichlookslikealloftheotherif-statementsinyourcode,soit’snotasclearthatthisoneisanerrorcheck.Bywrappingtheif-statementinamacrocalledcheck,youmakeitclearthatthisisjusterrorchecking,andnotpartofthemainflow.Finally,CPPhastheabilitytoconditionallycompileportionsofcode,soyoucanhavecodethat’s

onlypresentwhenyoubuildadeveloperordebugversionoftheprogram.Youcanseethisalreadyinthedbg.hfilewherethedebugmacroonlyhasabodyifthecompilerasksforit.Withoutthisability,you’dneedawastedif-statementthatchecksfordebugmode,andthenwastesCPUcapacitydoingthatcheckfornovalue.

ExtraCredit•Put#defineNDEBUGatthetopofthefileandcheckthatallofthedebugmessagesgoaway.•Undothatline,andadd-DNDEBUGtoCFLAGSatthetopoftheMakefile,andthenrecompiletoseethesamething.•Modifytheloggingsothatitincludesthefunctionname,aswellasthefile:line.

Exercise20.AdvancedDebuggingTechniques

I’vealreadytaughtyouaboutmyawesomedebugmacros,andyou’vebeenusingthem.WhenIdebugcodeIusethedebug()macroalmostexclusivelytoanalyzewhat’sgoingonandtrackdowntheproblem.Inthisexercise,I’mgoingtoteachyouthebasicsofusingGDBtoinspectasimpleprogramthatrunsanddoesn’texit.You’lllearnhowtouseGDBtoattachtoarunningprocess,stopit,andseewhat’shappening.Afterthat,I’llgiveyousomelittletipsandtricksthatyoucanusewithGDB.Thisisanothervideo-focusedexercisewhereIshowyouadvanceddebuggingtrickswithmytechnique.Thediscussionbelowreinforcesthevideo,sowatchthevideofirst.Debuggingwillbemucheasiertolearnbywatchingmedoitfirst.

DebugPrintingversusGDBIapproachdebuggingprimarilywitha“scientificmethod”style:Icomeupwithpossiblecausesandthenrulethemoutorprovethattheycausethedefect.Theproblemmanyprogrammershavewiththisapproachisthattheyfeellikeitwillslowthemdown.Theypanicandrushtosolvethebug,butintheirrushtheyfailtonoticethatthey’rereallyjustflailingaroundandgatheringnousefulinformation.Ifindthatlogging(debugprinting)forcesmetosolveabugscientifically,andit’salsojusteasiertogatherinformationinmostsituations.Inaddition,Ihavethesereasonsforusingdebugprintingasmyprimarydebuggingtool:

•Youseeanentiretracingofaprogram’sexecutionwithdebugprintingofvariables,whichletsyoutrackhowthingsaregoingwrong.WithGDB,youhavetoplacewatchanddebugstatementsallovertheplaceforeverythingyouwant,andit’sdifficulttogetasolidtraceoftheexecution.•Thedebugprintscanstayinthecode,andwhenyouneedthem,youcanrecompileandtheycomeback.WithGDB,youhavetoconfigurethesameinformationuniquelyforeverydefectyouhavetohuntdown.•It’seasiertoturnondebugloggingonaserverthat’snotworkingright,andtheninspectthelogswhileitrunstoseewhat’sgoingon.Systemadministratorsknowhowtohandlelogging,buttheydon’tknowhowtouseGDB.•Printingthingsisjusteasier.Debuggersarealwaysobtuseandweirdwiththeirownquirkyinterfacesandinconsistencies.There’snothingcomplicatedaboutdebug("Yo,disright?%d",my_stuff);.•Whenyouwritedebugprintstofindadefect,you’reforcedtoactuallyanalyzethecodeandusethescientificmethod.Youcanthinkofdebugusageas,“Ihypothesizethatthecodeisbrokenhere.”Thenwhenyourunit,yougetyourhypothesistested,andifit’snotbroken,thenyoucanmovetoanotherpartwhereitcouldbe.Thismayseemlikeittakeslonger,butit’sactuallyfasterbecauseyougothroughaprocessofdifferentialdiagnosisandruleoutpossiblecausesuntilyoufindtherealone.•Debugprintingworksbetterwithunittesting.Youcanactuallyjustcompilethedebugswhileyouwork,andwhenaunittestexplodes,justgolookatthelogsatanytime.WithGDB,you’dhavetoreruntheunittestunderGDBandthentracethroughittoseewhat’sgoingon.

DespiteallofthesereasonsthatIrelyondebugoverGDB,IstilluseGDBinafewsituations,andI

thinkyoushouldhaveanytoolthathelpsyougetyourworkdone.Sometimes,youjusthavetoconnecttoabrokenprogramandpokearound.Or,maybeyou’vegotaserverthat’scrashingandyoucanonlygetatcorefilestoseewhy.Intheseandafewothercases,GDBisthewaytogo,andit’salwaysgoodtohaveasmanytoolsaspossibletohelpsolveproblems.Here’sabreakdownofwhenIuseGDBversusValgrindversusdebugprinting:

•IuseValgrindtocatchallmemoryerrors.IuseGDBifValgrindishavingproblemsorifusingValgrindwouldslowtheprogramdowntoomuch.•Iuseprintwithdebugtodiagnoseandfixdefectsrelatedtologicorusage.Thisamountstoabout90%ofthedefectsafteryoustartusingValgrind.•IuseGDBfortheremainingmysteriouslyweirdstufforemergencysituationstogatherinformation.IfValgrindisn’tturninganythingup,andIcan’tevenprintouttheinformationthatIneed,thenIbustoutGDBandstartpokingaround.MyuseofGDBinthiscaseisentirelytogatherinformation.OnceIhaveanideaofwhat’sgoingon,I’llgobacktowritingaunittesttocausethedefect,andthendoprintstatementstofindoutwhy.

ADebuggingStrategyThisprocesswillactuallyworkwithanydebuggingtechniqueyou’reusing.I’mgoingtodescribeitintermsofusingGDBsinceitseemspeopleskipthisprocessthemostwhenusingdebuggers.Usethisforeverybuguntilyouonlyneeditontheverydifficultones.

•Startalittletextfilecallednotes.txtanduseitasakindoflabnotesforideas,bugs,problems,andsoon.•BeforeyouuseGDB,writeoutthebugyou’regoingtofixandwhatcouldbecausingit.•Foreachcause,writeoutthefilesandfunctionswhereyouthinkthecauseiscomingfrom,orjustwritethatyoudon’tknow.•NowstartGDBandpickthefirstpossiblecausewithgoodfileandfunctionvariablesandsetbreakpointsthere.•UseGDBtothenruntheprogramandconfirmwhetherthatisthecause.Thebestwayistoseeifyoucanusethesetcommandtoeitherfixtheprogrameasilyorcausetheerrorimmediately.•Ifthisisn’tthecause,thenmarkinthenotes.txtthatitwasn’t,andwhy.Moveontothenextpossiblecausethat’seasiesttodebug,andkeepaddinginformation.

Incaseyouhaven’tnoticed,thisisbasicallythescientificmethod.Youwritedownasetofhypotheses,thenyouusedebuggingtoproveordisprovethem.Thisgivesyouinsightintomorepossiblecausesandeventuallyyoufindit.Thisprocesshelpsyouavoidgoingoverthesamepossiblecausesrepeatedlyafteryou’vefoundthattheyaren’tpossible.Youcanalsodothiswithdebugprinting,theonlydifferenceisthatyouactuallywriteoutyourhypothesesinthesourcecodeinsteadofinthenotes.txt.Inaway,debugprintingforcesyoutotacklebugsscientificallybecauseyouhavetowriteouthypothesesasprintstatements.

ExtraCredit•FindagraphicaldebuggerandcompareusingittorawGDB.Theseareusefulwhentheprogramyou’relookingatislocal,buttheyarepointlessifyouhavetodebugaprogramonaserver.

•YoucanenablecoredumpsonyourOS,andwhenaprogramcrashes,you’llgetacorefile.Thiscorefileislikeapostmortemoftheprogramthatyoucanloaduptoseewhathappenedrightatthecrashandwhatcausedit.Changeex18.csothatitcrashesafterafewiterations,thentrytogetacoredumpandanalyzeit.

Exercise21.AdvancedDataTypesandFlowControl

ThisexercisewillbeacompletecompendiumoftheavailableCdatatypesandflowcontrolstructuresyoucanuse.Itwillworkasareferencetocompleteyourknowledge,andwon’thaveanycodeforyoutoenter.I’llhaveyoumemorizesomeoftheinformationbycreatingflashcardssoyoucangettheimportantconceptssolidinyourmind.Forthisexercisetobeuseful,youshouldspendatleastaweekhammeringinthecontentandfillingoutalloftheelementsthataremissinghere.You’llbewritingoutwhateachonemeans,andthenwritingaprogramtoconfirmwhatyou’veresearched.

AvailableDataTypes

TypeModifiers

TypeQualifiers

TypeConversionCusesasortofsteppedtypepromotionmechanism,whereitlooksattwooperandsoneithersideofanexpression,andpromotesthesmallersidetomatchthelargersidebeforedoingtheoperation.Ifonesideofanexpressionisonthislist,thentheothersideisconvertedtothattypebeforetheoperationisdone.Itgoesinthisorder:

1.longdouble2.double3.float4.int(butonlycharandshortint);5.long

Ifyoufindyourselftryingtofigureouthowyourconversionsareworkinginanexpression,thendon’tleaveittothecompiler.Useexplicitcastingoperationstomakeitexactlywhatyouwant.Forexample,ifyouhaveClickheretoviewcodeimage

long+char-int*double

Ratherthantryingtofigureoutifitwillbeconvertedtodoublecorrectly,justusecasts:Clickheretoviewcodeimage

(double)long-(double)char-(double)int*double

Puttingthetypeyouwantinparenthesesbeforethevariablenameishowyouforceitintothetypeyoureallyneed.Theimportantthing,though,isalwayspromoteup,notdown.Don’tcastlongintocharunlessyouknowwhatyou’redoing.

TypeSizesThestdint.hdefinesbothasetoftypdefsforexact-sizedintegertypes,aswellasasetofmacrosforthesizesofallthetypes.Thisiseasiertoworkwiththantheolderlimits.hsinceitisconsistent.Herearethetypesdefined:

Thepatternhereisintheform(u)int(BITS)_twhereauisputinfronttoindicate“unsigned,”andBITSisanumberforthenumberofbits.Thispatternisthenrepeatedformacrosthatreturnthemaximumvaluesofthesetypes:

INT(N)_MAXMaximumpositivenumberofthesignedintegerofbits(N),suchasINT16_MAX.INT(N)_MINMinimumnegativenumberofsignedintegerofbits(N).UINT(N)_MAXMaximumpositivenumberofunsignedintegerofbits(N).Sinceit’sunsigned,theminimumis0anditcan’thaveanegativevalue.

Warning!Payattention!Don’tgolookingforaliteralINT(N)_MAXdefinitioninanyheaderfile.I’musingthe(N)asaplaceholderforanynumberofbitsyourplatformcurrentlysupports.This(N)couldbeanynumber—8,16,32,64,maybeeven128.IusethisnotationinthisexercisesothatIdon’thavetoliterallywriteouteverypossiblecombination.

Therearealsomacrosinstdint.hforsizesofthesize_ttype,integerslargeenoughtoholdpointers,andotherhandysizedefiningmacros.Compilershavetoatleasthavethese,andthentheycanallowother,largertypes.Hereisafulllistthatshouldbeinstdint.h:

AvailableOperatorsThisisacomprehensivelistofalltheoperatorsintheClanguage.Inthislist,I’mindicatingthefollowing:

MathOperatorsTheseperformyourbasicmathoperations,plusIinclude()sinceitcallsafunctionandisclosetoamathoperation.

DataOperatorsTheseareusedtoaccessdataindifferentwaysandforms.

LogicOperatorsThesehandletestingequalityandinequalityofvariables.

BitOperatorsThesearemoreadvancedandareforshiftingandmodifyingtherawbitsinintegers.

BooleanOperatorsTheseareusedintruthtesting.Studytheternaryoperatorcarefully.It’sveryhandy.

AssignmentOperatorsHerearecompoundassignmentoperatorsthatassignavalue,and/orperformanoperationatthesametime.Mostoftheaboveoperationscanalsobecombinedintoacompoundassignmentoperator.

AvailableControlStructuresThereareafewcontrolstructuresthatyouhaven’tencounteredyet.

do-whiledo{...}while(X);Firstdoesthecodeintheblock,thenteststheXexpressionbeforeexiting.

breakPutsabreakinaloop,endingitearly.continueStopsthebodyofaloopandjumpstothetestsoitcancontinue.gotoJumpstoaspotinthecodewhereyou’veplacedalabel:,andyou’vebeenusingthisinthedbg.hmacrostogototheerror:label.

ExtraCredit•Readstdint.horadescriptionofit,andwriteoutalltheavailablesizeidentifiers.•Gothrougheachitemhereandwriteoutwhatitdoesincode.Researchitonlinesoyouknowyougotitright.•Getthisinformationmemorizedbymakingflashcardsandspending15minutesadaypracticingit.•Createaprogramthatprintsoutexamplesofeachtype,andconfirmthatyourresearchisright.

Exercise22.TheStack,Scope,andGlobals

Theconceptofscopeseemstoconfusequiteafewpeoplewhentheyfirststartprogramming.Itoriginallycamefromtheuseofthesystemstack(whichwelightlycoveredearlier),andhowitwasusedtostoretemporaryvariables.Inthisexercise,we’lllearnaboutscopebylearninghowastackdatastructureworks,andthenfeedingthatconceptbackintohowmodernCdoesscoping.Therealpurposeofthisexercise,though,istolearnwherethehellthingsliveinC.Whensomeonedoesn’tgrasptheconceptofscope,it’salmostalwaysafailureinunderstandingwherevariablesarecreated,exist,anddie.Onceyouknowwherethingsare,theconceptofscopebecomeseasier.Thisexercisewillrequirethreefiles:

ex22.hAheaderfilethatsetsupsomeexternalvariablesandsomefunctions.ex22.cThisisn’tyourmainlikenormal,butinsteadasourcefilethatwillbecometheobjectfileex22.o,whichwillhavesomefunctionsandvariablesinitdefinedfromex22.h.

ex22_main.cTheactualmainthatwillincludetheothertwo,anddemonstratewhattheycontain,aswellasotherscopeconcepts.

ex22.handex22.cYourfirststepistocreateyourownheaderfilenamedex22.hthatdefinesthefunctionsandexternvariables:

ex22.h


#ifndef_ex22_h

#define_ex22_h

//makesTHE_SIZEinex22.cavailabletoother.cfiles

externintTHE_SIZE;

//getsandsetsaninternalstaticvariableinex22.c

intget_age();voidset_age(intage);

//updatesastaticvariablethat'sinsideupdate_ratio

doubleupdate_ratio(doubleratio);

voidprint_size();

#endif

TheimportantthingtoseehereistheuseofexternintTHE_SIZE,whichI’llexplainafteryoucreatethismatchingex22.c:

ex22.c


1#include<stdio.h>2#include"ex22.h"

3#include"dbg.h"45intTHE_SIZE=1000;67staticintTHE_AGE=37;89intget_age()10{11returnTHE_AGE;12}1314voidset_age(intage)15{16THE_AGE=age;17}1819doubleupdate_ratio(doublenew_ratio)20{21staticdoubleratio=1.0;2223doubleold_ratio=ratio;24ratio=new_ratio;2526returnold_ratio;27}2829voidprint_size()30{31log_info("Ithinksizeis:%d",THE_SIZE);32}

Thesetwofilesintroducesomenewkindsofstorageforvariables:externThiskeywordisawaytotellthecompiler“thevariableexists,butit’sinanother‘external’location.”Typicallythismeansthatone.cfileisgoingtouseavariablethat’sbeendefinedinanother.cfile.Inthiscase,we’resayingex22.chasavariableTHE_SIZEthatwillbeaccessedfromex22_main.c.

static(file)Thiskeywordiskindoftheinverseofextern,andsaysthatthevariableisonlyusedinthis.cfileandshouldnotbeavailabletootherpartsoftheprogram.Keepinmindthatstaticatthefilelevel(aswithTHE_AGEhere)isdifferentthaninotherplaces.

static(function)Ifyoudeclareavariableinafunctionstatic,thenthatvariableactslikeastaticdefinedinthefile,butit’sonlyaccessiblefromthatfunction.It’sawayofcreatingconstantstateforafunction,butinrealityit’srarelyusedinmodernCprogrammingbecausetheyarehardtousewiththreads.

Inthesetwofiles,youshouldunderstandthefollowingvariablesandfunctions:THE_SIZEThisisthevariableyoudeclaredexternthatyou’llplaywithfromex22_main.c.get_ageandset_ageThesearetakingthestaticvariableTHE_AGE,butexposingittootherpartsoftheprogramthroughfunctions.Youcan’taccessTHE_AGEdirectly,butthesefunctionscan.

update_ratioThistakesanewratiovalue,andreturnstheoldone.Itusesafunctionlevelstaticvariableratiotokeeptrackofwhattheratiocurrentlyis.

print_sizeThisprintsoutwhatex22.cthinksTHE_SIZEiscurrently.

ex22_main.cOnceyouhavethatfilewritten,youcanthenmakethemainfunction,whichusesalloftheseanddemonstratessomemorescopeconventions.

ex22_main.c


1#include"ex22.h"2#include"dbg.h"34constchar*MY_NAME="ZedA.Shaw";56voidscope_demo(intcount)7{8log_info("countis:%d",count);910if(count>10){11intcount=100;//BAD!BUGS!1213log_info("countinthisscopeis%d",count);14}1516log_info("countisatexit:%d",count);1718count=3000;1920log_info("countafterassign:%d",count);21}2223intmain(intargc,char*argv[])24{25//testoutTHE_AGEaccessors26log_info("Myname:%s,age:%d",MY_NAME,get_age());2728set_age(100);2930log_info("Myageisnow:%d",get_age());3132//testoutTHE_SIZEextern33log_info("THE_SIZEis:%d",THE_SIZE);34print_size();3536THE_SIZE=9;3738log_info("THESIZEisnow:%d",THE_SIZE);39print_size();4041//testtheratiofunctionstatic42log_info("Ratioatfirst:%f",update_ratio(2.0));43log_info("Ratioagain:%f",update_ratio(10.0));44log_info("Ratiooncemore:%f",update_ratio(300.0));4546//testthescopedemo47intcount=4;48scope_demo(count);49scope_demo(count*20);5051log_info("countaftercallingscope_demo:%d",count);52

53return0;54}

I’llbreakthisfiledownlinebyline,butasIdo,youshouldfindeachvariableandwhereitlives.ex22_main.c:4Aconst,whichstandsforconstant,andisanalternativetousingadefinetocreateaconstantvariable.

ex22_main.c:6Asimplefunctionthatdemonstratesmorescopeissuesinafunction.ex22_main.c:8Thisprintsoutthevalueofcountasitisatthetopofthefunction.ex22_main.c:10Anif-statementthatstartsanewscopeblock,andthenhasanothercountvariableinit.Thisversionofcountisactuallyawholenewvariable.It’skindofliketheif-statementstartedanewminifunction.

ex22_main.c:11Thecountthatislocaltothisblockisactuallydifferentfromtheoneinthefunction’sparameterlist.

ex22_main.c:13Thisprintsitoutsoyoucanseeit’sactually100here,notwhatwaspassedtoscope_demo.

ex22_main.c:16Nowforthefreakypart.Youhavecountintwoplaces:theparameterstothisfunction,andintheif-statement.Theif-statementcreatedanewblock,sothecountonline11doesnotimpacttheparameterwiththesamename.Thislineprintsitout,andyou’llseethatitprintsthevalueoftheparameter,not100.

ex22_main.c:18-20Then,Isettheparametercountto3000andprintthatout,whichwilldemonstratethatyoucanchangefunctionparametersandtheydon’timpactthecaller ’sversionofthevariable.

Makesurethatyoutracethroughthisfunction,butdon’tthinkthatyouunderstandscopequiteyet.Juststarttorealizethatifyoumakeavariableinsideablock(asinif-statementsorwhile-loops),thenthosevariablesarenewvariablesthatexistonlyinthatblock.Thisiscrucialtounderstand,andisalsoasourceofmanybugs.We’lladdresswhyyoushouldn’tmakeavariableinsideablockshortly.Therestoftheex22_main.cthendemonstratesallofthesebymanipulatingandprintingthemout:

ex22_main.c:26ThisprintsoutthecurrentvaluesofMY_NAME,andgetsTHE_AGEfromex22.cbyusingtheaccessorfunctionget_age.

ex22_main.c:27-30Thisusesset_ageinex22.ctochangeTHE_AGEandthenprintitout.ex22_main.c:33-39ThenIdothesamethingtoTHE_SIZEfromex22.c,butthistimeI’maccessingitdirectly.I’malsodemonstratingthatit’sactuallychanginginthatfilebyprintingithereandwithprint_size.

ex22_main.c:42-44Here,Ishowhowthestaticvariableratioinsideupdate_ratioismaintainedbetweenfunctioncalls.

ex22_main.c:46-51Finally,I’mrunningscope_demoafewtimessoyoucanseethescopeinaction.Thebigthingtonoticeisthatthelocalcountvariableremainsunchanged.Youmustunderstandthatpassinginavariablelikethiswon’tletyouchangeitinthefunction.Todothat,youneedouroldfriendthepointer.Ifyouweretopassapointertothiscount,thenthecalledfunctionwouldhavetheaddressofitandcouldchangeit.

Thatexplainswhat’sgoingon,butyoushouldtracethroughthesefilesandmakesureyouknow

whereeverythingisasyoustudyit.

WhatYouShouldSeeThistime,insteadofusingyourMakefile,Iwantyoutobuildthesetwofilesmanuallysoyoucanseehowthecompileractuallyputsthemtogether.Here’swhatyoushoulddoandseeforoutput:

Exercise22Session


$cc-Wall-g-DNDEBUG-c-oex22.oex22.c$cc-Wall-g-DNDEBUGex22_main.cex22.o-oex22_main$./ex22_main[INFO](ex22_main.c:26)Myname:ZedA.Shaw,age:37

[INFO](ex22_main.c:30)Myageisnow:100

[INFO](ex22_main.c:33)THE_SIZEis:1000

[INFO](ex22.c:32)Ithinksizeis:1000

[INFO](ex22_main.c:38)THESIZEisnow:9

[INFO](ex22.c:32)Ithinksizeis:9

[INFO](ex22_main.c:42)Ratioatfirst:1.000000

[INFO](ex22_main.c:43)Ratioagain:2.000000

[INFO](ex22_main.c:44)Ratiooncemore:10.000000

[INFO](ex22_main.c:8)countis:4

[INFO](ex22_main.c:16)countisatexit:4

[INFO](ex22_main.c:20)countafterassign:3000

[INFO](ex22_main.c:8)countis:80

[INFO](ex22_main.c:13)countinthisscopeis100

[INFO](ex22_main.c:16)countisatexit:80

[INFO](ex22_main.c:20)countafterassign:3000

[INFO](ex22_main.c:51)countaftercallingscope_demo:4

Makesureyoutracehoweachvariableischangingandmatchittothelinethatgetsoutput.I’musinglog_infofromthedbg.hmacrossoyoucangettheexactlinenumberwhereeachvariableisprinted,andfinditinthefilesfortracing.

Scope,Stack,andBugsIfyou’vedonethisright,youshouldnowseemanyofthedifferentwaysyoucanplacevariablesinyourCcode.Youcanuseexternoraccessfunctionslikeget_agetocreateglobals.Youcanmakenewvariablesinsideanyblocks,andthey’llretaintheirownvaluesuntilthatblockexits,leavingtheoutervariablesalone.Youalsocanpassavaluetoafunction,andchangetheparameterbutwithoutchangingthecaller ’sversionofit.Themostimportantthingtorealizeisthatallofthiscausesbugs.C’sabilitytoplacethingsinmanyplacesinyourmachine,andthenletyouaccessitinthoseplaces,meansthatyoucangeteasilyconfusedaboutwheresomethinglives.Ifyoudon’tknowwhereitlives,thenthere’sachanceyouwon’tmanageitproperly.Withthatinmind,herearesomerulestofollowwhenwritingCcodesoyoucanavoidbugsrelatedtothestack:

•DonotshadowavariablelikeI’vedoneherewithcountinscope_demo.Itleavesyouopentosubtleandhiddenbugswhereyouthinkyou’rechangingavaluebutyou’reactuallynot.•Avoidusingtoomanyglobals,especiallyifacrossmultiplefiles.Ifyouhavetousethem,thenuseaccessorfunctionslikeI’vedonewithget_age.Thisdoesn’tapplytoconstants,since

thoseareread-only.I’mtalkingaboutvariableslikeTHE_SIZE.Ifyouwantpeopletomodifyorsetthisvariable,thenmakeaccessorfunctions.•Whenindoubt,putitontheheap.Don’trelyonthesemanticsofthestackorspecializedlocations.Justcreatethingswithmalloc.•Don’tusefunctionstaticvariableslikeIdidinupdate_ratio.They’rerarelyusefulandendupbeingahugepainwhenyouneedtomakeyourcodeconcurrentinthreads.They’realsohardashelltofindcomparedtoawell-doneglobalvariable.•Avoidreusingfunctionparameters.It’sconfusingastowhetheryou’rejustreusingitorifyouthinkyou’rechangingthecaller’sversionofit.

Aswithallthings,theserulescanbebrokenwhenit’spractical.Infact,Iguaranteeyou’llrunintocodethatbreaksalloftheserulesandisperfectlyfine.Theconstraintsofdifferentplatformsevenmakeitnecessarysometimes.

HowtoBreakItForthisexercise,trytoaccessorchangesomethingsyoucan’ttobreaktheprogram.

•Trytodirectlyaccessvariablesinex22.cfromex22_main.cthatyouthinkyoucan’taccess.Forexample,canyougetatratioinsideupdate_ratio?Whatifyouhadapointertoit?•Ditchtheexterndeclarationinex22.htoseewhaterrorsorwarningsyouget.•Addstaticorconstspecifierstodifferentvariables,andthentrytochangethem.

ExtraCredit•Researchtheconceptofpassbyvalueversuspassbyreference.Writeanexampleofboth.•Usepointerstogainaccesstothingsyoushouldn’thaveaccessto.•Useyourdebuggertoseewhatthiskindofaccesslookslikewhenyoudoitwrong.•Writearecursivefunctionthatcausesastackoverflow.Don’tknowwhatarecursivefunctionis?Trycallingscope_demoatthebottomofscope_demoitselfsothatitloops.•RewritetheMakefilesothatitcanbuildthis.

Exercise23.MeetDuff’sDevice

ThisexerciseisabrainteaserwhereIintroduceyoutooneofthemostfamoushacksinCcalledDuff’sdevice,namedafterTomDuff,itsinventor.Thislittlesliceofawesome(evil?)hasnearlyeverythingyou’vebeenlearningwrappedinonetiny,littlepackage.Figuringouthowitworksisalsoagood,funpuzzle.

Warning!PartofthefunofCisthatyoucancomeupwithcrazyhackslikethis,butthisisalsowhatmakesCannoyingtouse.It’sgoodtolearnaboutthesetricksbecauseitgivesyouadeeperunderstandingofthelanguageandyourcomputer.Butyoushouldneverusethis.Alwaysstriveforeasy-to-readcode.

DiscoveredbyTomDuff,Duff’sdeviceisatrickwiththeCcompilerthatactuallyshouldn’twork.Iwon’ttellyouwhatitdoesyetsincethisismeanttobeapuzzleforyoutoponderandtrytosolve.You’llgetthiscoderunningandthentrytofigureoutwhatitdoes,andwhyitdoesitthisway.

ex23.c


1#include<stdio.h>2#include<string.h>3#include"dbg.h"45intnormal_copy(char*from,char*to,intcount)6{7inti=0;89for(i=0;i<count;i++){10to[i]=from[i];11}1213returni;14}1516intduffs_device(char*from,char*to,intcount)17{18{19intn=(count+7)/8;2021switch(count%8){22case0:23do{24*to++=*from++;25case7:26*to++=*from++;27case6:28*to++=*from++;29case5:30*to++=*from++;31case4:32*to++=*from++;33case3:34*to++=*from++;

35case2:36*to++=*from++;37case1:38*to++=*from++;39}while(--n>0);40}41}4243returncount;44}4546intzeds_device(char*from,char*to,intcount)47{48{49intn=(count+7)/8;5051switch(count%8){52case0:53again:*to++=*from++;5455case7:56*to++=*from++;57case6:58*to++=*from++;59case5:60*to++=*from++;61case4:62*to++=*from++;63case3:64*to++=*from++;65case2:66*to++=*from++;67case1:68*to++=*from++;69if(--n>0)70gotoagain;71}72}7374returncount;75}7677intvalid_copy(char*data,intcount,charexpects)78{79inti=0;80for(i=0;i<count;i++){81if(data[i]!=expects){82log_err("[%d]%c!=%c",i,data[i],expects);83return0;84}85}8687return1;88}8990intmain(intargc,char*argv[])91{92charfrom[1000]={'a'};93charto[1000]={'c'};94intrc=0;9596//setupthefromtohavesomestuff97memset(from,'x',1000);98//setittoafailuremode

99memset(to,'y',1000);100check(valid_copy(to,1000,'y'),"Notinitializedright.");101102//usenormalcopyto103rc=normal_copy(from,to,1000);104check(rc==1000,"Normalcopyfailed:%d",rc);105check(valid_copy(to,1000,'x'),"Normalcopyfailed.");106107//reset108memset(to,'y',1000);109110//duffsversion111rc=duffs_device(from,to,1000);112check(rc==1000,"Duff'sdevicefailed:%d",rc);113check(valid_copy(to,1000,'x'),"Duff'sdevicefailedcopy.");114115//reset116memset(to,'y',1000);117118//myversion119rc=zeds_device(from,to,1000);120check(rc==1000,"Zed'sdevicefailed:%d",rc);121check(valid_copy(to,1000,'x'),"Zed'sdevicefailedcopy.");122123return0;124error:125return1;126}

Inthiscode,Ihavethreeversionsofacopyfunction:normal_copyThisisjustaplainfor-loopthatcopiescharactersfromonearraytoanother.duffs_deviceThisiscalledDuff’sdevice,namedafterTomDuff,thepersontoblameforthisdeliciousevil.

zeds_deviceAversionofDuff’sdevicethatjustusesagotosoyoucanclueintowhat’shappeningwiththeweirddo-whileplacementinduffs_device.

Studythesethreefunctionsbeforecontinuing.Trytoexplainwhat’sgoingontoyourself.

WhatYouShouldSeeThere’snooutputfromthisprogram,itjustrunsandexits.Runitunderyourdebuggertoseeifyoucancatchanymoreerrors.Trycausingsomeofyourown,asIshowedyouinExercise4.

SolvingthePuzzleThefirstthingtounderstandisthatCisratherlooseregardingsomeofitssyntax.Thisiswhyyoucanputhalfofado-whileinonepartofaswitch-statement,thentheotherhalfsomewhereelse,andthecodewillstillwork.Ifyoulookatmyversionwiththegotoagain,it’sactuallymoreclearwhat’sgoingon,butmakesureyouunderstandhowthatpartworks.Thesecondthingishowthedefaultfallthroughsemanticsofswitch-statementsletyoujumptoaparticularcase,andthenitwilljustkeeprunninguntiltheendoftheswitch.Thefinalclueisthecount%8andthecalculationofnatthetop.Now,tosolvehowthesefunctionswork,dothefollowing:

•Printthiscodeoutsothatyoucanwriteonsomepaper.

•Writeeachofthevariablesinatableastheylookwhentheygetinitializedrightbeforetheswitch-statement.•Followthelogictotheswitch,thendothejumptotherightcase.•Updatethevariables,includingtheto,from,andthearraystheypointat.•Whenyougettothewhilepartormygotoalternative,checkyourvariables,andthenfollowthelogiceitherbacktothetopofthedo-whileortowheretheagainlabelislocated.•Followthroughthismanualtracing,updatingthevariables,untilyou’resureyouseehowthisflows.

WhyBother?Whenyou’vefiguredouthowitactuallyworks,thefinalquestionis:Whywouldyoueverwanttodothis?Thepurposeofthistrickistomanuallydoloopunrolling.Large,longloopscanbeslow,soonewaytospeedthemupistofindsomefixedchunkoftheloop,andthenjustduplicatethecodeintheloopthatmanytimessequentially.Forexample,ifyouknowalooprunsaminimumof20times,thenyoucanputthecontentsoftheloop20timesinthesourcecode.Duff’sdeviceisbasicallydoingthisautomaticallybychunkinguptheloopintoeightiterationchunks.It’scleverandactuallyworks,butthesedaysagoodcompilerwilldothisforyou.Youshouldn’tneedthisexceptintherarecasewhereyouhaveprovenitwouldimproveyourspeed.

ExtraCredit•Neverusethisagain.•GolookattheWikipediaentryforDuff’sdeviceandseeifyoucanspottheerror.Readthearticle,compareittotheversionIhavehere,andtrytounderstandwhytheWikipediacodewon’tworkforyoubutworkedforTomDuff.•Createasetofmacrosthatletsyoucreateanylengthofdevicelikethis.Forexample,whatifyouwantedtohave32casestatementsanddidn’twanttowriteoutallofthem?Canyoudoamacrothatlaysdowneightatatime?•Changethemaintoconductsomespeedteststoseewhichoneisreallythefastest.•Readaboutmemcpy,memmove,andmemset,andalsocomparetheirspeed.•Neverusethisagain!

Exercise24.Input,Output,Files

You’vebeenusingprintftoprintthings,andthat’sgreatandall,butyouneedmore.Inthisexercise,you’llbeusingthefunctionsfscanfandfgetstobuildinformationaboutapersoninastructure.Afterthissimpleintroductionaboutreadinginput,you’llgetafulllistofthefunctionsthatChasforI/O.Someoftheseyou’vealreadyseenandused,sothiswillbeanothermemorizationexercise.

ex24.c


1#include<stdio.h>2#include"dbg.h"34#defineMAX_DATA10056typedefenumEyeColor{7BLUE_EYES,GREEN_EYES,BROWN_EYES,8BLACK_EYES,OTHER_EYES9}EyeColor;1011constchar*EYE_COLOR_NAMES[]={12"Blue","Green","Brown","Black","Other"13};1415typedefstructPerson{16intage;17charfirst_name[MAX_DATA];18charlast_name[MAX_DATA];19EyeColoreyes;20floatincome;21}Person;2223intmain(intargc,char*argv[])24{25Personyou={.age=0};26inti=0;27char*in=NULL;2829printf("What'syourFirstName?");30in=fgets(you.first_name,MAX_DATA-1,stdin);31check(in!=NULL,"Failedtoreadfirstname.");3233printf("What'syourLastName?");34in=fgets(you.last_name,MAX_DATA-1,stdin);35check(in!=NULL,"Failedtoreadlastname.");3637printf("Howoldareyou?");38intrc=fscanf(stdin,"%d",&you.age);39check(rc>0,"Youhavetoenteranumber.");4041printf("Whatcolorareyoureyes:\n");42for(i=0;i<=OTHER_EYES;i++){43printf("%d)%s\n",i+1,EYE_COLOR_NAMES[i]);44}45printf(">");46

47inteyes=-1;48rc=fscanf(stdin,"%d",&eyes);49check(rc>0,"Youhavetoenteranumber.");5051you.eyes=eyes-1;52check(you.eyes<=OTHER_EYES53&&you.eyes>=0,"Doitright,that'snotanoption.");5455printf("Howmuchdoyoumakeanhour?");56rc=fscanf(stdin,"%f",&you.income);57check(rc>0,"Enterafloatingpointnumber.");5859printf("-----RESULTS-----\n");6061printf("FirstName:%s",you.first_name);62printf("LastName:%s",you.last_name);63printf("Age:%d\n",you.age);64printf("Eyes:%s\n",EYE_COLOR_NAMES[you.eyes]);65printf("Income:%f\n",you.income);6667return0;68error:6970return-1;71}

Thisprogramisdeceptivelysimple,andintroducesafunctioncalledfscanf,whichisthefilescanf.Thescanffamilyoffunctionsaretheinverseoftheprintfversions.Whereprintfprintedoutdatabasedonaformat,scanfreads(orscans)inputbasedonaformat.There’snothingoriginalinthebeginningofthefile,sohere’swhatthemainisdoingintheprogram:

ex24.c:24-28Setsupsomevariableswe’llneed.ex24.c:30-32Getsyourfirstnameusingthefgetsfunction,whichreadsastringfromtheinput(inthiscasestdin),butmakessureitdoesn’toverflowthegivenbuffer.

ex24.c:34-36Samethingforyou.last_name,againusingfgets.ex24.c:38-39Usesfscanftoreadanintegerfromstdinandputitintoyou.age.Youcanseethatthesameformatstringisusedasprintftoprintaninteger.Youshouldalsoseethatyouhavetogivetheaddressofyou.agesothatfscanfhasapointertoitandcanmodifyit.Thisisagoodexampleofusingapointertoapieceofdataasanoutparameter.

ex24.c:41-45Printsoutalloftheoptionsavailableforeyecolor,withamatchingnumberthatworkswiththeEyeColorenumabove.

ex24.c:47-50Usingfscanfagain,getsanumberfortheyou.eyes,butmakesuretheinputisvalid.ThisisimportantbecausesomeonecanenteravalueoutsidetheEYE_COLOR_NAMESarrayandcauseasegmentationfault.

ex24.c:52-53Getshowmuchyoumakeasafloatfortheyou.income.ex24.c:55-61Printseverythingoutsoyoucanseeifyouhaveitright.NoticethatEYE_COLOR_NAMESisusedtoprintoutwhattheEyeColorenumisactuallycalled.

WhatYouShouldSeeWhenyourunthisprogram,youshouldseeyourinputsbeingproperlyconverted.Makesureyoutrytogiveitbogusinputtoo,soyoucanseehowitprotectsagainsttheinput.

Exercise24Session


$makeex24cc-Wall-g-DNDEBUGex24.c-oex24

$./ex24What'syourFirstName?Zed

What'syourLastName?Shaw

Howoldareyou?37

Whatcolorareyoureyes:

1)Blue

2)Green

3)Brown

4)Black

5)Other

>1Howmuchdoyoumakeanhour?1.2345

-----RESULTS-----

FirstName:Zed

LastName:Shaw

Age:37

Eyes:Blue

Income:1.234500

HowtoBreakItThisisallfineandgood,butthereallyimportantpartofthisexerciseishowscanfactuallysucks.It’sfineforasimpleconversionofnumbers,butfailsforstringsbecauseit’sdifficulttotellscanfhowbigabufferisbeforeyoureadit.There’salsoaproblemwiththefunctiongets(notfgets,thenon-fversion),whichweavoided.Thatfunctionhasnoideahowbigtheinputbufferisatallandwilljusttrashyourprogram.Todemonstratetheproblemswithfscanfandstrings,changethelinesthatusefgetssotheyarefscanf(stdin,"%50s",you.first_name),andthentrytouseitagain.Noticeitseemstoreadtoomuchandtheneatyourenterkey?Thisdoesn’tdowhatyouthinkitdoes,andratherthandealwithweirdscanfissues,youshouldjustusefgets.Next,changethefgetstousegets,thenrunyourdebuggeronex24.Dothisinside:

"run<</dev/urandom"

Thisfeedsrandomgarbageintoyourprogram.Thisiscalledfuzzingyourprogram,andit’sagoodwaytofindinputbugs.Inthiscase,you’refeedinggarbagefromthe/dev/urandomfile(device),andthenwatchingitcrash.Insomeplatforms,youmayhavetodothisafewtimes,orevenadjusttheMAX_DATAdefinesoit’ssmallenough.Thegetsfunctionissobadthatsomeplatformsactuallywarnyouwhentheprogramrunsthatyou’reusinggets.Youshouldneverusethisfunction,ever.Finally,taketheinputforyou.eyesandremovethecheckthatthenumberiswithintherightrange.Then,feeditbadnumberslike-1or1000.Dothisunderthedebuggersoyoucanseewhathappens

there,too.

TheI/OFunctionsThisisashortlistofvariousI/Ofunctionsthatyoushouldlookup.Createflashcardsthathavethefunctionnameandallthevariantssimilartoit.

•fscanf•fgets•fopen•freopen•fdopen•fclose•fcloseall•fgetpos•fseek•ftell•rewind•fprintf•fwrite•fread

Gothroughtheseandmemorizethedifferentvariantsandwhattheydo.Forexample,forthecardfscanf,you’llhavescanf,sscanf,vscanf,etc.,andthenwhateachofthosedoesontheback.Finally,usemantoreadthehelpforeachvarianttogettheinformationyouneedfortheflashcards.Forexample,thepageforfscanfcomesfrommanfscanf.

ExtraCredit•Rewritethistonotusefscanfatall.You’llneedtousefunctionslikeatoitoconverttheinputstringstonumbers.•Changethistouseplainscanfinsteadoffscanftoseewhatthedifferenceis.•Fixitsothattheirinputnamesgetstrippedofthetrailingnewlinecharactersandanywhitespace.•Usescanftowriteafunctionthatreadsonecharacteratatimeandfillsinthenamesbutdoesn’tgopasttheend.Makethisfunctiongenericsoitcantakeasizeforthestring,butjustmakesureyouendthestringwith'\0'nomatterwhat.

Exercise25.VariableArgumentFunctions

InC,youcancreateyourownversionsoffunctionslikeprintfandscanfbycreatingavariableargumentfunction,orvarargfunction.Thesefunctionsusetheheaderstdarg.h,andwiththem,youcancreatenicerinterfacestoyourlibrary.Theyarehandyforcertaintypesofbuilderfunctions,formattingfunctions,andanythingthattakesvariablearguments.UnderstandingvarargfunctionsisnotessentialtocreatingCprograms.IthinkI’veuseditmaybe20timesinmycodeinalloftheyearsI’vebeenprogramming.However,knowinghowavarargfunctionworkswillhelpyoudebugtheprogramsyouuseandgivesyouabetterunderstandingofthecomputer.

ex25.c


123#include<stdlib.h>4#include<stdio.h>5#include<stdarg.h>6#include"dbg.h"78#defineMAX_DATA100910intread_string(char**out_string,intmax_buffer)11{12*out_string=calloc(1,max_buffer+1);13check_mem(*out_string);1415char*result=fgets(*out_string,max_buffer,stdin);16check(result!=NULL,"Inputerror.");1718return0;1920error:21if(*out_string)free(*out_string);22*out_string=NULL;23return-1;24}2526intread_int(int*out_int)27{28char*input=NULL;29intrc=read_string(&input,MAX_DATA);30check(rc==0,"Failedtoreadnumber.");3132*out_int=atoi(input);3334free(input);35return0;3637error:38if(input)free(input);39return-1;40}4142intread_scan(constchar*fmt,...)

43{44inti=0;45intrc=0;46int*out_int=NULL;47char*out_char=NULL;48char**out_string=NULL;49intmax_buffer=0;5051va_listargp;52va_start(argp,fmt);5354for(i=0;fmt[i]!='\0';i++){55if(fmt[i]=='%'){56i++;57switch(fmt[i]){58case'\0':59sentinel("Invalidformat,youendedwith%%.");60break;6162case'd':63out_int=va_arg(argp,int*);64rc=read_int(out_int);65check(rc==0,"Failedtoreadint.");66break;6768case'c':69out_char=va_arg(argp,char*);70*out_char=fgetc(stdin);71break;7273case's':74max_buffer=va_arg(argp,int);75out_string=va_arg(argp,char**);76rc=read_string(out_string,max_buffer);77check(rc==0,"Failedtoreadstring.");78break;7980default:81sentinel("Invalidformat.");82}83}else{84fgetc(stdin);85}8687check(!feof(stdin)&&!ferror(stdin),"Inputerror.");88}8990va_end(argp);91return0;9293error:94va_end(argp);95return-1;96}9798intmain(intargc,char*argv[])99{100char*first_name=NULL;101charinitial='';102char*last_name=NULL;103intage=0;104105printf("What'syourfirstname?");106intrc=read_scan("%s",MAX_DATA,&first_name);

107check(rc==0,"Failedfirstname.");108109printf("What'syourinitial?");110rc=read_scan("%c\n",&initial);111check(rc==0,"Failedinitial.");112113printf("What'syourlastname?");114rc=read_scan("%s",MAX_DATA,&last_name);115check(rc==0,"Failedlastname.");116117printf("Howoldareyou?");118rc=read_scan("%d",&age);119120printf("----RESULTS----\n");121printf("FirstName:%s",first_name);122printf("Initial:'%c'\n",initial);123printf("LastName:%s",last_name);124printf("Age:%d\n",age);125126free(first_name);127free(last_name);128return0;129error:130return-1;131}

Thisprogramissimilartothepreviousexercise,exceptIhavewrittenmyownscanffunctiontohandlestringsthewayIwant.Themainfunctionshouldbecleartoyou,aswellasthetwofunctionsread_stringandread_int,sincetheydonothingnew.Thevarargsfunctioniscalledread_scan,anditdoesthesamethingthatscanfisdoingusingtheva_listdatastructureandsupportingmacrosandfunctions.Here’show:

•Isetasthelastparameterofthefunctionthekeyword...toindicatetoCthatthisfunctionwilltakeanynumberofargumentsafterthefmtargument.Icouldputmanyotherargumentsbeforethis,butIcan’tputanymoreafterthis.•Aftersettingupsomevariables,Icreateava_listvariableandinitializeitwithva_start.Thisconfiguresthegearinstdarg.hthathandlesvariablearguments.•Ithenuseafor-looptoloopthroughtheformatstringfmtandprocessthesamekindofformatsthatscanfhas,onlymuchsimpler.Ijusthaveintegers,characters,andstrings.•WhenIhitaformat,Iusetheswitch-statementtofigureoutwhattodo.•Now,togetavariablefromtheva_listargp,Iusethemacrova_arg(argp,TYPE)whereTYPEistheexacttypeofwhatIwillassignthisfunctionparameterto.Thedownsidetothisdesignisthatyou’reflyingblind,soifyoudon’thaveenoughparameters,thenohwell,you’llmostlikelycrash.•TheinterestingdifferencefromscanfisI’massumingthatpeoplewantread_scantocreatethestringsitreadswhenithitsan's'formatsequence.Whenyougivethissequence,thefunctiontakestwoparametersofftheva_listargpstack:themaxfunctionsizetoread,andtheoutputcharacterstringpointer.Usingthatinformation,itjustrunsread_stringtodotherealwork.•Thismakesread_scanmoreconsistentthanscanf,sinceyoualwaysgiveanaddress-of&onvariablestohavethemsetappropriately.

•Finally,ifthefunctionencountersacharacterthat’snotinthecorrectformat,itjustreadsonechartoskipit.Itdoesn’tcarewhatthatcharis,justthatitshouldskipit.

WhatYouShouldSeeWhenyourunthisone,it’ssimilartothelastone.

Exercise25Session


$makeex25cc-Wall-g-DNDEBUGex25.c-oex25

$./ex25What'syourfirstname?Zed

What'syourinitial?A

What'syourlastname?Shaw

Howoldareyou?37

----RESULTS----

FirstName:Zed

Initial:'A'

LastName:Shaw

Age:37

HowtoBreakItThisprogramshouldbemorerobustagainstbufferoverflows,butitdoesn’thandletheformattedinputaswellasscanf.Totrytobreakthis,changethecodesothatyouforgettopassintheinitialsizefor‘%s’formats.TrygivingitmoredatathanMAX_DATA,andthenseehowomittingcallocinread_stringchangeshowitworks.Finally,there’saproblemwherefgetseatsthenewlines,sotrytofixthatusingfgetc,butleaveoutthe\0thatendsthestring.

ExtraCredit•Makedoubleandtriplesurethatyouknowwhateachoftheout_variablesisdoing.Mostimportantly,youshouldknowwhatout_stringisandhowit’sapointertoapointer,sothatyouunderstandwhenyou’resettingthepointerversusthecontentsisimportant.•Writeasimilarfunctiontoprintfthatusesthevarargssystem,andrewritemaintouseit.•Asusual,readthemanpageonallofthissothatyouknowwhatitdoesonyourplatform.Someplatformswillusemacros,otherswillusefunctions,andsomewillhavethesedonothing.Italldependsonthecompilerandtheplatformyouuse.

Exercise26.Projectlogfind

Thisisasmallprojectforyoutoattemptonyourown.TobeeffectiveatC,you’llneedtolearntoapplywhatyouknowtoproblems.Inthisexercise,IdescribeatoolIwantyoutoimplement,andIdescribeitinavaguewayonpurpose.Thisisdonesothatyouwilltrytoimplementwhateveryoucan,howeveryoucan.Whenyou’redone,youcanthenwatchavideofortheexercisethatshowsyouhowIdidit,andthenyoucangetthecodeandcompareittoyours.Thinkofthisprojectasareal-worldpuzzlethatyoumighthavetosolve.

ThelogfindSpecificationIwantatoolcalledlogfindthatletsmesearchthroughlogfilesfortext.Thistoolisaspecializedversionofanothertoolcalledgrep,butdesignedonlyforlogfilesonasystem.TheideaisthatIcantype:

logfindzedshaw

And,itwillsearchallthecommonplacesthatlogfilesarestored,andprintouteveryfilethathastheword“zedshaw”init.Thelogfindtoolshouldhavethesebasicfeatures:

1.ThistooltakesanysequenceofwordsandassumesImean“and”forthem.Sologfindzedshawsmartguywillfindallfilesthathavezedshawandsmartandguyinthem.

2.Ittakesanoptionalargumentof-oiftheparametersaremeanttobeorlogic.3.Itloadsthelistofallowedlogfilesfrom~/.logfind.4.Thelistoffilenamescanbeanythingthattheglobfunctionallows.Refertoman3globtoseehowthisworks.Isuggeststartingwithjustaflatlistofexactfiles,andthenaddglobfunctionality.

5.Youshouldoutputthematchinglinesasyouscan,andtrytomatchthemasfastaspossible.That’stheentiredescription.Rememberthatthismaybeveryhard,sotakeitatinybitatatime.Writesomecode,testit,writemore,testthat,andsooninlittlechunksuntilyouhaveitworking.Startwiththesimplestthingthatgetsitworking,andthenslowlyaddtoitandrefineituntileveryfeatureisdone.

Exercise27.CreativeandDefensiveProgramming

YouhavenowlearnedmostofthebasicsofCprogrammingandarereadytostartbecomingaseriousprogrammer.Thisiswhereyougofrombeginnertoexpert,bothwithCandhopefullywithcorecomputerscienceconcepts.Iwillbeteachingyouafewofthecoredatastructuresandalgorithmsthateveryprogrammershouldknow,andthenafewveryinterestingonesI’veusedinrealsoftwareforyears.BeforeIcandothat,Ihavetoteachyousomebasicskillsandideasthatwillhelpyoumakebettersoftware.Exercises27through31willteachyouadvancedconcepts,featuringmoretalkingthancoding.Afterthat,you’llapplywhatyou’velearnedtomakeacorelibraryofusefuldatastructures.ThefirststepingettingbetteratwritingCcode(andreallyanylanguage)istolearnanewmind-setcalleddefensiveprogramming.Defensiveprogrammingassumesthatyouaregoingtomakemanymistakes,andthenattemptstopreventthemateverypossiblestep.Inthisexercise,I’mgoingtoteachyouhowtothinkaboutprogrammingdefensively.

TheCreativeProgrammerMind-SetIt’snotpossibletoshowyouhowtobecreativeinashortexerciselikethis,butIwilltellyouthatcreativityinvolvestakingrisksandbeingopen-minded.Fearwillquicklykillcreativity,sothemind-setIadopt,andmanyprogrammerscopy,isthataccidentsaredesignedtomakeyouunafraidoftakingchancesandlookinglikeanidiot.Here’smymind-set:

•Ican’tmakeamistake.•Itdoesn’tmatterwhatpeoplethink.•Whatevermybraincomesupwithisgoingtobeagreatidea.

Ionlyadoptthismind-settemporarily,andevenhavelittletrickstoturniton.Bydoingthis,Icancomeupwithideas,findcreativesolutions,openmythoughtstooddconnections,andjustgenerallyinventweirdnesswithoutfear.Inthismind-set,I’lltypicallywriteahorriblefirstversionofsomethingjusttogettheideaout.However,whenI’vefinishedmycreativeprototype,Iwillthrowitoutandgetseriousaboutmakingitsolid.Whereotherpeoplemakeamistakeiscarryingthecreativemind-setintotheirimplementationphase.Thisthenleadstoaverydifferent,destructivemind-set:thedarksideofthecreativemind-set:

•It’spossibletowriteperfectsoftware.•Mybraintellsmethetruth,anditcan’tfindanyerrors:Ihavethereforewrittenperfectsoftware.•MycodeiswhoIamandpeoplewhocriticizeitsperfectionarecriticizingme.

Thesearelies.Youwillfrequentlyrunintoprogrammerswhofeelintenseprideaboutwhatthey’vecreated,whichisnatural,butthispridegetsinthewayoftheirabilitytoobjectivelyimprovetheircraft.Becauseofthisprideandattachmenttowhatthey’vewritten,theycancontinuetobelievethatwhattheywriteisperfect.Aslongastheyignoreotherpeople’scriticismoftheircode,theycanprotecttheirfragileegosandneverimprove.Thetricktobeingcreativeandmakingsolidsoftwareistheabilitytoadoptadefensiveprogrammingmind-set.

TheDefensiveProgrammerMind-SetAfteryouhaveaworking,creativeprototypeandyou’refeelinggoodabouttheidea,it’stimetoswitchtobeingadefensiveprogrammer.Thedefensiveprogrammerbasicallyhatesyourcodeandbelievesthesethings:

•Softwarehaserrors.•Youaren’tyoursoftware,yetyou’reresponsiblefortheerrors.•Youcanneverremovetheerrors,onlyreducetheirprobability.

Thismind-setletsyoubehonestaboutyourworkandcriticallyanalyzeitforimprovements.Noticethatitdoesn’tsayyouarefulloferrors?Itsaysyourcodeisfulloferrors.Thisisasignificantthingtounderstandbecauseitgivesyouthepowerofobjectivityforthenextimplementation.Justlikethecreativemind-set,thedefensiveprogrammingmind-sethasadarkside,aswell.Defensiveprogrammersareparanoid,andthisfearpreventsthemfromeverpossiblybeingwrongormakingmistakes.That’sgreatwhenyou’retryingtoberuthlesslyconsistentandcorrect,butit’smurderoncreativeenergyandconcentration.

TheEightDefensiveProgrammerStrategiesOnceyou’veadoptedthismind-set,youcanthenrewriteyourprototypeandfollowasetofeightstrategiestomakeyourcodeassolidaspossible.WhileIworkontherealversion,IruthlesslyfollowthesestrategiesandtrytoremoveasmanyerrorsasIcan,thinkinglikesomeonewhowantstobreakthesoftware.

NeverTrustInputNevertrustthedatayou’regivenandalwaysvalidateit.PreventErrorsIfanerrorispossible,nomatterhowprobable,trytopreventit.FailEarlyandOpenlyFailearly,cleanly,andopenly,statingwhathappened,where,andhowtofixit.

DocumentAssumptionsClearlystatethepre-conditions,post-conditions,andinvariants.PreventionoverDocumentationDon’tdowithdocumentationthatwhichcanbedonewithcodeoravoidedcompletely.

AutomateEverythingAutomateeverything,especiallytesting.SimplifyandClarifyAlwayssimplifythecodetothesmallest,cleanestformthatworkswithoutsacrificingsafety.

QuestionAuthorityDon’tblindlyfolloworrejectrules.Thesearen’ttheonlystrategies,butthey’rethecorethingsIfeelprogrammershavetofocusonwhentryingtomakegood,solidcode.NoticethatIdon’treallysayexactlyhowtodothese.I’llgointoeachoftheseinmoredetail,andsomeoftheexerciseswillactuallycoverthemextensively.

ApplyingtheEightStrategiesTheseideasareallasgreatpop-psychologyplatitudes,buthowdoyouactuallyapplythemtoworkingcode?I’mnowgoingtogiveyouasetofthingstoalwaysdointhisbook’scodethatdemonstrateseachonewithaconcreteexample.Theideasaren’tlimitedtojusttheseexamples,soyoushouldusetheseasaguidetomakingyourowncodemoresolid.

NeverTrustInputLet’slookatanexampleofbaddesignandbetterdesign.Iwon’tsaygooddesignbecausethiscouldbedoneevenbetter.Takealookatthesetwofunctionsthatbothcopyastringandasimplemaintotestoutthebetterone.

ex27_1.c


1#undefNDEBUG2#include"dbg.h"3#include<stdio.h>4#include<assert.h>56/*7*Naivecopythatassumesallinputsarealwaysvalid8*takenfromK&RCandcleanedupabit.9*/10voidcopy(charto[],charfrom[])11{12inti=0;1314//whileloopwillnotendiffromisn't'\0'terminated15while((to[i]=from[i])!='\0'){16++i;17}18}1920/*21*Asaferversionthatchecksformanycommonerrorsusingthe22*lengthofeachstringtocontroltheloopsandtermination.23*/24intsafercopy(intfrom_len,char*from,intto_len,char*to)25{26assert(from!=NULL&&to!=NULL&&"fromandtocan'tbeNULL");27inti=0;28intmax=from_len>to_len-1?to_len-1:from_len;2930//to_lenmusthaveatleast1byte31if(from_len<0||to_len<=0)32return-1;3334for(i=0;i<max;i++){35to[i]=from[i];36}3738to[to_len-1]='\0';3940returni;41}4243intmain(intargc,char*argv[])44{45//carefultounderstandwhywecangetthesesizes46charfrom[]="0123456789";47intfrom_len=sizeof(from);4849//noticethatit's7chars+\050charto[]="0123456";51intto_len=sizeof(to);

5253debug("Copying'%s':%dto'%s':%d",from,from_len,to,to_len);5455intrc=safercopy(from_len,from,to_len,to);56check(rc>0,"Failedtosafercopy.");57check(to[to_len-1]=='\0',"Stringnotterminated.");5859debug("Resultis:'%s':%d",to,to_len);6061//nowtrytobreakit62rc=safercopy(from_len*-1,from,to_len,to);63check(rc==-1,"safercopyshouldfail#1");64check(to[to_len-1]=='\0',"Stringnotterminated.");6566rc=safercopy(from_len,from,0,to);67check(rc==-1,"safercopyshouldfail#2");68check(to[to_len-1]=='\0',"Stringnotterminated.");6970return0;7172error:73return1;74}

ThecopyfunctionistypicalCcodeandit’sthesourceofahugenumberofbufferoverflows.It’sflawedbecauseitassumesthatitwillalwaysreceiveavalid,terminatedCstring(with'\0'),andjustusesawhile-looptoprocessit.Problemis,toensurethatisincrediblydifficult,andifit’snothandledright,itcausesthewhile-looptoloopinfinitely.Acornerstoneofwritingsolidcodeisneverwritingloopsthatcanpossiblyloopforever.Thesafercopyfunctiontriestosolvethisbyrequiringthecallertogivethelengthsofthetwostringsitmustdealwith.Bydoingthis,itcanmakecertainchecksaboutthesestringsthatthecopyfunctioncan’t.Itcancheckthatthelengthsareright,andthatthetostringhasenoughspace,anditwillalwaysterminate.It’simpossibleforthisfunctiontorunonforeverlikethecopyfunction.Thisistheideabehindnevertrustingtheinputsyoureceive.Ifyouassumethatyourfunctionisgoingtogetastringthat’snotterminated(whichiscommon),thenyoucandesignyourfunctionsothatitdoesn’trelyonittoworkproperly.IfyouneedtheargumentstoneverbeNULL,thenyoushouldcheckforthat,too.Ifthesizesshouldbewithinsanelevels,thencheckthat.Yousimplyassumethatwhoeveriscallingyougotitwrong,andthentrytomakeitdifficultforthemtogiveyouanotherbadstate.Thisextendstosoftwareyouwritethatgetsinputfromtheexternaluniverse.Thefamouslastwordsoftheprogrammerare,“Nobody’sgoingtodothat.”I’veseenthemsaythatandthenthenextdaysomeonedoesexactlythat,crashingorhackingtheirapplication.Ifyousaynobodyisgoingtodothat,justthrowinthecodetomakesuretheysimplycan’thackyourapplication.You’llbegladyoudid.Thereisadiminishingreturnonthis,buthere’salistofthingsItrytodoinallofthefunctionsIwriteinC:

•Foreachparameter,identifywhatitspreconditionsare,andwhetherthepreconditionshouldcauseafailureorreturnanerror.Ifyouarewritingalibrary,favorerrorsoverfailures.•Addassertcallsatthebeginningthatcheckforeachfailurepreconditionusingassert(test&&"message");.Thislittlehackdoesthetest,andwhenitfails,theOSwilltypicallyprinttheassertlineforyouthatincludesthatmessage.Thisisveryhelpful

whenyou’retryingtofigureoutwhythatassertisthere.•Fortheotherpreconditions,returntheerrorcodeorusemycheckmacrotogiveanerrormessage.Ididn’tusecheckinthisexamplesinceitwouldconfusethecomparison.•Documentwhythesepreconditionsexistsothatwhenaprogrammerhitstheerror,heorshecanfigureoutifthey’rereallynecessaryornot.•Ifyou’remodifyingtheinputs,makesurethattheyarecorrectlyformedwhenthefunctionexits,orabortiftheyaren’t.•Alwayschecktheerrorcodesoffunctionsyouuse.Forexample,peoplefrequentlyforgettocheckthereturncodesfromfopenorfread,whichcausesthemtousetheresourcesthereturncodesgivedespitetheerror.Thiscausesyourprogramtocrashoropenanavenueforanattack.•Youalsoneedtobereturningconsistenterrorcodessothatyoucandothisforallofyourfunctions.Onceyougetinthishabit,you’llthenunderstandwhymycheckmacrosworkthewaytheydo.

Justdoingthesesimplethingswillimproveyourresourcehandlingandpreventquiteafewerrors.

PreventErrorsInresponsetothepreviousexample,youmighthearpeoplesay,“Well,it’snotverylikelysomeonewillusecopywrong.”Despitethemountainofattacksmadeagainstthisverykindoffunction,somepeoplestillbelievethattheprobabilityofthiserrorisverylow.Probabilityisafunnythingbecausepeopleareincrediblybadatguessingtheprobabilityofanyevent.Peopleare,however,muchbetteratdeterminingifsomethingispossible.Theymightsaytheerrorincopyisnotprobable,buttheycan’tdenythatit’spossible.Thekeyreasonisthatforsomethingtobeprobable,itfirsthastobepossible.Determiningthepossibilityiseasy,sincewecanallimaginesomethinghappening.What’snotsoeasyisdeterminingitsprobabilityafterthat.Isthechancethatsomeonemightusecopywrong20%,10%,or1%?Whoknows?You’dneedtogatherevidence,lookatratesoffailureinmanysoftwarepackages,andprobablysurveyrealprogrammersabouthowtheyusethefunction.Thismeans,ifyou’regoingtopreventerrors,youstillneedtotrytopreventwhat’spossiblebutfirstfocusyourenergiesonwhat’smostprobable.Itmaynotbefeasibletohandleallofthepossiblewaysyoursoftwarecanbebroken,butyouhavetoattemptit.Butatthesametime,ifyoudon’tconstrainyoureffortstothemostprobableevents,thenyou’llbewastingtimeonirrelevantattacks.Here’saprocessfordeterminingwhattopreventinyoursoftware:

•Listallthepossibleerrorsthatcanhappen,nomatterhowprobable(withinreason,ofcourse).Nopointlisting“alienssuckingyourmemoriesouttostealyourpasswords.”•Giveeachpossibleerroraprobabilitythat’sapercentageoftheoperationsthatcanbevulnerable.IfyouarehandlingrequestsfromtheInternet,thenit’sthepercentageofrequeststhatcancausetheerror.Iftheyarefunctioncalls,thenit’swhatpercentageoffunctioncallscancausetheerror.•Calculatetheeffortinnumberofhoursoramountofcodetopreventit.Youcouldalsojustgiveaneasyorhardmetric,oranymetricthatpreventsyoufromworkingontheimpossiblewhenthereareeasierthingstofixstillonthelist.•Rankthembyeffort(lowesttohighest),andprobability(highesttolowest).Thisisnowyour

tasklist.•Preventalloftheerrorsyoucaninthislist,aimingforremovingthepossibility,thenreducingtheprobabilityifyoucan’tmakeitimpossible.•Ifthereareerrorsyoucan’tfix,thendocumentthemsosomeoneelsecanfixthem.

Thislittleprocesswillgiveyouanicelistofthingstodo,butmoreimportantly,keepyoufromworkingonuselessthingswhenthereareothermoreimportantthingstoworkon.Youcanalsobemoreorlessformalwiththisprocess.Ifyou’redoingafullsecurityaudit,thiswillbebetterdonewithawholeteamandanicespreadsheet.Ifyou’rejustwritingafunction,thensimplyreviewthecodeandscratchtheseoutintosomecomments.What’simportantisthatyoustopassumingthaterrorsdon’thappen,andyouworkonremovingthemwhenyoucanwithoutwastingeffort.

FailEarlyandOpenlyIfyouencounteranerrorinCyouhavetwochoices:

•Returnanerrorcode.•Aborttheprocess.

Thisisjusthowitis,sowhatyouneedtodoismakesurethefailureshappenquickly,areclearlydocumented,giveanerrormessage,andareeasyfortheprogrammertoavoid.ThisiswhythecheckmacrosI’vegivenyouworkthewaytheydo.Foreveryerroryoufind,itprintsamessage,thefileandlinenumberwhereithappened,andforcesareturncode.Ifyoujustusemymacros,you’llendupdoingtherightthinganyway.Itendtopreferreturninganerrorcodetoabortingtheprogram.Ifit’scatastrophic,thenIwill,butveryfewerrorsaretrulycatastrophic.AgoodexampleofwhenI’llabortaprogramisifI’mgivenaninvalidpointer,asIdidinsafercopy.Insteadofhavingtheprogrammerexperienceasegmentationfaultexplosionsomewhere,Icatchitrightawayandabort.However,ifit’scommontopassinaNULL,thenI’llprobablychangethattoacheckinsteadsothatthecallercanadaptandkeeprunning.Inlibraries,however,Itrymyhardesttoneverabort.Thesoftwareusingmylibrarycandecideifitshouldabort,andI’lltypicallyabortonlyifthelibraryisverybadlyused.Finally,abigpartofbeingopenabouterrorsisnotusingthesamemessageorerrorcodeformorethanonepossibleerror.Youtypicallyseethiswitherrorsinexternalresources.Alibrarywillreceiveanerroronasocket,andthensimplyreport“badsocket.”Whattheyshoulddoisreturntheerroronthesocketsothatitcanbeproperlydebuggedandfixed.Whendesigningyourerrorreporting,makesureyougiveadifferenterrormessageforthedifferentpossibleerrors.

DocumentAssumptionsIfyou’refollowingalongandusingthisadvice,thenwhatyou’redoingisbuildingacontractofhowyourfunctionsexpecttheworldtobe.You’vecreatedpreconditionsforeachargument,you’vehandledpossibleerrors,andyou’refailingelegantly.Thenextstepistocompletethecontractandaddinvariantsandpostconditions.Aninvariantisaconditionthatmustbeheldtrueinsomestatewhilethefunctionruns.Thisisn’tverycommoninsimplefunctions,butwhenyou’redealingwithcomplexstructures,itbecomesmorenecessary.Agoodexampleofaninvariantisaconditionwhereastructureisalwaysinitializedproperlywhileit’sbeingused.Anotherexamplewouldbethatasorteddatastructureisalwayssortedduringprocessing.

Apostconditionisaguaranteeontheexitvalueorresultofafunctionrunning.Thiscanblendtogetherwithinvariants,butthisissomethingassimpleas“functionalwaysreturns0or-1onerror.”Usuallythesearedocumented,butifyourfunctionreturnsanallocatedresource,youcanaddapostconditionthatcheckstomakesureit’sreturningsomething,andnotNULL.Or,youcanuseNULLtoindicateanerror,sothatyourpostconditionchecksthattheresourceisdeallocatedonanyerrors.InCprogramming,invariantsandpostconditionsareusuallyusedmoreindocumentationthanactualcodeorassertions.Thebestwaytohandlethemistoaddassertcallsfortheonesyoucan,thendocumenttherest.Ifyoudothat,whenpeoplehitanerrortheycanseewhatassumptionsyoumadewhenwritingthefunction.

PreventionoverDocumentationAcommonproblemwhenprogrammerswritecodeisthattheywilldocumentacommonbugratherthansimplyfixit.MyfavoriteiswhentheRubyonRailssystemsimplyassumedthatallmonthshad30days.Calendarsarehard,soratherthanfixit,programmersthrewatinylittlecommentsomewherethatsaidthiswasonpurpose,andthentheyrefusedtofixitforyears.Everytimesomeonewouldcomplain,theywouldblusterandyell,“Butit’sdocumented!”Documentationdoesn’tmatterifyoucanactuallyfixtheproblem,andifthefunctionhasafatalflaw,thenjustdon’tincludeituntilyoucanfixit.InthecaseofRubyonRails,nothavingdatefunctionswouldhavebeenbetterthanincludingpurposefullybrokenonesthatnobodycoulduse.Asyougothroughyourdefensiveprogrammingcleanups,trytofixeverythingyoucan.Ifyoufindyourselfdocumentingmoreandmoreproblemsyoucan’tfix,thenconsiderredesigningthefeatureorsimplyremovingit.Ifyoureallyhavetokeepthishorriblybrokenfeature,thenIsuggestyouwriteit,documentit,andthenfindanewjobbeforeyouareblamedforit.

AutomateEverythingYouareaprogrammer,andthatmeansyourjobisputtingotherpeopleoutofjobswithautomation.Thepinnacleofthisisputtingyourselfoutofajobwithyourownautomation.Obviously,youwon’tcompletelyeliminatewhatyoudo,butifyou’respendingyourwholedayrerunningmanualtestsinyourterminal,thenyourjobisn’tprogramming.YouaredoingQA,andyoushouldautomateyourselfoutofthisQAjobthatyouprobablydon’treallywantanyway.Theeasiestwaytodothisistowriteautomatedtests,orunittests.InthisbookI’mgoingtogetintohowtodothiseasily,butI’llavoidmostofthedogmaaboutwhenyoushouldwritetests.I’llfocusonhowtowritethem,whattotest,andhowtobeefficientatthetesting.Herearecommonthingsprogrammersfailtoautomatewhentheyshould:

•Testingandvalidation•Buildprocesses•Deploymentofsoftware•Systemadministration•Errorreporting

Trytodevotesomeofyourtimetoautomatingthisandyou’llhavemoretimetoworkonthefunstuff.Or,ifthisisfuntoyou,thenmaybeyoushouldworkonsoftwarethatmakesautomatingthesethingseasier.

SimplifyandClarifyTheconceptofsimplicityisaslipperyonetomanypeople,especiallysmartpeople.Theygenerallyconfusecomprehensionwithsimplicity.Iftheyunderstandit,clearlyit’ssimple.Theactualtestofsimplicityiscomparingsomethingwithsomethingelsethatcouldbesimpler.Butyou’llseepeoplewhowritecodegorunningtothemostcomplex,obtusestructurespossiblebecausetheythinkthesimplerversionofthesamethingisdirty.Aloveaffairwithcomplexityisaprogrammingsickness.Youcanfightthisdiseasebyfirsttellingyourself,“Simpleandclearisnotdirty,nomatterwhateveryoneelseisdoing.”Ifeveryoneelseiswritinginsanevisitorpatternsinvolving19classesover12interfaces,andyoucandoitwithtwostringoperations,thenyouwin.Theyarewrong,nomatterhoweleganttheythinktheircomplexmonstrosityis.Here’sthesimplesttestofwhichfunctionisbetter:

•Makesurebothfunctionshavenoerrors.Itdoesn’tmatterhowfastorsimpleafunctionisifithaserrors.•Ifyoucan’tfixone,thenpicktheother.•Dotheyproducethesameresult?Ifnot,thenpicktheonethathastheresultyouneed.•Iftheyproducethesameresult,thenpicktheonethateitherhasfewerfeatures,fewerbranches,oryoujustthinkissimpler.•Makesureyou’renotjustpickingtheonethatismostimpressive.Simpleanddirtybeatscomplexandcleananyday.

You’llnoticethatImostlygiveupattheendandtellyoutouseyourjudgment.Simplicityisironicallyaverycomplexthing,sousingyourtasteasaguideisthebestwaytogo.Justmakesurethatyouadjustyourviewofwhat’s“good”asyougrowandgainmoreexperience.

QuestionAuthorityThefinalstrategyisthemostimportantbecauseitbreaksyououtofthedefensiveprogrammingmind-setandletsyoutransitionintothecreativemind-set.Defensiveprogrammingisauthoritarianandcanbecruel.Thejobofthismind-setistomakeyoufollowrules,becausewithoutthemyou’llmisssomethingorgetdistracted.Thisauthoritarianattitudehasthedisadvantageofdisablingindependentcreativethought.Rulesarenecessaryforgettingthingsdone,butbeingaslavetothemwillkillyourcreativity.Thisfinalstrategymeansyoushouldperiodicallyquestiontherulesyoufollowandassumethattheycouldbewrong,justlikethesoftwareyouarereviewing.WhatIwilltypicallydoisgotakeanonprogrammingbreakandlettherulesgoafterasessionofdefensiveprogramming.ThenI’llbereadytodosomecreativeworkormoredefensivecodingifIneedto.

OrderIsNotImportantThefinalthingI’llsayonthisphilosophyisthatI’mnottellingyoutodothisinastrictorderof“CREATE!DEFEND!CREATE!DEFEND!”Atfirstyoumightwanttodothat,butI’dactuallydoeitherinvaryingamountsdependingonwhatIwantedtodo,andImightevenmeldthemtogetherwithnodefinedboundary.Ialsodon’tthinkonemind-setisbetterthananother,orthatthere’sastrictseparationbetweenthem.Youneedbothcreativityandstrictnesstodoprogrammingwell,soworkonbothifyouwanttoimprove.

ExtraCredit•Thecodeinthebookuptothispoint(andfortherestofit)potentiallyviolatestheserules.Gobackandapplywhatyou’velearnedtooneexercisetoseeifyoucanimproveitorfindbugs.•Findanopensourceprojectandgivesomeofthefilesasimilarcodereview.Submitapatchthatfixesabug.

Exercise28.IntermediateMakefiles

Inthenextthreeexercisesyou’llcreateaskeletonprojectdirectorytouseinbuildingyourCprogramslater.Thisskeletondirectorywillbeusedfortherestofthebook.Inthisexercise,I’llcoverjusttheMakefilesoyoucanunderstandit.Thepurposeofthisstructureistomakeiteasytobuildmedium-sizedprogramswithouthavingtoresorttoconfiguretools.Ifdoneright,youcangetveryfarwithjustGNUmakeandsomesmallshellscripts.

TheBasicProjectStructureThefirstthingtodoismakeac-skeletondirectory,andthenputasetofbasicfilesanddirectoriesinitthatmanyprojectshave.Here’smystarter:

Exercise28Session


$mkdirc-skeleton$cdc-skeleton/$touchLICENSEREADME.mdMakefile$mkdirbinsrctests$cpdbg.hsrc/#thisisfromEx19$ls-ltotal8

-rw-r--r--1zedshawstaff0Mar3116:38LICENSE

-rw-r--r--1zedshawstaff1168Apr117:00Makefile

-rw-r--r--1zedshawstaff0Mar3116:38README.md

drwxr-xr-x2zedshawstaff68Mar3116:38bin

drwxr-xr-x2zedshawstaff68Apr110:07build

drwxr-xr-x3zedshawstaff102Apr316:28src

drwxr-xr-x2zedshawstaff68Mar3116:38tests

$ls-lsrctotal8

-rw-r--r--1zedshawstaff982Apr316:28dbg.h

$

Attheendyouseemedoals-lsothatyoucanseethefinalresults.Here’sabreakdown:

LICENSEIfyoureleasethesourceofyourprojects,you’llwanttoincludealicense.Ifyoudon’t,though,thecodeiscopyrightbyyouandnobodyelsehasrightstoitbydefault.

README.mdBasicinstructionsforusingyourprojectgohere.Itendsin.mdsothatitwillbeinterpretedasmarkdown.

MakefileThemainbuildfilefortheproject.bin/Whereprogramsthatuserscanrungo.Thisisusuallyempty,andtheMakefilewillcreateitifit’snotthere.

build/Wherelibrariesandotherbuildartifactsgo.Alsoempty,andtheMakefilewillcreateitifit’snotthere.

src/Wherethesourcecodegoes,usually.cand.hfiles.

tests/Whereautomatedtestsgo.src/dbg.hIcopiedthedbg.hfromExercise19intosrc/forlater.

I’llnowbreakdowneachofthecomponentsofthisskeletonprojectsothatyoucanunderstandhowitworks.

Makefile

ThefirstthingI’llcoveristheMakefile,becausefromthatyoucanunderstandhoweverythingelseworks.TheMakefileinthisexerciseismuchmoredetailedthanonesyou’veusedsofar,soI’llbreakitdownafteryoutypeitin:

Makefile


1CFLAGS=-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUG$(OPTFLAGS)2LIBS=-ldl$(OPTLIBS)3PREFIX?=/usr/local45SOURCES=$(wildcardsrc/**/*.csrc/*.c)6OBJECTS=$(patsubst%.c,%.o,$(SOURCES))78TEST_SRC=$(wildcardtests/*_tests.c)9TESTS=$(patsubst%.c,%,$(TEST_SRC))1011TARGET=build/libYOUR_LIBRARY.a12SO_TARGET=$(patsubst%.a,%.so,$(TARGET))1314#TheTargetBuild15all:$(TARGET)$(SO_TARGET)tests1617dev:CFLAGS=-g-Wall-Isrc-Wall-Wextra$(OPTFLAGS)18dev:all1920$(TARGET):CFLAGS+=-fPIC21$(TARGET):build$(OBJECTS)22arrcs$@$(OBJECTS)23ranlib$@24$(SO_TARGET):$(TARGET)$(OBJECTS)25$(CC)-shared-o$@$(OBJECTS)2627build:28@mkdir-pbuild29@mkdir-pbin3031#TheUnitTests32.PHONY:tests33tests:CFLAGS+=$(TARGET)34tests:$(TESTS)35sh./tests/runtests.sh3637#TheCleaner38clean:39rm-rfbuild$(OBJECTS)$(TESTS)40rm-ftests/tests.log41find.-name"*.gc*"-execrm{}\;42rm-rf`find.-name"*.dSYM"-print`4344#TheInstall

45install:all46install-d$(DESTDIR)/$(PREFIX)/lib/47install$(TARGET)$(DESTDIR)/$(PREFIX)/lib/4849#TheChecker50check:[email protected]@egrep'[^_.>a-zA-Z0-9](str(n?cpy|n?cat|xfrm|n?dup|str|pbrk|tok|_)\53|stpn?cpy|a?sn?printf|byte_)'$(SOURCES)||true

RememberthatyouneedtoconsistentlyindenttheMakefilewithtabcharacters.Yourtexteditorshouldknowthatanddotherightthing.Ifitdoesn’t,getadifferenttexteditor.Noprogrammershoulduseaneditorthatfailsatsomethingsosimple.

TheHeaderThisMakefileisdesignedtobuildalibraryreliablyonalmostanyplatformusingspecialfeaturesofGNUmake.We’llbeworkingonthislibrarylater,soI’llbreakdowneachpartinsections,startingwiththeheader.

Makefile:1ThesearetheusualCFLAGSthatyousetinallofyourprojects,alongwithafewothersthatmaybeneededtobuildlibraries.Youmayneedtoadjustthesefordifferentplatforms.NoticetheOPTFLAGSvariableattheendthatletspeopleaugmentthebuildoptionsasneeded.

Makefile:2Theseoptionsareusedwhenlinkingalibrary.SomeoneelsecanthenaugmentthelinkingoptionsusingtheOPTLIBSvariable.

Makefile:3ThiscodesetsanoptionalvariablecalledPREFIXthatwillonlyhavethisvalueifthepersonrunningtheMakefiledidn’talreadygiveaPREFIXsetting.That’swhatthe?=does.

Makefile:5ThisfancylineofawesomenessdynamicallycreatestheSOURCESvariablebydoingawildcardsearchforall*.cfilesinthesrc/directory.Youhavetogivebothsrc/**/*.candsrc/*.csothatGNUmakewillincludethefilesinsrcandthefilesbelowit.

Makefile:6Onceyouhavethelistofsourcefiles,youcanthenusethepatsubsttotaketheSOURCESlistof*.cfilesandmakeanewlistofalltheobjectfiles.Youdothisbytellingpatsubsttochangeall%.cextensionsto%.o,andthenthoseextensionsareassignedtoOBJECTS.

Makefile:8We’reusingthewildcardagaintofindallofthetestsourcefilesfortheunittests.Theseareseparatefromthelibrary’ssourcefiles.

Makefile:9Then,we’reusingthesamepatsubsttricktodynamicallygetalltheTESTtargets.Inthiscase,I’mstrippingawaythe.cextensionsothatafullprogramwillbemadewiththesamename.Previously,Ihadreplacedthe.cwith{.o}soanobjectfileiscreated.

Makefile:11Finally,wesaytheultimatetargetisbuild/libYOUR_LIBRARY.a,whichyouwillchangetobewhateverlibraryyou’reactuallytryingtobuild.

ThiscompletesthetopoftheMakefile,butIshouldexplainwhatImeanby“letspeopleaugmentthebuild.”WhenyourunMake,youcandothis:Clickheretoviewcodeimage

#WARNING!Justademonstration,won'treallyworkrightnow.#thisinstallsthelibraryinto/tmp$makePREFIX=/tmpinstall#thistellsittoaddpthreads

$makeOPTFLAGS=-pthread

IfyoupassinoptionsthatmatchthesamekindofvariablesyouhaveinyourMakefile,thenthosewillshowupinyourbuild.YoucanthenusethistochangehowtheMakefileruns.ThefirstvariablealtersthePREFIXsothatitinstallsinto/tmpinstead.ThesecondonesetsOPTFLAGSsothatthe-pthreadoptionispresent.

TheTargetBuildContinuingwiththebreakdownoftheMakefile,I’mactuallybuildingtheobjectfilesandtargets:

Makefile:14Rememberthatthefirsttargetiswhatmakerunsbydefaultwhennotargetisgiven.Inthis,it’scalledall:anditgives$(TARGET)testsasthetargetstobuild.LookupattheTARGETvariableandyouseethat’sthelibrary,soall:willfirstbuildthelibrary.TheteststargetisfurtherdownintheMakefileandbuildstheunittests.

Makefile:16Here’sanothertargetformaking“developerbuilds”thatintroducesatechniqueforchangingoptionsforjustonetarget.IfIdoa“devbuild,”IwanttheCFLAGStoincludeoptionslike-Wextrathatareusefulforfindingbugs.Ifyouplacethemonthetargetlineasoptionslikethis,thengiveanotherlinethatsaystheoriginaltarget(inthiscaseall),thenitwillchangetheoptionsyouset.Iusethisforsettingdifferentflagsondifferentplatformsthatneedit.

Makefile:19ThisbuildstheTARGETlibrary,whateverthatis.Italsousesthesametrickfromline15,givingatargetwithjustoptionsandwaystoalterthemforthisrun.Inthiscase,I’madding-fPICjustforthelibrarybuild,usingthe+=syntaxtoadditon.

Makefile:20Nowweseetherealtarget,whereIsayfirstmakethebuilddirectory,andthencompilealloftheOBJECTS.

Makefile:21ThisrunsthearcommandthatactuallymakestheTARGET.Thesyntax$@$(OBJECTS)isawayofsaying,“putthetargetforthisMakefilesourcehereandalltheOBJECTSafterthat.”Inthiscase,the$@mapsbacktothe$(TARGET)online19,whichmapstobuild/libYOUR_LIBRARY.a.Itseemslikealottokeeptrackofinthisindirection,anditcanbe,butonceyougetitworking,youjustchangeTARGETatthetopandbuildawholenewlibrary.

Makefile:22Finally,tomakethelibrary,yourunranlibontheTARGETandit’sbuilt.Makefile:23-24Thisjustmakesthebuild/orbin/directoriesiftheydon’texist.Thisisthenreferencedfromline19whenitgivesthebuildtargettomakesurethebuild/directoryismade.

Younowhaveallofthestuffyouneedtobuildthesoftware,sowe’llcreateawaytobuildandrununitteststodoautomatedtesting.

TheUnitTestsCisdifferentfromotherlanguagesbecauseit’seasiertocreateonetinylittleprogramforeachthingyou’retesting.Sometestingframeworkstrytoemulatethemoduleconceptotherlanguageshaveanddodynamicloading,butthisdoesn’tworkwellinC.It’salsounnecessary,becauseyoucanjustmakeasingleprogramthat’srunforeachtestinstead.I’llcoverthispartoftheMakefile,andthenlateryou’llseethecontentsofthetests/directory

thatmakeitactuallywork.Makefile:29Ifyouhaveatargetthat’snotreal,butthereisadirectoryorfilewiththatname,thenyouneedtotagthetargetwith.PHONY:somakewillignorethefileandalwaysrun.

Makefile:30IusethesametrickformodifyingtheCFLAGSvariabletoaddtheTARGETtothebuildsothateachofthetestprogramswillbelinkedwiththeTARGETlibrary.Inthiscase,itwilladdbuild/libYOUR_LIBRARY.atothelinking.

Makefile:31ThenIhavetheactualtests:target,whichdependsonalloftheprogramslistedintheTESTSvariablethatwecreatedintheheader.Thisonelineactuallysays,“Make,usewhatyouknowaboutbuildingprogramsandthecurrentCFLAGSsettingstobuildeachprograminTESTS.”

Makefile:32Finally,whenalloftheTESTSarebuilt,there’sasimpleshellscriptI’llcreatelaterthatknowshowtorunthemallandreporttheiroutput.Thislineactuallyrunsitsoyoucanseethetestresults.

Fortheunittestingtowork,you’llneedtocreatealittleshellscriptthatknowshowtoruntheprograms.Goaheadandcreatethistests/runtests.shscript:

runtests.sh


1echo"Runningunittests:"23foriintests/*_tests4do5iftest-f$i6then7if$VALGRIND./$i2>>tests/tests.log8then9echo$iPASS10else11echo"ERRORintest$i:here'stests/tests.log"12echo"------"13tailtests/tests.log14exit115fi16fi17done1819echo""

I’llbeusingthislaterwhenIcoverhowunittestswork.

TheCleanerInowhavefullyworkingunittests,sonextupismakingthingscleanwhenIneedtoreseteverything.

Makefile:38Theclean:targetstartsthingsoffwhenweneedtocleanuptheproject.Makefile:39-42Thiscleansoutmostofthejunkthatvariouscompilersandtoolsleavebehind.Italsogetsridofthebuild/directoryandusesatrickattheendtocleanlyerasetheweird*.dSYMdirectoriesthatApple’sXCodeleavesbehindfordebuggingpurposes.

Ifyourunintojunkthatyouneedtocleanout,simplyaugmentthelistofthingsbeingdeletedinthis

target.

TheInstallAfterthat,I’llneedawaytoinstalltheproject,andforaMakefilethat’sbuildingalibrary,IjustneedtoputsomethinginthecommonPREFIXdirectory,usually/usr/local/lib.

Makefile:45Thismakesinstall:dependontheall:target,sothatwhenyourunmakeinstall,itwillbesuretobuildeverything.

Makefile:46Ithenusetheprograminstalltocreatethetargetlibdirectoryifitdoesn’texist.Inthiscase,I’mtryingtomaketheinstallasflexibleaspossiblebyusingtwovariablesthatareconventionsforinstallers.DESTDIRishandedtomakebyinstallers,whichdotheirbuildsinsecureoroddlocations,tobuildpackages.PREFIXisusedwhenpeoplewanttheprojecttobeinstalledinsomeplaceotherthan/usr/local.

Makefile:47Afterthat,I’mjustusinginstalltoactuallyinstallthelibrarywhereitneedstogo.Thepurposeoftheinstallprogramistomakesurethingshavetherightpermissionsset.Whenyourunmakeinstall,youusuallyhavetodoitastherootuser,sothetypicalbuildprocessismake&&sudomakeinstall.

TheCheckerTheverylastpartofthisMakefileisabonusthatIincludeinmyCprojectstohelpmedigoutanyattemptstousethebadfunctionsinC.Thesearenamelythestringfunctionsandotherunprotectedbufferfunctions.

Makefile:50Thissetsavariablethat’sabigregexlookingforbadfunctionslikestrcpy.Makefile:51Thecheck:targetallowsyoutorunacheckwheneveryouneedto.Makefile:52Thisisjustawaytoprintamessage,butdoing@echotellsmaketonotprintthecommand,justitsoutput.

Makefile:53Runtheegrepcommandonthesourcefilestolookforanybadpatterns.The||trueattheendisawaytopreventmakefromthinkingthategrepfailedifitdoesn’tfinderrors.

Whenyourunthis,itwillhavetheoddeffectofreturninganerrorwhenthere’snothingbadgoingon.

WhatYouShouldSeeIhavetwomoreexercisestogobeforeI’mdonebuildingtheprojectskeletondirectory,buthere’smetestingoutthefeaturesoftheMakefile.

Exercise28Session


$makecleanrm-rfbuild

rm-ftests/tests.log

find.-name"*.gc*"-execrm{}\;

rm-rf`find.-name"*.dSYM"-print`

$makecheck$make

WhenIruntheclean:target,itworks,butbecauseIdon’thaveanysourcefilesinthesrc/directory,noneoftheothercommandsreallywork.I’llfinishthatupinthenextexercises.

ExtraCredit•TrytogettheMakefiletoactuallyworkbyputtingasourceandheaderfileinsrc/andmakingthelibrary.Youshouldn’tneedamainfunctioninthesourcefile.•Researchwhatfunctionsthecheck:targetislookingforintheBADFUNCSregularexpressionthatit’susing.•Ifyoudon’tdoautomatedunittesting,thengoreadaboutitsoyou’repreparedlater.

Exercise29.LibrariesandLinking

AcentralpartofanyCprogramistheabilitytolinkittolibrariesthatyourOSprovides.Linkingishowyougetadditionalfeaturesforyourprogramthatsomeoneelsecreatedandpackagedonthesystem.You’vebeenusingsomestandardlibrariesthatareautomaticallyincluded,butI’mgoingtoexplainthedifferenttypesoflibrariesandwhattheydo.Firstoff,librariesarepoorlydesignedineveryprogramminglanguage.Ihavenoideawhy,butitseemslanguagedesignersthinkoflinkingassomethingtheyjustslaponlater.Librariesareusuallyconfusing,hardtodealwith,can’tdoversioningright,andendupbeinglinkeddifferentlyeverywhere.Cisnodifferent,butthewaylinkingandlibrariesaredoneinCisanartifactofhowtheUNIXoperatingsystemandexecutableformatsweredesignedyearsago.LearninghowClinksthingshelpsyouunderstandhowyourOSworksandhowitrunsyourprograms.Tostartoff,therearetwobasictypesoflibraries:

staticYoumadeoneofthesewhenyouusedarandranlibtocreatethelibYOUR_LIBRARY.ainthelastexercise.Thiskindoflibraryisnothingmorethanacontainerforasetof.oobjectfilesandtheirfunctions,andyoucantreatitlikeonebig.ofilewhenbuildingyourprograms.

dynamicThesetypicallyendin.so,.dlloraboutonemillionotherendingsonOSX,dependingontheversionandwhohappenedtobeworkingthatday.Seriouslythough,OSXadds.dylib,.bundle,and.frameworkwithnotmuchdistinctionamongthethree.Thesefilesarebuiltandthenplacedinacommonlocation.Whenyourunyourprogram,theOSdynamicallyloadsthesefilesandlinksthemtoyourprogramonthefly.

Itendtolikestaticlibrariesforsmall-tomedium-sizedprojects,becausetheyareeasiertodealwithandworkonmoreoperatingsystems.IalsoliketoputallofthecodeIcanintoastaticlibrarysothatIcanthenlinkittounittestsandtothefileprogramsasneeded.Dynamiclibrariesaregoodforlargersystems,whenspaceistight,orifyouhavealargenumberofprogramsthatusecommonfunctionality.Inthiscase,youdon’twanttostaticallylinkallofthecodeforthecommonfeaturestoeveryprogram,soyouputitinadynamiclibrarysothatitisloadedonlyonceforallofthem.Inthepreviousexercise,Ilaidouthowtomakeastaticlibrary(a.afile),andthat’swhatI’lluseintherestofthebook.Inthisexercise,I’mgoingtoshowyouhowtomakeasimple.solibrary,andhowtodynamicallyloaditwiththeUNIXdlopensystem.I’llhaveyoudothismanuallysothatyouunderstandeverythingthat’sactuallyhappening,thenforextracredityou’llusethec-skeletonskeletontocreateit.

DynamicallyLoadingaSharedLibraryTodothis,Iwillcreatetwosourcefiles:Onewillbeusedtomakealibex29.solibrary,theotherwillbeaprogramcalledex29thatcanloadthislibraryandrunfunctionsfromit.

libex29.c


1#include<stdio.h>2#include<ctype.h>3#include"dbg.h"456intprint_a_message(constchar*msg)7{8printf("ASTRING:%s\n",msg);910return0;11}121314intuppercase(constchar*msg)15{16inti=0;1718//BUG:\0terminationproblems19for(i=0;msg[i]!='\0';i++){20printf("%c",toupper(msg[i]));21}2223printf("\n");2425return0;26}2728intlowercase(constchar*msg)29{30inti=0;3132//BUG:\0terminationproblems33for(i=0;msg[i]!='\0';i++){34printf("%c",tolower(msg[i]));35}3637printf("\n");3839return0;40}4142intfail_on_purpose(constchar*msg)43{44return1;45}

There’snothingfancyinthere,althoughtherearesomebugsI’mleavinginonpurposetoseeifyou’vebeenpayingattention.You’llfixthoselater.Whatwewanttodoisusethefunctionsdlopen,dlsym,anddlclosetoworkwiththeabovefunctions.

ex29.c


1#include<stdio.h>2#include"dbg.h"3#include<dlfcn.h>45typedefint(*lib_function)(constchar*data);6

7intmain(intargc,char*argv[])8{9intrc=0;10check(argc==4,"USAGE:ex29libex29.sofunctiondata");1112char*lib_file=argv[1];13char*func_to_run=argv[2];14char*data=argv[3];1516void*lib=dlopen(lib_file,RTLD_NOW);17check(lib!=NULL,"Failedtoopenthelibrary%s:%s",lib_file,18dlerror());1920lib_functionfunc=dlsym(lib,func_to_run);21check(func!=NULL,22"Didnotfind%sfunctioninthelibrary%s:%s",func_to_run,23lib_file,dlerror());2425rc=func(data);26check(rc==0,"Function%sreturn%dfordata:%s",func_to_run,27rc,data);2829rc=dlclose(lib);30check(rc==0,"Failedtoclose%s",lib_file);3132return0;3334error:35return1;36}

I’llnowbreakthisdownsoyoucanseewhat’sgoingoninthissmallbitofusefulcode:ex29.c:5I’llusethisfunctionpointerdefinitionlatertocallfunctionsinthelibrary.Thisisnothingnew,butmakesureyouunderstandwhatit’sdoing.

ex29.c:17Aftertheusualsetupforasmallprogram,Iusethedlopenfunctiontoloadupthelibrarythat’sindicatedbylib_file.Thisfunctionreturnsahandlethatweuselater,whichworksalotlikeopeningafile.

ex29.c:18Ifthere’sanerror,Idotheusualcheckandexit,butnoticeatthenendthatI’musingdlerrortofindoutwhatthelibrary-relatederrorwas.

ex29.c:20Iusedlsymtogetafunctionoutofthelibbyitsstringnameinfunc_to_run.Thisisthepowerfulpart,sinceI’mdynamicallygettingapointertoafunctionbasedonastringIgotfromthecommandlineargv.

ex29.c:23Ithencallthefuncfunctionthatwasreturned,andcheckitsreturnvalue.ex29.c:26Finally,IclosethelibraryupjustlikeIwouldafile.Usually,youkeeptheseopenthewholetimetheprogramisrunning,soclosingitattheendisn’tasuseful,butI’mdemonstratingithere.

WhatYouShouldSeeNowthatyouknowwhatthisfiledoes,here’sashellsessionofmebuildingthelibex29.so,ex29andthenworkingwithit.Followalongsoyoulearnhowthesethingsaremanuallybuilt.

Exercise29Session


#compilethelibfileandmakethe.so#youmayneed-fPIChereonsomeplatforms.addthatifyougetanerror$cc-clibex29.c-olibex29.o$cc-shared-olibex29.solibex29.o

#maketheloaderprogram$cc-Wall-g-DNDEBUGex29.c-ldl-oex29

#tryitoutwithsomethingsthatwork$ex29./libex29.soprint_a_message"hellothere"-bash:ex29:commandnotfound

$./ex29./libex29.soprint_a_message"hellothere"ASTRING:hellothere

$./ex29./libex29.souppercase"hellothere"HELLOTHERE

$./ex29./libex29.solowercase"HELLOtHeRe"hellothere

$./ex29./libex29.sofail_on_purpose"ifail"[ERROR](ex29.c:23:errno:None)Functionfail_on_purposereturn1for\

data:ifail

#trytogiveitbadargs$./ex29./libex29.sofail_on_purpose[ERROR](ex29.c:11:errno:None)USAGE:ex29libex29.sofunctiondata

#trycallingafunctionthatisnotthere$./ex29./libex29.soadfasfasdfasdfadff[ERROR](ex29.c:20:errno:None)Didnotfindadfasfasdf

functioninthelibrarylibex29.so:dlsym(0x1076009b0,adfasfasdf):\symbolnotfound

#tryloadinga.sothatisnotthere$./ex29./libex.soadfasfasdfasdfadfas[ERROR](ex29.c:17:errno:Nosuchfileordirectory)Failedtoopen

thelibrarylibex.so:dlopen(libex.so,2):imagenotfound$

OnethingthatyoumayrunintoisthateveryOS,everyversionofeveryOS,andeverycompileroneveryversionofeveryOS,seemstowanttochangethewayyoubuildasharedlibraryeverytimesomenewprogrammerthinksit’swrong.IfthelineIusetomakethelibex29.sofileiswrong,thenletmeknowandI’lladdsomecommentsforotherplatforms.

Warning!Sometimesyou’lldowhatyouthinkisnormal,andrunthiscommandcc-Wall-g-DNDEBUG-ldlex29.c-oex29thinkingeverythingwillwork,butnope.Yousee,onsomeplatformstheorderofwherelibrariesgomakesthemworkornot,andfornorealreason.InDebianorUbuntu,youhavetodocc-Wall-g-DNDEBUGex29.c-ldl-oex29fornoreasonatall.It’sjustthewayitis.SosincethisworksonOSXI’mdoingithere,butinthefuture,ifyoulinkagainstadynamiclibraryanditcan’tfindafunction,tryshufflingthingsaround.Theirritationhereisthere’sanactualplatformdifferenceonnothingmorethantheorderof

commandlinearguments.Onnorationalplanetshouldputtingan-ldlatonepositionbedifferentfromanother.It’sanoption,andhavingtoknowthesethingsisincrediblyannoying.

HowtoBreakItOpenlibex29.soandedititwithaneditorthatcanhandlebinaryfiles.Changeacoupleofbytes,thencloseitlibex29.so.Trytoseeifyoucangetthedlopenfunctiontoloaditeventhoughyou’vecorruptedit.

ExtraCredit•WereyoupayingattentiontothebadcodeIhaveinthelibex29.cfunctions?Doyouseehow,eventhoughIuseafor-looptheystillcheckfor'\0'endings?Fixthissothatthefunctionsalwaystakealengthforthestringtoworkwithinsidethefunction.•Takethec-skeletonskeleton,andcreateanewprojectforthisexercise.Putthelibex29.cfileinthesrc/directory.ChangetheMakefilesothatitbuildsthisasbuild/libex29.so.•Taketheex29.cfileandputitintests/ex29_tests.csothatitrunsasaunittest.Makethisallwork,whichmeansthatyou’llhavetochangeitsothatitloadsthebuild/libex29.sofileandrunstestssimilartowhatIdidmanuallyabove.•Readthemandlopendocumentationandreadaboutalloftherelatedfunctions.TrysomeoftheotheroptionstodlopenbesideRTLD_NOW.

Exercise30.AutomatedTesting

AutomatedtestingisusedfrequentlyinotherlanguageslikePythonandRuby,butrarelyusedinC.PartofthereasoncomesfromthedifficultyofautomaticallyloadingandtestingpiecesofCcode.Inthischapter,we’llcreateaverysmalltestingframeworkandgetyourskeletondirectorytobuildanexampletestcase.TheframeworkI’mgoingtouse,andyou’llincludeinyourc-skeletonskeleton,iscalledminunitwhichstartedwithatinysnippetofcodebyJeraDesign.Ievolveditfurther,tobethis:

minunit.h


1#undefNDEBUG2#ifndef_minunit_h3#define_minunit_h45#include<stdio.h>6#include<dbg.h>7#include<stdlib.h>89#definemu_suite_start()char*message=NULL1011#definemu_assert(test,message)if(!(test)){\12log_err(message);returnmessage;}13#definemu_run_test(test)debug("\n-----%s",""#test);\14message=test();tests_run++;if(message)returnmessage;1516#defineRUN_TESTS(name)intmain(intargc,char*argv[]){\17argc=1;\18debug("-----RUNNING:%s",argv[0]);\19printf("----\nRUNNING:%s\n",argv[0]);\20char*result=name();\21if(result!=0){\22printf("FAILED:%s\n",result);\23}\24else{\25printf("ALLTESTSPASSED\n");\26}\27printf("Testsrun:%d\n",tests_run);\28exit(result!=0);\29}3031inttests_run;3233#endif

There’spracticallynothingleftoftheoriginal,sincenowI’musingthedbg.hmacrosandalargemacrothatIcreatedattheendfortheboilerplatetestrunner.Evenwiththistinyamountofcode,we’llcreateafullyfunctioningunittestsystemthatyoucanuseinyourCcodeonceit’scombinedwithashellscripttorunthetests.

WiringUptheTestFrameworkTocontinuethisexercise,youshouldhaveyoursrc/libex29.cworking.YoushouldhavealsocompletedtheExercise29ExtraCredittogettheex29.cloaderprogramtoproperlyrun.InExercise29,Iaskyoutomakeitworklikeaunittest,butI’mgoingtostartoverandshowyouhowtodothatwithminunit.h.Thefirstthingtodoiscreateasimpleemptyunittestname,tests/libex29_tests.cwiththisinit:

ex30.c


1#include"minunit.h"23char*test_dlopen()4{56returnNULL;7}89char*test_functions()10{1112returnNULL;13}1415char*test_failures()16{1718returnNULL;19}2021char*test_dlclose()22{2324returnNULL;25}2627char*all_tests()28{29mu_suite_start();3031mu_run_test(test_dlopen);32mu_run_test(test_functions);33mu_run_test(test_failures);34mu_run_test(test_dlclose);3536returnNULL;37}3839RUN_TESTS(all_tests);

ThiscodeisdemonstratingtheRUN_TESTSmacrointests/minunit.handhowtousetheothertestrunnermacros.Ihavetheactualtestfunctionsstubbedoutsothatyoucanseehowtostructureaunittest.I’llbreakthisfiledownfirst:

libex29_tests.c:1Thisincludestheminunit.hframework.

libex29_tests.c:3-7Afirsttest.Testsarestructuredsothattheytakenoargumentsandreturnachar*that’sNULLonsuccess.Thisisimportantbecausetheothermacroswillbeusedtoreturnanerrormessagetothetestrunner.

libex29_tests.c:9-25Thesearemoreteststhatarethesameasthefirst.libex29_tests.c:27Therunnerfunctionthatwillcontrolalloftheothertests.Ithasthesameformasanyothertestcase,butitgetsconfiguredwithsomeadditionalgear.

libex29_tests.c:28Thissetsupsomecommonstuffforatestwithmu_suite_start.libex29_tests.c:30Thisishowyousaywhatteststorun,usingthemu_run_testmacro.libex29_tests.c:35Afteryousaywhatteststorun,youthenreturnNULLjustlikeanormaltestfunction.

libex29_tests.c:38Finally,youjustusethebigRUN_TESTSmacrotowireupthemainmethodwithallofthegoodies,andtellittoruntheall_testsstarter.

That’sallthereistorunningatest,andnowyoushouldtrygettingjustthistorunwithintheprojectskeleton.Here’swhatitlookslikewhenIdoit:

Exercise30Session

notprintable

IfirstdidamakecleanandthenIranthebuild,whichremadethetemplatelibYOUR_LIBRARY.aandlibYOUR_LIBRARY.sofiles.RememberthatyoudidthisintheExtraCreditforExercise29,butjustincaseyoudidn’tgetit,here’sthedifffortheMakefileI’musingnow:

ex30.Makefile.diff


diff--gita/code/c-skeleton/Makefileb/code/c-skeleton/Makefile

index135d538..21b92bf100644

---a/code/c-skeleton/Makefile+++b/code/c-skeleton/Makefile@@-9,9+9,10@@TEST_SRC=$(wildcardtests/*_tests.c)

TESTS=$(patsubst%.c,%,$(TEST_SRC))

TARGET=build/libYOUR_LIBRARY.a+SO_TARGET=$(patsubst%.a,%.so,$(TARGET))

#TheTargetBuild-all:$(TARGET)tests+all:$(TARGET)$(SO_TARGET)tests

dev:CFLAGS=-g-Wall-Isrc-Wall-Wextra$(OPTFLAGS)dev:all@@-21,6+22,9@@$(TARGET):build$(OBJECTS)

arrcs$@$(OBJECTS)ranlib$@

+$(SO_TARGET):$(TARGET)$(OBJECTS)+$(CC)-shared-o$@$(OBJECTS)+build:

@mkdir-pbuild@mkdir-pbin

Withthosechangesyoushouldnowbebuildingeverythingandfinallybeabletofillintheremainingunittestfunctions:

libex29_tests.c


1#include"minunit.h"2#include<dlfcn.h>34typedefint(*lib_function)(constchar*data);5char*lib_file="build/libYOUR_LIBRARY.so";6void*lib=NULL;78intcheck_function(constchar*func_to_run,constchar*data,9intexpected)10{11lib_functionfunc=dlsym(lib,func_to_run);12check(func!=NULL,13"Didnotfind%sfunctioninthelibrary%s:%s",func_to_run,14lib_file,dlerror());1516intrc=func(data);17check(rc==expected,"Function%sreturn%dfordata:%s",18func_to_run,rc,data);1920return1;21error:22return0;23}2425char*test_dlopen()26{27lib=dlopen(lib_file,RTLD_NOW);28mu_assert(lib!=NULL,"Failedtoopenthelibrarytotest.");2930returnNULL;31}3233char*test_functions()34{35mu_assert(check_function("print_a_message","Hello",0),36"print_a_messagefailed.");37mu_assert(check_function("uppercase","Hello",0),38"uppercasefailed.");39mu_assert(check_function("lowercase","Hello",0),40"lowercasefailed.");4142returnNULL;43}4445char*test_failures()46{47mu_assert(check_function("fail_on_purpose","Hello",1),48"fail_on_purposeshouldfail.");4950returnNULL;51}52

53char*test_dlclose()54{55intrc=dlclose(lib);56mu_assert(rc==0,"Failedtocloselib.");5758returnNULL;59}6061char*all_tests()62{63mu_suite_start();6465mu_run_test(test_dlopen);66mu_run_test(test_functions);67mu_run_test(test_failures);68mu_run_test(test_dlclose);6970returnNULL;71}7273RUN_TESTS(all_tests);

Hopefullybynowyoucanfigureoutwhat’sgoingon,sincethere’snothingnewinthisexceptforthecheck_functionfunction.ThisisacommonpatternwhereIuseachunkofcoderepeatedly,andthensimplyautomateitbyeithercreatingafunctionoramacroforit.Inthiscase,I’mgoingtorunfunctionsinthe.sothatIload,soIjustmadealittlefunctiontodoit.

ExtraCredit•Thisworksbutit’sprobablyabitmessy.Cleanthec-skeletondirectoryupsothatithasallofthesefiles,butremoveanyofthecoderelatedtoExercise29.Youshouldbeabletocopythisdirectoryoverandkick-startnewprojectswithoutmuchediting.•Studytheruntests.sh,andthengoreadaboutbashsyntaxsoyouknowwhatitdoes.DoyouthinkyoucouldwriteaCversionofthisscript?

Exercise31.CommonUndefinedBehavior

Atthispointinthebook,it’stimetointroduceyoutothemostcommonkindsofUBthatyouwillencounter.Chas191behaviorsthatthestandardscommitteehasdecidedaren’tdefinedbythestandard,andthereforeanythinggoes.Someofthesebehaviorsarelegitimatelynotthecompiler ’sjob,butthevastmajorityaresimplylazycapitulationsbythestandardscommitteethatcauseannoyances,orworse,defects.Anexampleoflaziness:

Anunmatched“or”characterisencounteredonalogicalsourcelineduringtokenization.Inthisinstance,theC99standardactuallyallowsacompilerwritertofailataparsingtaskthatajuniorincollegecouldgetright.Whyisthis?Whoknows,butmostlikelysomeoneonthestandardscommitteewasworkingonaCcompilerwiththisdefectandmanagedtogetthisinthestandardratherthanfixtheircompiler.Or,asIsaid,simplelaziness.ThecruxoftheissuewithUBisthedifferencebetweentheCabstractmachine,definedinthestandardandrealcomputers.TheCstandarddescribestheClanguageaccordingtoastrictlydefinedabstractmachine.Thisisaperfectlyvalidwaytodesignalanguage,exceptwheretheCstandardgoeswrong:Itdoesn’trequirecompilerstoimplementthisabstractmachineandenforceitsspecification.Instead,acompilerwritercancompletelyignoretheabstractmachinein191instancesofthestandard.Itshouldreallybecalledan“abstractmachine,but”,asin,“It’sastrictlydefinedabstractmachine,but...”Thisallowsthestandardscommitteeandcompilerimplementerstohavetheircakeandeatit,too.Theycanhaveastandardthatisfullofomissions,laxspecification,anderrors,butwhenyouencounteroneofthese,theycanpointattheabstractmachineandsimplysayintheirbestrobotvoice,“THEABSTRACTMACHINEISALLTHATMATTERS.YOUDONOTCONFORM!”Yet,in191instancesthatcompilerwritersdon’thavetoconform,youdo.Youareasecondclasscitizen,eventhoughthelanguageisreallywrittenforyoutouse.Thismeansthatyou,notthecompilerwriter,arelefttoenforcetherulesofanabstractcomputationalmachine,andwhenyouinevitablyfail,it’syourfault.Thecompilerdoesn’thavetoflagtheUB,doanythingreasonable,andit’syourfaultfornotmemorizingall191rulesthatshouldbeavoided.Youarejuststupidfornotmemorizing191complexpotholesontheroadtoC.Thisisawonderfulsituationfortheclassicknow-it-alltypewhocanmemorizethese191finerpointsofannoyancewithwhichtobeatbeginnerstointellectualdeath.There’sanadditionalhypocrisywithUBthatisdoublyinfuriating.IfyoushowaCfanaticcodethatproperlyusesCstringsbutcanoverwritethestringterminator,theywillsay,“That’sUB.It’snottheClanguage’sfault!”However,ifyoushowthemUBthathaswhile(x)x<<=1init,theywillsay,“That’sUBidiot!Fixyourdamncode!”ThisletstheCfanaticsimultaneouslyuseUBtodefendthepurityofC’sdesign,andalsobeatyouupforbeinganidiotwhowritesbadcode.SomeUBismeantas,“youcanignorethesecurityofthissinceit’snotC’sfault”,andotherUBismeantas,“youareanidiotforwritingthiscode,”andthedistinctionbetweenthetwoisnotspecifiedinthestandard.Asyoucansee,IamnotafanofthehugelistofUB.IhadtomemorizeallofthesebeforetheC99standard,andjustdidn’tbothertomemorizethechanges.I’dsimplymovedontoawayandfoundawaytoavoidasmuchUBasIpossiblycould,tryingtostaywithintheabstractmachinespecificationwhilealsoworkingwithrealmachines.Thisturnsouttobealmostimpossible,soIjustdon’twritenewcodeinCanymorebecauseofitsglaringlyobviousproblems.

Warning!ThetechnicalexplanationastowhyCUBiswrongcomesfromAlanTuring:1.CUBcontainsbehaviorsthatarelexical,semantic,andexecutionbased.2.Thelexicalandsemanticbehaviorscanbedetectedbythecompiler.3.Theexecution-basedbehaviorsfallintoTuring’sdefinitionofthehaltingproblem,andarethereforeNP-complete.

4.ThismeansthattoavoidCUB,itrequiressolvingoneoftheoldestprovenunsolvableproblemsincomputerscience,makingUBeffectivelyimpossibleforacomputertoavoid.

Toputitmoresuccinctly:“Iftheonlywaytoknowthatyou’veviolatedtheabstractmachinewithUBistorunyourCprogram,thenyouwillneverbeabletocompletelyavoidUB.”

UB20Becauseofthis,I’mgoingtolistthetop20undefinedbehaviorsinC,andtellyouhowtoavoidthemasbestIcan.Ingeneral,thewaytoavoidUBistowritecleancode,butsomeofthesebehaviorsareimpossibletoavoid.Forexample,writingpasttheendofaCstringisanundefinedbehavior,yetit’seasilydonebyaccidentandexternallyaccessibletoanattacker.ThislistwillalsoincluderelatedUBthatallfallintothesamecategorybutwithdifferingcontexts.

CommonUBs1.Anobjectisreferredtooutsideofitslifetime(6.2.4).

•Thevalueofapointertoanobjectwhoselifetimehasendedisused(6.2.4).•Thevalueofanobjectwithautomaticstoragedurationisusedwhileitisindeterminate(6.2.4,6.7.8,6.8).

2.Conversiontoorfromanintegertypeproducesavalueoutsidetherangethatcanberepresented(6.3.1.4).•Demotionofonerealfloatingtypetoanotherproducesavalueoutsidetherangethatcanberepresented(6.3.1.5).

3.Twodeclarationsofthesameobjectorfunctionspecifytypesthatarenotcompatible(6.2.7).4.Anlvaluehavingarraytypeisconvertedtoapointertotheinitialelementofthearray,andthearrayobjecthasregisterstorageclass(6.3.2.1).•Anattemptismadetousethevalueofavoidexpression,oranimplicitorexplicitconversion(excepttovoid)isappliedtoavoidexpression(6.3.2.2).•Conversionofapointertoanintegertypeproducesavalueoutsidetherangethatcanberepresented(6.3.2.3).•Conversionbetweentwopointertypesproducesaresultthatisincorrectlyaligned(6.3.2.3).•Apointerisusedtocallafunctionwhosetypeisnotcompatiblewiththepointed-totype(6.3.2.3).•Theoperandoftheunary*operatorhasaninvalidvalue(6.5.3.2).•Apointerisconvertedtootherthananintegerorpointertype(6.5.4).•Additionorsubtractionofapointerinto,orjustbeyond,anarrayobjectandanintegertypeproducesaresultthatdoesnotpointinto,orjustbeyond,thesamearrayobject(6.5.6).

•Additionorsubtractionofapointerinto,orjustbeyond,anarrayobjectandanintegertypeproducesaresultthatpointsjustbeyondthearrayobjectandisusedastheoperandofaunary*operatorthatisevaluated(6.5.6).•Pointersthatdonotpointinto,orjustbeyond,thesamearrayobjectaresubtracted(6.5.6).•Anarraysubscriptisoutofrange,evenifanobjectisapparentlyaccessiblewiththegivensubscript(asinthelvalueexpressiona[1][7]giventhedeclarationinta[4][5])(6.5.6).•Theresultofsubtractingtwopointersisnotrepresentableinanobjectoftypeptrdiff_t(6.5.6).•Pointersthatdonotpointtothesameaggregateorunion(norjustbeyondthesamearrayobject)arecomparedusingrelationaloperators(6.5.8).•Anattemptismadetoaccess,orgenerateapointertojustpast,aflexiblearraymemberofastructurewhenthereferencedobjectprovidesnoelementsforthatarray(6.7.2.1).•Twopointertypesthatarerequiredtobecompatiblearenotidenticallyqualified,orarenotpointerstocompatibletypes(6.7.5.1).•Thesizeexpressioninanarraydeclarationisnotaconstantexpressionandevaluatesatprogramexecutiontimetoanonpositivevalue(6.7.5.2).•Thepointerpassedtoalibraryfunctionarrayparameterdoesnothaveavaluesuchthatalladdresscomputationsandobjectaccessesarevalid(7.1.4).

5.Theprogramattemptstomodifyastringliteral(6.4.5).6.Anobjecthasitsstoredvalueaccessedotherthanbyanlvalueofanallowabletype(6.5).7.Anattemptismadetomodifytheresultofafunctioncall,aconditionaloperator,anassignmentoperator,oracommaoperator,ortoaccessitafterthenextsequencepoint(6.5.2.2,6.5.15,6.5.16,6.5.17).

8.Thevalueofthesecondoperandofthe/or%operatoriszero(6.5.5).9.Anobjectisassignedtoaninexactlyoverlappingobjectortoanexactlyoverlappingobjectwithincompatibletype(6.5.16.1).

10.Aconstantexpressioninaninitializerisnot,ordoesnotevaluateto,oneofthefollowing:anarithmeticconstantexpression,anullpointerconstant,anaddressconstant,oranaddressconstantforanobjecttypeplusorminusanintegerconstantexpression(6.6).•Anarithmeticconstantexpressiondoesnothavearithmetictype;hasoperandsthatarenotintegerconstants,floatingconstants,enumerationconstants,characterconstants,orsizeofexpressions;orcontainscasts(outsideoperandstosizeofoperators)otherthanconversionsofarithmetictypestoarithmetictypes(6.6).

11.Anattemptismadetomodifyanobjectdefinedwithaconst-qualifiedtypethroughuseofanlvaluewithnon-const-qualifiedtype(6.7.3).

12.Afunctionwithexternallinkageisdeclaredwithaninlinefunctionspecifier,butisnotalsodefinedinthesametranslationunit(6.7.4).

13.Thevalueofanunnamedmemberofastructureorunionisused(6.7.8).14.The}thatterminatesafunctionisreached,andthevalueofthefunctioncallisusedbythe

caller(6.9.1).15.Afilewiththesamenameasoneofthestandardheaders,notprovidedaspartofthe

implementation,isplacedinanyofthestandardplacesthataresearchedforincludedsourcefiles(7.1.2).

16.ThevalueofanargumenttoacharacterhandlingfunctionisneitherequaltothevalueofEOFnorrepresentableasanunsignedchar(7.4).

17.Thevalueoftheresultofanintegerarithmeticorconversionfunctioncannotberepresented(7.8.2.1,7.8.2.2,7.8.2.3,7.8.2.4,7.20.6.1,7.20.6.2,7.20.1).

18.ThevalueofapointertoaFILEobjectisusedaftertheassociatedfileisclosed(7.19.3).•Thestreamforthefflushfunctionpointstoaninputstreamortoanupdatestreaminwhichthemostrecentoperationwasinput(7.19.5.2).•Thestringpointedtobythemodeargumentinacalltothefopenfunctiondoesnotexactlymatchoneofthespecifiedcharactersequences(7.19.5.3).•Anoutputoperationonanupdatestreamisfollowedbyaninputoperationwithoutaninterveningcalltothefflushfunctionorafilepositioningfunction,oraninputoperationonanupdatestreamisfollowedbyanoutputoperationwithaninterveningcalltoafilepositioningfunction(7.19.5.3).

19.Aconversionspecificationforaformattedoutputfunctionusesa#or0flagwithaconversionspecifierotherthanthosedescribed(7.19.6.1,7.24.2.1).*Ansconversionspecifierisencounteredbyoneoftheformattedoutputfunctions,andtheargumentismissingthenullterminator(unlessaprecisionisspecifiedthatdoesnotrequirenulltermination)(7.19.6.1,7.24.2.1).*Thecontentsofthearraysuppliedinacalltothefgets,gets,orfgetwsfunctionareusedafterareaderroroccurred(7.19.7.2,7.19.7.7,7.24.3.2).

20.Anon-nullpointerreturnedbyacalltothecalloc,malloc,orreallocfunctionwithazerorequestedsizeisusedtoaccessanobject(7.20.3).*Thevalueofapointerthatreferstospacedeallocatedbyacalltothefreeorreallocfunctionisused(7.20.3).*Thepointerargumenttothefreeorreallocfunctiondoesnotmatchapointerearlierreturnedbycalloc,malloc,orrealloc,orthespacehasbeendeallocatedbyacalltofreeorrealloc(7.20.3.2,7.20.3.4).

Therearemanymore,buttheseseemtobetheonesthatIrunintothemostoftenorthatcomeupthemostofteninCcode.Theyarealsothemostdifficulttoavoid,soifyouatleastrememberthese,you’llbeabletoavoidthemajorones.

Exercise32.DoubleLinkedLists

Thepurposeofthisbookistoteachyouhowyourcomputerreallyworks,andincludedinthatishowvariousdatastructuresandalgorithmsfunction.Computersbythemselvesdon’tdoalotofusefulprocessing.Tomakethemdousefulthings,youneedtostructurethedataandthenorganizetheprocessingofthesestructures.Otherprogramminglanguageseitherincludelibrariesthatimplementallofthesestructures,ortheyhavedirectsyntaxforthem.Cmakesyouimplementallofthedatastructuresthatyouneedyourself,whichmakesittheperfectlanguagetolearnhowtheyactuallywork.Mygoalistohelpyoudothreethings:

•Understandwhat’sreallygoingoninPython,Ruby,orJavaScriptcodelikethis:data={"name":"Zed"}•GetevenbetteratCcodebyusingdatastructurestoapplywhatyouknowtoasetofsolvedproblems.•Learnacoresetofdatastructuresandalgorithmssothatyouarebetterinformedaboutwhatworksbestincertainsituations.

WhatAreDataStructuresThenamedatastructureisself-explanatory.It’sanorganizationofdatathatfitsacertainmodel.Maybethemodelisdesignedtoallowprocessingthedatainanewway.Maybeit’sjustorganizedtostoreitondiskefficiently.Inthisbook,I’llfollowasimplepatternformakingdatastructuresthatworkreliably:

•Defineastructureforthemainouterstructure.•Defineastructureforthecontents,usuallynodeswithlinksbetweenthem.•Createfunctionsthatoperateonthesetwostructures.

ThereareotherstylesofdatastructuresinC,butthispatternworkswellandisconsistentformakingmostdatastructures.

MakingtheLibraryFortherestofthisbook,you’llbecreatingalibrarythatyoucanusewhenyou’redone.Thislibrarywillhavethefollowingelements:

•Header(.h)filesforeachdatastructure.•Implementation(.c)filesforthealgorithms.•Unitteststhattestallofthemtomakesuretheykeepworking.•Documentationthatwe’llauto-generatefromtheheaderfiles.

Youalreadyhavethec-skeleton,souseittocreatealiblcthwproject:

Exercise32Session


$cp-rc-skeletonliblcthw$cdliblcthw/

$lsLICENSEMakefileREADME.mdbinbuildsrctests

$vimMakefile$lssrc/dbg.hlibex29.clibex29.o

$mkdirsrc/lcthw$mvsrc/dbg.hsrc/lcthw$vimtests/minunit.h$rmsrc/libex29.*tests/libex29*$makecleanrm-rfbuildtests/libex29_tests

rm-ftests/tests.log



$lstests/minunit.hruntests.sh

$

InthissessionIdothefollowing:•Copythec-skeletonover.•EdittheMakefiletochangelibYOUR_LIBRARY.atoliblcthw.aasthenewTARGET.•Makethesrc/lcthwdirectory,wherewe’llputourcode.•Movethesrc/dbg.hintothisnewdirectory.•Edittests/minunit.hsothatituses#include<lcthw/dbg.h>astheinclude.•Getridofthesourceandtestfilesthatwedon’tneedforlibex29.*.•Cleanupeverythingthat’sleftover.

Nowthatyou’rereadytostartbuildingthelibrary,thefirstdatastructurethatI’llbuildisthedoublylinkedlist.

DoublyLinkedListsThefirstdatastructurethatwe’lladdtoliblcthwisadoublylinkedlist.Thisisthesimplestdatastructureyoucanmake,andithasusefulpropertiesforcertainoperations.Alinkedlistworksbynodeshavingpointerstotheirnextorpreviouselement.Adoublylinkedlistcontainspointerstoboth,whileasinglylinkedlistonlypointsatthenextelement.Becauseeachnodehaspointerstothenextandpreviouselements,andbecauseyoukeeptrackofthefirstandlastelementsofthelist,youcandosomeoperationsveryquicklywithdoublylinkedlists.Anythingthatinvolvesinsertingordeletinganelementwillbeveryfast.They’realsoeasytoimplementbymostprogrammers.Themaindisadvantageofalinkedlististhattraversingitinvolvesprocessingeverysinglepointeralongtheway.Thismeansthatsearching,mostsorting,anditeratingovertheelementswillbeslow.Italsomeansthatyoucan’treallyjumptorandompartsofthelist.Ifyouhadanarrayofelements,youcouldjustindexrightintothemiddleofthelist,butalinkedlistusesastreamofpointers.Thatmeansifyouwantthetenthelement,youhavetogothroughthefirstnineelements.

DefinitionAsIsaidintheintroductiontothisexercise,firstwriteaheaderfilewiththerightCstructurestatementsinit.

list.h


#ifndeflcthw_List_h

#definelcthw_List_h

#include<stdlib.h>

structListNode;typedefstructListNode{structListNode*next;structListNode*prev;void*value;}ListNode;

typedefstructList{intcount;ListNode*first;ListNode*last;}List;

List*List_create();voidList_destroy(List*list);voidList_clear(List*list);voidList_clear_destroy(List*list);

#defineList_count(A)((A)->count)

#defineList_first(A)((A)->first!=NULL?(A)->first->value:NULL)

#defineList_last(A)((A)->last!=NULL?(A)->last->value:NULL)

voidList_push(List*list,void*value);void*List_pop(List*list);

voidList_unshift(List*list,void*value);void*List_shift(List*list);

void*List_remove(List*list,ListNode*node);

#defineLIST_FOREACH(L,S,M,V)ListNode*_node=NULL;\

ListNode*V=NULL;\for(V=_node=L->S;_node!=NULL;V=_node=_node->M)

#endif

ThefirstthingIdoiscreatetwostructuresfortheListNodeandtheListthatwillcontainthosenodes.Thiscreatesthedatastructure,whichI’lluseinthefunctionsandmacrosthatIdefineafterthat.Ifyoureadthesefunctions,you’llseethatthey’rerathersimple.I’llbeexplainingthemwhenIcovertheimplementation,buthopefullyyoucanguesswhattheydo.EachListNodehasthreecomponentswithinthedatastructure:

•Avalue,whichisapointertoanything,andstoresthethingwewanttoputinthelist.•AListNode*nextpointer,whichpointsatanotherListNodethatholdsthenextelementinthelist.•AListNode*prevthatholdsthepreviouselement.Complex,right?Callingthepreviousthing“previous.”Icouldhaveused“anterior”and“posterior,”butonlyajerkwoulddothat.

TheListstructisthennothingmorethanacontainerfortheseListNodestructsthathavebeen

linkedtogetherinachain.Itkeepstrackofthecount,first,andlastelementsofthelist.Finally,takealookatsrc/lcthw/list.h:37whereIdefinetheLIST_FOREACHmacro.Thisisacommonprogrammingidiomwhereyoumakeamacrothatgeneratesiterationcodesopeoplecan’tmessitup.Gettingthiskindofprocessingrightcanbedifficultwithdatastructures,sowritingmacroshelpspeopleout.You’llseehowIusethiswhenItalkabouttheimplementation.

ImplementationYoushouldmostlyunderstandhowadoublylinkedlistworks.It’snothingmorethannodeswithtwopointerstothenextandpreviouselementsofthelist.Youcanthenwritethesrc/lcthw/list.ccodetoseehoweachoperationisimplemented.

list.c


1#include<lcthw/list.h>2#include<lcthw/dbg.h>34List*List_create()5{6returncalloc(1,sizeof(List));7}89voidList_destroy(List*list)10{11LIST_FOREACH(list,first,next,cur){12if(cur->prev){13free(cur->prev);14}15}1617free(list->last);18free(list);19}2021voidList_clear(List*list)22{23LIST_FOREACH(list,first,next,cur){24free(cur->value);25}26}2728voidList_clear_destroy(List*list)29{30List_clear(list);31List_destroy(list);32}3334voidList_push(List*list,void*value)35{36ListNode*node=calloc(1,sizeof(ListNode));37check_mem(node);3839node->value=value;4041if(list->last==NULL){42list->first=node;43list->last=node;

44}else{45list->last->next=node;46node->prev=list->last;47list->last=node;48}4950list->count++;5152error:53return;54}5556void*List_pop(List*list)57{58ListNode*node=list->last;59returnnode!=NULL?List_remove(list,node):NULL;60}6162voidList_unshift(List*list,void*value)63{64ListNode*node=calloc(1,sizeof(ListNode));65check_mem(node);6667node->value=value;6869if(list->first==NULL){70list->first=node;71list->last=node;72}else{73node->next=list->first;74list->first->prev=node;75list->first=node;76}7778list->count++;7980error:81return;82}8384void*List_shift(List*list)85{86ListNode*node=list->first;87returnnode!=NULL?List_remove(list,node):NULL;88}8990void*List_remove(List*list,ListNode*node)91{92void*result=NULL;9394check(list->first&&list->last,"Listisempty.");95check(node,"nodecan'tbeNULL");9697if(node==list->first&&node==list->last){98list->first=NULL;99list->last=NULL;100}elseif(node==list->first){101list->first=node->next;102check(list->first!=NULL,103"Invalidlist,somehowgotafirstthatisNULL.");104list->first->prev=NULL;105}elseif(node==list->last){106list->last=node->prev;107check(list->last!=NULL,

108"Invalidlist,somehowgotanextthatisNULL.");109list->last->next=NULL;110}else{111ListNode*after=node->next;112ListNode*before=node->prev;113after->prev=before;114before->next=after;115}116117list->count--;118result=node->value;119free(node);120121error:122returnresult;123}

Ithenimplementalloftheoperationsonadoublylinkedlistthatcan’tbedonewithsimplemacros.Ratherthancovereverytiny,littlelineofthisfile,I’mgoingtogiveahigh-leveloverviewofeveryoperationinboththelist.handlist.cfiles,andthenleaveyoutoreadthecode.

list.h:List_countReturnsthenumberofelementsinthelist,whichismaintainedaselementsareaddedandremoved.

list.h:List_firstReturnsthefirstelementofthelist,butdoesn’tremoveit.list.h:List_lastReturnsthelastelementofthelist,butdoesn’tremoveit.list.h:LIST_FOREACHIteratesovertheelementsinthelist.list.c:List_createSimplycreatesthemainListstruct.list.c:List_destroyDestroysaListandanyelementsitmighthave.list.c:List_clearAconvenientfunctionforfreeingthevaluesineachnode,notthenodes.list.c:List_clear_destroyClearsanddestroysalist.It’snotveryefficientsinceitloopsthroughthemtwice.

list.c:List_pushThefirstoperationthatdemonstratestheadvantageofalinkedlist.Itaddsanewelementtotheendofthelist,andbecausethat’sjustacoupleofpointerassignments,itdoesitveryfast.

list.c:List_popTheinverseofList_push,thistakesthelastelementoffandreturnsit.list.c:List_unshiftTheotherthingyoucaneasilydotoalinkedlistisaddelementstothefrontofthelistveryquickly.Inthiscase,IcallthatList_unshiftforlackofabetterterm.

list.c:List_shiftJustlikeList_pop,thisremovesthefirstelementandreturnsit.list.c:List_removeThisisactuallydoingalloftheremovalwhenyoudoList_poporList_shift.Somethingthatseemstoalwaysbedifficultindatastructuresisremovingthings,andthisfunctionisnodifferent.Ithastohandlequiteafewconditionsdependingoniftheelementbeingremovedisatthefront,theend,boththefrontandtheend,orthemiddle.

Mostofthesefunctionsarenothingspecial,andyoushouldbeabletoeasilydigestthisandunderstanditfromjustthecode.YoushoulddefinitelyfocusonhowtheLIST_FOREACHmacroisusedinList_destroysothatyoucanunderstandhowmuchitsimplifiesthiscommonoperation.

TestsAfteryouhavethosecompiling,it’stimetocreatethetestthatmakessuretheyoperatecorrectly.

list_tests.c


1#include"minunit.h"2#include<lcthw/list.h>3#include<assert.h>45staticList*list=NULL;6char*test1="test1data";7char*test2="test2data";8char*test3="test3data";910char*test_create()11{12list=List_create();13mu_assert(list!=NULL,"Failedtocreatelist.");1415returnNULL;16}1718char*test_destroy()19{20List_clear_destroy(list);2122returnNULL;2324}2526char*test_push_pop()27{28List_push(list,test1);29mu_assert(List_last(list)==test1,"Wronglastvalue.");3031List_push(list,test2);32mu_assert(List_last(list)==test2,"Wronglastvalue");3334List_push(list,test3);35mu_assert(List_last(list)==test3,"Wronglastvalue.");36mu_assert(List_count(list)==3,"Wrongcountonpush.");3738char*val=List_pop(list);39mu_assert(val==test3,"Wrongvalueonpop.");4041val=List_pop(list);42mu_assert(val==test2,"Wrongvalueonpop.");4344val=List_pop(list);45mu_assert(val==test1,"Wrongvalueonpop.");46mu_assert(List_count(list)==0,"Wrongcountafterpop.");4748returnNULL;49}5051char*test_unshift()52{53List_unshift(list,test1);54mu_assert(List_first(list)==test1,"Wrongfirstvalue.");5556List_unshift(list,test2);57mu_assert(List_first(list)==test2,"Wrongfirstvalue");58

59List_unshift(list,test3);60mu_assert(List_first(list)==test3,"Wronglastvalue.");61mu_assert(List_count(list)==3,"Wrongcountonunshift.");6263returnNULL;64}6566char*test_remove()67{68//weonlyneedtotestthemiddleremovecasesincepush/shift69//alreadyteststheothercases7071char*val=List_remove(list,list->first->next);72mu_assert(val==test2,"Wrongremovedelement.");73mu_assert(List_count(list)==2,"Wrongcountafterremove.");74mu_assert(List_first(list)==test3,"Wrongfirstafterremove.");75mu_assert(List_last(list)==test1,"Wronglastafterremove.");7677returnNULL;78}7980char*test_shift()81{82mu_assert(List_count(list)!=0,"Wrongcountbeforeshift.");8384char*val=List_shift(list);85mu_assert(val==test3,"Wrongvalueonshift.");8687val=List_shift(list);88mu_assert(val==test1,"Wrongvalueonshift.");89mu_assert(List_count(list)==0,"Wrongcountaftershift.");9091returnNULL;92}9394char*all_tests()95{96mu_suite_start();9798mu_run_test(test_create);99mu_run_test(test_push_pop);100mu_run_test(test_unshift);101mu_run_test(test_remove);102mu_run_test(test_shift);103mu_run_test(test_destroy);104105returnNULL;106}107108RUN_TESTS(all_tests);

Thistestsimplygoesthrougheveryoperationandmakessureitworks.IuseasimplificationinthetestwhereIcreatejustoneList*listforthewholeprogram,andthenhavethetestsworkonit.ThissavesthetroubleofbuildingaListforeverytest,butitcouldmeanthatsometestsonlypassbecauseofhowtheprevioustestran.Inthiscase,Itrytomakeeachtestkeepthelistclearoractuallyusetheresultsfromtheprevioustest.

WhatYouShouldSeeIfyoudideverythingright,thenwhenyoudoabuildandruntheunittests,itshouldlooklikethis:

Exercise32.buildSession


$makecc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUG-fPIC-c-o\

src/lcthw/list.osrc/lcthw/list.carrcsbuild/liblcthw.asrc/lcthw/list.o

ranlibbuild/liblcthw.a

cc-shared-obuild/liblcthw.sosrc/lcthw/list.o

cc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUGbuild/liblcthw.a

tests/list_tests.c-otests/list_testssh./tests/runtests.sh

Runningunittests:

----

RUNNING:./tests/list_tests

ALLTESTSPASSED

Testsrun:6

tests/list_testsPASS

$

Makesuresixtestsran,itbuildswithoutwarningsorerrors,andit’smakingthebuild/liblcthw.aandbuild/liblcthw.sofiles.

HowtoImproveItInsteadofbreakingthis,I’mgoingtotellyouhowtoimprovethecode:

•YoucanmakeList_clear_destroymoreefficientbyusingLIST_FOREACHanddoingbothfreecallsinsideoneloop.•Youcanaddassertsforpreconditionssothattheprogramisn’tgivenaNULLvaluefortheList*listparameters.•Youcanaddinvariantsthatcheckthatthelist’scontentsarealwayscorrect,suchascountisnever<0,andifcount>0,thenfirstisn’tNULL.•Youcanadddocumentationtotheheaderfileintheformofcommentsbeforeeachstruct,function,andmacrothatdescribeswhatitdoes.

TheseimprovementsspeaktothedefensiveprogrammingpracticesItalkedaboutearlier,hardeningthiscodeagainstflawsandimprovingusability.Goaheadanddothesethings,andthenfindasmanyotherwaystoimprovethecodeasyoucan.

ExtraCredit•Researchdoublyversussinglylinkedlistsandwhenoneispreferredovertheother.•Researchthelimitationsofadoublylinkedlist.Forexample,whiletheyareefficientforinsertinganddeletingelements,theyareveryslowforiteratingoverthemall.•Whatoperationsaremissingthatyoucanimagineneeding?Someexamplesarecopying,joining,andsplitting.Implementtheseoperationsandwritetheunittestsforthem.

Exercise33.LinkedListAlgorithms

Imgoingtocovertwoalgorithmsforalinkedlistthatinvolvesorting.I’mgoingtowarnyoufirstthatifyouneedtosortthedata,thendon’tusealinkedlist.Thesearehorribleforsortingthings,andtherearemuchbetterdatastructuresyoucanuseifthat’sarequirement.I’mcoveringthesetwoalgorithmsbecausetheyareslightlydifficulttopulloffwithalinkedlist,andtogetyouthinkingabouthowtoefficientlymanipulatethem.Intheinterestofwritingthisbook,I’mgoingtoputthealgorithmsintwodifferentfileslist_algos.handlist_algos.cthenwriteatestinlist_algos_test.c.Fornow,justfollowmystructure,sinceitkeepsthingsclean,butifyoueverworkonotherlibraries,rememberthatthisisn’tacommonstructure.Inthisexercise,I’malsogoingtogiveyouanextrachallengeandIwantyoutotrynottocheat.I’mgoingtogiveyoutheunittestfirst,andIwantyoutotypeitin.Then,IwantyoutotryandimplementthetwoalgorithmsbasedontheirdescriptionsinWikipediabeforeseeingifyourcodelookslikemycode.

BubbleandMergeSortsYouknowwhat’sawesomeabouttheInternet?Icanjustreferyoutothe“bubblesort”and“mergesort”pagesonWikipediaandtellyoutoreadthose.Man,thatsavesmeaboatloadoftyping.NowIcantellyouhowtoactuallyimplementeachoftheseusingthepseudo-codetheyhavethere.Here’showyoucantackleanalgorithmlikethis:

•Readthedescriptionandlookatanyvisualizationsithas.•Eitherdrawthealgorithmonpaperusingboxesandlines,oractuallytakeadeckofplayingcards(orcardswithnumbers)andtrytodothealgorithmmanually.Thisgivesyouaconcretedemonstrationofhowthealgorithmworks.•Createtheskeletonfunctionsinyourlist_algos.cfileandmakeaworkinglist_algos.hfile,thensetupyourtestharness.•Writeyourfirstfailingtestandgeteverythingtocompile.•GobacktotheWikipediapageandcopy-pastethepseudo-code(nottheCcode!)intothegutsofthefirstfunctionthatyou’remaking.•Translatethispseudo-codeintogoodCcodethewayI’vetaughtyou,usingyourunittesttomakesureit’sworking.•Filloutsomemoretestsforedgecaseslikeemptylists,alreadysortedlists,andthelike.•Repeatthisforthenextalgorithmandtestit.

Ijustgaveyouthesecrettofiguringoutmostofthealgorithmsoutthere—untilyougettosomeofthemoreinsaneones,thatis.Inthiscase,you’rejustdoingthebubbleandmergesortsfromWikipedia,butthosewillbegoodstarters.

TheUnitTestHereistheunittestyoushoulduseforthepseudo-code:

list_algos_tests.c


1#include"minunit.h"2#include<lcthw/list_algos.h>3#include<assert.h>4#include<string.h>56char*values[]={"XXXX","1234","abcd","xjvef","NDSS"};78#defineNUM_VALUES5910List*create_words()11{12inti=0;13List*words=List_create();1415for(i=0;i<NUM_VALUES;i++){16List_push(words,values[i]);17}1819returnwords;20}2122intis_sorted(List*words)23{24LIST_FOREACH(words,first,next,cur){25if(cur->next&&strcmp(cur->value,cur->next->value)>0){26debug("%s%s",(char*)cur->value,27(char*)cur->next->value);28return0;29}30}3132return1;33}3435char*test_bubble_sort()36{37List*words=create_words();3839//shouldworkonalistthatneedssorting40intrc=List_bubble_sort(words,(List_compare)strcmp);41mu_assert(rc==0,"Bubblesortfailed.");42mu_assert(is_sorted(words),43"Wordsarenotsortedafterbubblesort.");4445//shouldworkonanalreadysortedlist46rc=List_bubble_sort(words,(List_compare)strcmp);47mu_assert(rc==0,"Bubblesortofalreadysortedfailed.");48mu_assert(is_sorted(words),49"Wordsshouldbesortifalreadybubblesorted.");5051List_destroy(words);5253//shouldworkonanemptylist54words=List_create(words);55rc=List_bubble_sort(words,(List_compare)strcmp);56mu_assert(rc==0,"Bubblesortfailedonemptylist.");57mu_assert(is_sorted(words),"Wordsshouldbesortedifempty.");5859List_destroy(words);6061returnNULL;62}

6364char*test_merge_sort()65{66List*words=create_words();6768//shouldworkonalistthatneedssorting69List*res=List_merge_sort(words,(List_compare)strcmp);70mu_assert(is_sorted(res),"Wordsarenotsortedaftermergesort.");7172List*res2=List_merge_sort(res,(List_compare)strcmp);73mu_assert(is_sorted(res),74"Shouldstillbesortedaftermergesort.");75List_destroy(res2);76List_destroy(res);7778List_destroy(words);79returnNULL;80}8182char*all_tests()83{84mu_suite_start();8586mu_run_test(test_bubble_sort);87mu_run_test(test_merge_sort);8889returnNULL;90}9192RUN_TESTS(all_tests);

Isuggestthatyoustartwiththebubblesortandgetthatworking,andthenmoveontothemergesort.WhatIwoulddoislayoutthefunctionprototypesandskeletonsthatgetallthreefilescompiling,butnotpassingthetest.Then,I’djustfillintheimplementationuntilitstartsworking.

TheImplementationAreyoucheating?Infutureexercises,I’lljustgiveyouaunittestandtellyoutoimplementit,soit’sgoodpracticeforyoutonotlookatthiscodeuntilyougetyourownworking.Here’sthecodeforthelist_algos.candlist_algos.h:

list_algos.h


#ifndeflcthw_List_algos_h

#definelcthw_List_algos_h

#include<lcthw/list.h>

typedefint(*List_compare)(constvoid*a,constvoid*b);

intList_bubble_sort(List*list,List_comparecmp);

List*List_merge_sort(List*list,List_comparecmp);

#endif

list_algos.c


1#include<lcthw/list_algos.h>2#include<lcthw/dbg.h>34inlinevoidListNode_swap(ListNode*a,ListNode*b)5{6void*temp=a->value;7a->value=b->value;8b->value=temp;9}1011intList_bubble_sort(List*list,List_comparecmp)12{13intsorted=1;1415if(List_count(list)<=1){16return0;//alreadysorted17}1819do{20sorted=1;21LIST_FOREACH(list,first,next,cur){22if(cur->next){23if(cmp(cur->value,cur->next->value)>0){24ListNode_swap(cur,cur->next);25sorted=0;26}27}28}29}while(!sorted);3031return0;32}3334inlineList*List_merge(List*left,List*right,List_comparecmp)35{36List*result=List_create();37void*val=NULL;3839while(List_count(left)>0||List_count(right)>0){40if(List_count(left)>0&&List_count(right)>0){41if(cmp(List_first(left),List_first(right))<=0){42val=List_shift(left);43}else{44val=List_shift(right);45}4647List_push(result,val);48}elseif(List_count(left)>0){49val=List_shift(left);50List_push(result,val);51}elseif(List_count(right)>0){52val=List_shift(right);53List_push(result,val);54}55}5657returnresult;58}5960List*List_merge_sort(List*list,List_comparecmp)61{

62if(List_count(list)<=1){63returnlist;64}6566List*left=List_create();67List*right=List_create();68intmiddle=List_count(list)/2;6970LIST_FOREACH(list,first,next,cur){71if(middle>0){72List_push(left,cur->value);73}else{74List_push(right,cur->value);75}7677middle--;78}7980List*sort_left=List_merge_sort(left,cmp);81List*sort_right=List_merge_sort(right,cmp);8283if(sort_left!=left)84List_destroy(left);85if(sort_right!=right)86List_destroy(right);8788returnList_merge(sort_left,sort_right,cmp);89}

Thebubblesortisn’ttoohardtofigureout,althoughit’sreallyslow.Themergesortismuchmorecomplicated,andhonestly,IcouldprobablyspendabitmoretimeoptimizingthiscodeifIwantedtosacrificeclarity.Thereisanotherwaytoimplementamergesortusingabottom-upmethod,butit’salittlehardertounderstand,soIdidn’tdoit.AsI’vealreadysaid,sortingalgorithmsonlinkedlistsareentirelypointless.Youcouldspendalldaytryingtomakethisfasteranditwillstillbeslowerthanalmostanyothersortabledatastructure.Simplydon’tuselinkedlistsifyouneedtosortthings.

WhatYouShouldSeeIfeverythingworks,thenyoushouldgetsomethinglikethis:

Exercise33Session


$makecleanallrm-rfbuildsrc/lcthw/list.osrc/lcthw/list_algos.o\

tests/list_algos_teststests/list_testsrm-ftests/tests.log



cc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUG-fPIC-c-o\

src/lcthw/list.osrc/lcthw/list.ccc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUG-fPIC-c-o\

src/lcthw/list_algos.osrc/lcthw/list_algos.carrcsbuild/liblcthw.asrc/lcthw/list.osrc/lcthw/list_algos.o

ranlibbuild/liblcthw.a

cc-shared-obuild/liblcthw.sosrc/lcthw/list.osrc/lcthw/list_algos.o

cc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUGbuild/liblcthw.a\

tests/list_algos_tests.c-otests/list_algos_testscc-g-O2-Wall-Wextra-Isrc-rdynamic-DNDEBUGbuild/liblcthw.a\

tests/list_tests.c-otests/list_testssh./tests/runtests.sh

Runningunittests:

----

RUNNING:./tests/list_algos_tests

ALLTESTSPASSED

Testsrun:2

tests/list_algos_testsPASS

----

RUNNING:./tests/list_tests

ALLTESTSPASSED

Testsrun:6

tests/list_testsPASS

$

Afterthisexercise,I’mnotgoingtoshowyouthisoutputunlessit’snecessarytoshowyouhowitworks.Fromnowon,youshouldknowthatIranthetestsandthattheyallpassedandeverythingcompiled.

HowtoImproveItGoingbacktothedescriptionofthealgorithms,thereareseveralwaystoimprovetheseimplementations.Hereareafewobviousones:

•Themergesortdoesacrazyamountofcopyingandcreatinglists,sofindwaystoreducethis.•ThebubblesortdescriptioninWikipediamentionsafewoptimizations.Trytoimplementthem.•CanyouusetheList_splitandList_join(ifyouimplementedthem)toimprovemergesort?•Gothroughallofthedefensiveprogrammingchecksandimprovetherobustnessofthisimplementation,protectingagainstbadNULLpointers,andthencreateanoptionaldebuglevelinvariantthatworkslikeis_sorteddoesafterasort.

ExtraCredit•Createaunittestthatcomparestheperformanceofthetwoalgorithms.You’llwanttolookatman3timeforabasictimerfunction,andrunenoughiterationstoatleasthaveafewsecondsofsamples.•Playwiththeamountofdataintheliststhatneedtobesortedandseeifthatchangesyourtiming.•Findawaytosimulatefillingdifferentsizedrandomlists,measuringhowlongtheytake.Then,graphtheresulttoseehowitcomparestothedescriptionofthealgorithm.•Trytoexplainwhysortinglinkedlistsisareallybadidea.•ImplementaList_insert_sortedthatwilltakeagivenvalue,andusingtheList_compare,inserttheelementattherightpositionsothatthelistisalwayssorted.Howdoesusingthismethodcomparetosortingalistafteryou’vebuiltit?•Tryimplementingthebottom-upmergesortdescribedontheWikipediapage.ThecodethereisalreadyC,soitshouldbeeasytorecreate,buttrytounderstandhowit’sworkingcomparedtothesloweroneIhavehere.

Exercise34.DynamicArray

Thisisanarraythatgrowsonitsownandhasmostofthesamefeaturesasalinkedlist.Itwillusuallytakeuplessspace,runfaster,andhasotherbeneficialproperties.Thisexercisewillcoverafewofthedisadvantages,likeveryslowremovalfromthefront,withasolutiontojustdoitattheend.Adynamicarrayissimplyanarrayofvoid**pointersthat’spre-allocatedinoneshotandthatpointatthedata.Inthelinkedlist,youhadafullstructurethatstoredthevoid*valuepointer,butinadynamicarray,there’sjustasinglearraywithallofthem.Thismeansyoudon’tneedanyotherpointersfornextandpreviousrecordssinceyoucanjustindexintothedynamicarraydirectly.Tostart,I’llgiveyoutheheaderfileyoushouldtypeinfortheimplementation:

darray.h


#ifndef_DArray_h

#define_DArray_h

#include<stdlib.h>

#include<assert.h>

#include<lcthw/dbg.h>

typedefstructDArray{intend;intmax;size_telement_size;size_texpand_rate;void**contents;}DArray;

DArray*DArray_create(size_telement_size,size_tinitial_max);

voidDArray_destroy(DArray*array);

voidDArray_clear(DArray*array);

intDArray_expand(DArray*array);

intDArray_contract(DArray*array);

intDArray_push(DArray*array,void*el);

void*DArray_pop(DArray*array);

voidDArray_clear_destroy(DArray*array);

#defineDArray_last(A)((A)->contents[(A)->end-1])

#defineDArray_first(A)((A)->contents[0])

#defineDArray_end(A)((A)->end)

#defineDArray_count(A)DArray_end(A)

#defineDArray_max(A)((A)->max)

#defineDEFAULT_EXPAND_RATE300

staticinlinevoidDArray_set(DArray*array,inti,void*el){

check(i<array->max,"darrayattempttosetpastmax");

if(i>array->end)array->end=i;array->contents[i]=el;error:return;}

staticinlinevoid*DArray_get(DArray*array,inti){

check(i<array->max,"darrayattempttogetpastmax");returnarray->contents[i];error:returnNULL;}

staticinlinevoid*DArray_remove(DArray*array,inti){

void*el=array->contents[i];

array->contents[i]=NULL;

returnel;}

staticinlinevoid*DArray_new(DArray*array){

check(array->element_size>0,"Can'tuseDArray_newon0sizedarrays.");

returncalloc(1,array->element_size);

error:returnNULL;}

#defineDArray_free(E)free((E))

#endif

ThisheaderfileisshowingyouanewtechniquewhereIputstaticinlinefunctionsrightintheheader.Thesefunctiondefinitionswillworksimilarlytothe#definemacrosthatyou’vebeenmaking,butthey’recleanerandeasiertowrite.Ifyouneedtocreateablockofcodeforamacroandyoudon’tneedcodegeneration,thenuseastaticinlinefunction.ComparethistechniquetotheLIST_FOREACHthatgeneratesaproperfor-loopforalist.Thiswouldbeimpossibletodowithastaticinlinefunctionbecauseitactuallyhastogeneratetheinnerblockofcodefortheloop.Theonlywaytodothatiswithacallbackfunction,butthat’snotasfastandit’shardertouse.I’llthenchangethingsupandhaveyoucreatetheunittestforDArray:

darray_tests.c


1#include"minunit.h"2#include<lcthw/darray.h>34staticDArray*array=NULL;5staticint*val1=NULL;

6staticint*val2=NULL;78char*test_create()9{10array=DArray_create(sizeof(int),100);11mu_assert(array!=NULL,"DArray_createfailed.");12mu_assert(array->contents!=NULL,"contentsarewrongindarray");13mu_assert(array->end==0,"endisn'tattherightspot");14mu_assert(array->element_size==sizeof(int),15"elementsizeiswrong.");16mu_assert(array->max==100,"wrongmaxlengthoninitialsize");1718returnNULL;19}2021char*test_destroy()22{23DArray_destroy(array);2425returnNULL;26}2728char*test_new()29{30val1=DArray_new(array);31mu_assert(val1!=NULL,"failedtomakeanewelement");3233val2=DArray_new(array);34mu_assert(val2!=NULL,"failedtomakeanewelement");3536returnNULL;37}3839char*test_set()40{41DArray_set(array,0,val1);42DArray_set(array,1,val2);4344returnNULL;45}4647char*test_get()48{49mu_assert(DArray_get(array,0)==val1,"Wrongfirstvalue.");50mu_assert(DArray_get(array,1)==val2,"Wrongsecondvalue.");5152returnNULL;53}5455char*test_remove()56{57int*val_check=DArray_remove(array,0);58mu_assert(val_check!=NULL,"ShouldnotgetNULL.");59mu_assert(*val_check==*val1,"Shouldgetthefirstvalue.");60mu_assert(DArray_get(array,0)==NULL,"Shouldbegone.");61DArray_free(val_check);6263val_check=DArray_remove(array,1);64mu_assert(val_check!=NULL,"ShouldnotgetNULL.");65mu_assert(*val_check==*val2,"Shouldgetthefirstvalue.");66mu_assert(DArray_get(array,1)==NULL,"Shouldbegone.");67DArray_free(val_check);68

69returnNULL;70}7172char*test_expand_contract()73{74intold_max=array->max;75DArray_expand(array);76mu_assert((unsignedint)array->max==old_max+array->expand_rate,77"Wrongsizeafterexpand.");7879DArray_contract(array);80mu_assert((unsignedint)array->max==array->expand_rate+1,81"Shouldstayattheexpand_rateatleast.");8283DArray_contract(array);84mu_assert((unsignedint)array->max==array->expand_rate+1,85"Shouldstayattheexpand_rateatleast.");8687returnNULL;88}8990char*test_push_pop()91{92inti=0;93for(i=0;i<1000;i++){94int*val=DArray_new(array);95*val=i*333;96DArray_push(array,val);97}9899mu_assert(array->max==1201,"Wrongmaxsize.");100101for(i=999;i>=0;i--){102int*val=DArray_pop(array);103mu_assert(val!=NULL,"Shouldn'tgetaNULL.");104mu_assert(*val==i*333,"Wrongvalue.");105DArray_free(val);106}107108returnNULL;109}110111char*all_tests()112{113mu_suite_start();114115mu_run_test(test_create);116mu_run_test(test_new);117mu_run_test(test_set);118mu_run_test(test_get);119mu_run_test(test_remove);120mu_run_test(test_expand_contract);121mu_run_test(test_push_pop);122mu_run_test(test_destroy);123124returnNULL;125}126127RUN_TESTS(all_tests);

Thisshowsyouhowalloftheoperationsareused,whichthenmakesimplementingtheDArraymucheasier:

darray.c


1#include<lcthw/darray.h>2#include<assert.h>34DArray*DArray_create(size_telement_size,size_tinitial_max)5{6DArray*array=malloc(sizeof(DArray));7check_mem(array);8array->max=initial_max;9check(array->max>0,"Youmustsetaninitial_max>0.");1011array->contents=calloc(initial_max,sizeof(void*));12check_mem(array->contents);1314array->end=0;15array->element_size=element_size;16array->expand_rate=DEFAULT_EXPAND_RATE;1718returnarray;1920error:21if(array)22free(array);23returnNULL;24}2526voidDArray_clear(DArray*array)27{28inti=0;29if(array->element_size>0){30for(i=0;i<array->max;i++){31if(array->contents[i]!=NULL){32free(array->contents[i]);33}34}35}36}3738staticinlineintDArray_resize(DArray*array,size_tnewsize)39{40array->max=newsize;41check(array->max>0,"Thenewsizemustbe>0.");4243void*contents=realloc(44array->contents,array->max*sizeof(void*));45//checkcontentsandassumereallocdoesn'tharmtheoriginalonerror4647check_mem(contents);4849array->contents=contents;5051return0;52error:53return-1;54}5556intDArray_expand(DArray*array)57{58size_told_max=array->max;

59check(DArray_resize(array,array->max+array->expand_rate)==0,60"Failedtoexpandarraytonewsize:%d",61array->max+(int)array->expand_rate);6263memset(array->contents+old_max,0,array->expand_rate+1);64return0;6566error:67return-1;68}6970intDArray_contract(DArray*array)71{72intnew_size=array->end<(int)array->expand_rate?73(int)array->expand_rate:array->end;7475returnDArray_resize(array,new_size+1);76}7778voidDArray_destroy(DArray*array)79{80if(array){81if(array->contents)82free(array->contents);83free(array);84}85}8687voidDArray_clear_destroy(DArray*array)88{89DArray_clear(array);90DArray_destroy(array);91}9293intDArray_push(DArray*array,void*el)94{95array->contents[array->end]=el;96array->end++;9798if(DArray_end(array)>=DArray_max(array)){99returnDArray_expand(array);100}else{101return0;102}103}104105void*DArray_pop(DArray*array)106{107check(array->end-1>=0,"Attempttopopfromemptyarray.");108109void*el=DArray_remove(array,array->end-1);110array->end--;111112if(DArray_end(array)>(int)array->expand_rate113&&DArray_end(array)%array->expand_rate){114DArray_contract(array);115}116117returnel;118error:119returnNULL;120}

Thisshowsyouanotherwaytotacklecomplexcode.Insteadofdivingrightintothe.cimplementation,lookattheheaderfile,andthenreadtheunittest.Thisgivesyouanabstract-to-concreteunderstandingofhowthepiecesworktogether,makingiteasiertoremember.

AdvantagesandDisadvantagesADArrayisbetterwhenyouneedtooptimizetheseoperations:

•Iteration:Youcanjustuseabasicfor-loopandDArray_countwithDArray_get,andyou’redone.Nospecialmacrosneeded,andit’sfasterbecauseyouaren’twalkingthroughpointers.•Indexing:YoucanuseDArray_getandDArray_settoaccessanyelementatrandom,butwithaListyouhavetogothroughNelementstogettoN+1.•Destroying:Youcanjustfreethestructandthecontentsintwooperations.AListrequiresaseriesoffreecallsandwalkingeveryelement.•Cloning:Youcanalsocloneitinjusttwooperations(pluswhateverit’sstoring)bycopyingthestructandcontents.Again,alistrequireswalkingthroughthewholethingandcopyingeveryListNodeplusitsvalue.•Sorting:Asyousaw,Listishorribleifyouneedtokeepthedatasorted.ADArrayopensupawholeclassofgreatsortingalgorithms,becausenowyoucanaccesselementsrandomly.•LargeData:Ifyouneedtokeeparoundalotofdata,thenaDArraywinssinceitsbase,contents,takesuplessmemorythanthesamenumberofListNodestructs.

However,theListwinsontheseoperations:•Insertandremoveonthefront(whatIcalledshift):ADArrayneedsspecialtreatmenttobeabletodothisefficiently,andusuallyithastodosomecopying.•Splittingorjoining:AListcanjustcopysomepointersandit’sdone,butwithaDArray,youhavecopyallofthearraysinvolved.•SmallData:Ifyouonlyneedtostoreafewelements,thentypicallythestoragewillbesmallerinaListthanagenericDArray.ThisisbecausetheDArrayneedstoexpandthebackingstoretoaccommodatefutureinserts,whileaListonlymakeswhatitneeds.

Withthis,IprefertouseaDArrayformostofthethingsyouseeotherpeopleuseaListfor.IreserveusingListforanydatastructurethatrequiresasmallnumberofnodestobeaddedandremovedfromeitherend.I’llshowyoutwosimilardatastructurescalledaStackandQueuewherethisisimportant.

HowtoImproveItAsusual,gothrougheachfunctionandoperationandaddthedefensiveprogrammingchecks,pre-conditions,invariants,andanythingelseyoucanfindtomaketheimplementationmorebulletproof.

ExtraCredit•Improvetheunitteststocovermoreoftheoperations,andtestthemusingafor-looptoensurethattheywork.•ResearchwhatitwouldtaketoimplementbubblesortandmergesortforDArray,butdon’tdo

ityet.I’llbeimplementingDArrayalgorithmsnext,soyou’lldothisthen.•WritesomeperformancetestsforcommonoperationsandcomparethemtothesameoperationsinList.Youdidsomeofthisalready,butthistime,writeaunittestthatrepeatedlydoestheoperationinquestionandthen,inthemainrunner,dothetiming.•LookathowtheDArray_expandisimplementedusingaconstantincrease(size+300).Typically,dynamicarraysareimplementedwithamultiplicativeincrease(size×2),butI’vefoundthistocostneedlessmemoryfornorealperformancegain.Testmyassertionandseewhenyou’dwantamultiplicativeincreaseinsteadofaconstantincrease.

Exercise35.SortingandSearching

Inthisexercise,I’mgoingtocoverfoursortingalgorithmsandonesearchalgorithm.Thesortingalgorithmsaregoingtobequicksort,heapsort,mergesort,andradixsort.I’mthengoingtoshowyouhowdoatobinarysearchafteryou’vedonearadixsort.However,I’malazyguy,andinmoststandardClibrariesyouhaveexistingimplementationsoftheheapsort,quicksort,andmergesortalgorithms.Here’showyouusethem:

darray_algos.c


1#include<lcthw/darray_algos.h>2#include<stdlib.h>34intDArray_qsort(DArray*array,DArray_comparecmp)5{6qsort(array->contents,DArray_count(array),sizeof(void*),cmp);7return0;8}910intDArray_heapsort(DArray*array,DArray_comparecmp)11{12returnheapsort(array->contents,DArray_count(array),13sizeof(void*),cmp);14}1516intDArray_mergesort(DArray*array,DArray_comparecmp)17{18returnmergesort(array->contents,DArray_count(array),19sizeof(void*),cmp);20}

That’sthewholeimplementationofthedarray_algos.cfile,anditshouldworkonmostmodernUNIXsystems.WhateachofthesedoesissortthecontentsstoreofvoidpointersusingtheDArray_comparethatyougiveit.I’llshowyoutheheaderfileforthis,too:

darray_algos.h


#ifndefdarray_algos_h

#definedarray_algos_h

#include<lcthw/darray.h>

typedefint(*DArray_compare)(constvoid*a,constvoid*b);

intDArray_qsort(DArray*array,DArray_comparecmp);

intDArray_heapsort(DArray*array,DArray_comparecmp);

intDArray_mergesort(DArray*array,DArray_comparecmp);

#endif

It’saboutthesamesizeandshouldbewhatyouexpect.Next,youcanseehowthesefunctionsareusedintheunittestforthesethree:

darray_algos_tests.c


1#include"minunit.h"2#include<lcthw/darray_algos.h>34inttestcmp(char**a,char**b)5{6returnstrcmp(*a,*b);7}89DArray*create_words()10{11DArray*result=DArray_create(0,5);12char*words[]={"asdfasfd",13"werwar","13234","asdfasfd","oioj"};14inti=0;1516for(i=0;i<5;i++){17DArray_push(result,words[i]);18}1920returnresult;21}2223intis_sorted(DArray*array)24{25inti=0;2627for(i=0;i<DArray_count(array)-1;i++){28if(strcmp(DArray_get(array,i),DArray_get(array,i+1))>0){29return0;30}31}3233return1;34}3536char*run_sort_test(int(*func)(DArray*,DArray_compare),37constchar*name)38{39DArray*words=create_words();40mu_assert(!is_sorted(words),"Wordsshouldstartnotsorted.");4142debug("---Testing%ssortingalgorithm",name);43intrc=func(words,(DArray_compare)testcmp);44mu_assert(rc==0,"sortfailed");45mu_assert(is_sorted(words),"didn'tsortit");4647DArray_destroy(words);4849returnNULL;50}5152char*test_qsort()53{54returnrun_sort_test(DArray_qsort,"qsort");

55}5657char*test_heapsort()58{59returnrun_sort_test(DArray_heapsort,"heapsort");60}6162char*test_mergesort()63{64returnrun_sort_test(DArray_mergesort,"mergesort");65}6667char*all_tests()68{69mu_suite_start();7071mu_run_test(test_qsort);72mu_run_test(test_heapsort);73mu_run_test(test_mergesort);7475returnNULL;76}7778RUN_TESTS(all_tests);

Thethingtonotice,andactuallywhattrippedmeupforawholeday,isthedefinitionoftestcmponline4.Youhavetouseachar**andnotachar*becauseqsortgivesyouapointertothepointersinthecontentsarray.Thefunctionqsortandfriendsarescanningthearray,andhandingpointerstoeachelementinthearraytoyourcomparisonfunction.SincewhatIhaveinthecontentsarrayarepointers,thatmeansyougetapointertoapointer.Withthatoutoftheway,youhavejustimplementedthreedifficultsortingalgorithmsinabout20linesofcode.Youcouldstopthere,butpartofthisbookislearninghowthesealgorithmswork,sotheExtraCreditsectionisgoingtoinvolveimplementingeachofthese.

RadixSortandBinarySearchSinceyou’regoingtoimplementquicksort,heapsort,andmergesortonyourown,I’mgoingtoshowyouafunkyalgorithmcalledradixsort.Ithasaslightlynarrowusefulnessinsortingarraysofintegers,butseemstoworklikemagic.Inthiscase,I’mgoingtocreateaspecialdatastructurecalledaRadixMapthat’susedtomaponeintegertoanother.Here’stheheaderfileforthenewalgorithm,whichisbothalgorithmanddatastructureinone:

radixmap.h


#ifndef_radixmap_h

#include<stdint.h>

typedefunionRMElement{uint64_traw;struct{uint32_tkey;uint32_tvalue;}data;}RMElement;

typedefstructRadixMap{size_tmax;size_tend;uint32_tcounter;RMElement*contents;RMElement*temp;}RadixMap;

RadixMap*RadixMap_create(size_tmax);

voidRadixMap_destroy(RadixMap*map);

voidRadixMap_sort(RadixMap*map);

RMElement*RadixMap_find(RadixMap*map,uint32_tkey);

intRadixMap_add(RadixMap*map,uint32_tkey,uint32_tvalue);

intRadixMap_delete(RadixMap*map,RMElement*el);

#endif

YouseethatIhavealotofthesameoperationsasinaDynamicArrayoraListdatastructure,butthedifferenceisI’mworkingonlywithfixedsize32-bituin32_tintegers.I’malsointroducingyoutoanewCconceptcalledtheunionhere.

CUnionsAunionisawaytorefertothesamepieceofmemoryinanumberofdifferentways.Youdefineitlikeastruct,excepteveryelementissharingthesamespacewithalloftheothers.Youcanthinkofaunionasapictureofthememory,andtheelementsintheunionasdifferentcoloredlensestoviewthepicture.Whattheyareusedforistoeithersavememoryorconvertchunksofmemorybetweenformats.Thefirstusageistypicallydonewithvarianttypes,whereyoucreateastructurethathastagforthetype,andthenaunioninsideitforeachtype.Whenusedforconvertingbetweenformatsofmemory,youcansimplydefinethetwostructures,andthenaccesstherightone.First,letmeshowyouhowtomakeavarianttypewithCunions:

ex35.c


1#include<stdio.h>23typedefenum{4TYPE_INT,5TYPE_FLOAT,6TYPE_STRING,7}VariantType;89structVariant{10VariantTypetype;11union{12intas_integer;13floatas_float;14char*as_string;15}data;

16};1718typedefstructVariantVariant;1920voidVariant_print(Variant*var)21{22switch(var->type){23caseTYPE_INT:24printf("INT:%d\n",var->data.as_integer);25break;26caseTYPE_FLOAT:27printf("FLOAT:%f\n",var->data.as_float);28break;29caseTYPE_STRING:30printf("STRING:%s\n",var->data.as_string);31break;32default:33printf("UNKNOWNTYPE:%d",var->type);34}35}3637intmain(intargc,char*argv[])38{39Varianta_int={.type=TYPE_INT,.data.as_integer=100};40Varianta_float={.type=TYPE_FLOAT,.data.as_float=100.34};41Varianta_string={.type=TYPE_STRING,42.data.as_string="YODUDE!"};4344Variant_print(&a_int);45Variant_print(&a_float);46Variant_print(&a_string);4748//here'showyouaccessthem49a_int.data.as_integer=200;50a_float.data.as_float=2.345;51a_string.data.as_string="Hithere.";5253Variant_print(&a_int);54Variant_print(&a_float);55Variant_print(&a_string);5657return0;58}

Youfindthisinmanyimplementationsofdynamiclanguages.Thelanguagewilldefinesomebasevarianttypewithtagsforallthebasetypesofthelanguage,andthenusuallythere’sagenericobjecttagforthetypesyoucancreate.TheadvantageofdoingthisisthattheVariantonlytakesupasmuchspaceastheVariantTypetypetagandthelargestmemberoftheunion.ThisisbecauseCislayeringeachelementoftheVariant.datauniontogether,sotheyoverlap.Todothat,Csizestheunionbigenoughtoholdthelargestelement.Intheradixmap.hfile,IhavetheRMElementunion,whichdemonstratesusingauniontoconvertblocksofmemorybetweentypes.Inthiscase,Iwanttostoreauint64_t-sizedintegerforsortingpurposes,butIwanttwouint32_tintegersforthedatatorepresentakeyandvaluepair.Byusingaunion,I’mabletocleanlyaccessthesameblockofmemoryinthetwodifferentwaysIneed.

TheImplementationInexthavetheactualRadixMapimplementationforeachoftheseoperations:

radixmap.c


1/*2*BasedoncodebyAndreReinaldthenheavilymodifiedbyZedA.Shaw.3*/45#include<stdio.h>6#include<stdlib.h>7#include<assert.h>8#include<lcthw/radixmap.h>9#include<lcthw/dbg.h>1011RadixMap*RadixMap_create(size_tmax)12{13RadixMap*map=calloc(sizeof(RadixMap),1);14check_mem(map);1516map->contents=calloc(sizeof(RMElement),max+1);17check_mem(map->contents);1819map->temp=calloc(sizeof(RMElement),max+1);20check_mem(map->temp);2122map->max=max;23map->end=0;2425returnmap;26error:27returnNULL;28}2930voidRadixMap_destroy(RadixMap*map)31{32if(map){33free(map->contents);34free(map->temp);35free(map);36}37}3839#defineByteOf(x,y)(((uint8_t*)x)[(y)])4041staticinlinevoidradix_sort(shortoffset,uint64_tmax,42uint64_t*source,uint64_t*dest)43{44uint64_tcount[256]={0};45uint64_t*cp=NULL;46uint64_t*sp=NULL;47uint64_t*end=NULL;48uint64_ts=0;49uint64_tc=0;5051//countoccurencesofeverybytevalue52for(sp=source,end=source+max;sp<end;sp++){53count[ByteOf(sp,offset)]++;54}5556//transformcountintoindexbysumming57//elementsandstoringintosamearray58for(s=0,cp=count,end=count+256;cp<end;cp++){

59c=*cp;60*cp=s;61s+=c;62}6364//filldestwiththerightvaluesintherightplace65for(sp=source,end=source+max;sp<end;sp++){66cp=count+ByteOf(sp,offset);67dest[*cp]=*sp;68++(*cp);69}70}7172voidRadixMap_sort(RadixMap*map)73{74uint64_t*source=&map->contents[0].raw;75uint64_t*temp=&map->temp[0].raw;7677radix_sort(0,map->end,source,temp);78radix_sort(1,map->end,temp,source);79radix_sort(2,map->end,source,temp);80radix_sort(3,map->end,temp,source);81}8283RMElement*RadixMap_find(RadixMap*map,uint32_tto_find)84{85intlow=0;86inthigh=map->end-1;87RMElement*data=map->contents;8889while(low<=high){90intmiddle=low+(high-low)/2;91uint32_tkey=data[middle].data.key;9293if(to_find<key){94high=middle-1;95}elseif(to_find>key){96low=middle+1;97}else{98return&data[middle];99}100}101102returnNULL;103}104105intRadixMap_add(RadixMap*map,uint32_tkey,uint32_tvalue)106{107check(key<UINT32_MAX,"Keycan'tbeequaltoUINT32_MAX.");108109RMElementelement={.data={.key=key,.value=value}};110check(map->end+1<map->max,"RadixMapisfull.");111112map->contents[map->end++]=element;113114RadixMap_sort(map);115116return0;117118error:119return-1;120}121

122intRadixMap_delete(RadixMap*map,RMElement*el)123{124check(map->end>0,"Thereisnothingtodelete.");125check(el!=NULL,"Can'tdeleteaNULLelement.");126127el->data.key=UINT32_MAX;128129if(map->end>1){130//don'tbotherresortingamapof1length131RadixMap_sort(map);132}133134map->end--;135136return0;137error:138return-1;139}

Asusual,enterthisinandgetitworking,alongwiththeunittest,andthenI’llexplainwhat’shappening.Takespecialcarewiththeradix_sortfunctionsinceit’sveryparticularinhowit’simplemented.

radixmap_tests.c


1#include"minunit.h"2#include<lcthw/radixmap.h>3#include<time.h>45staticintmake_random(RadixMap*map)6{7size_ti=0;89for(i=0;i<map->max-1;i++){10uint32_tkey=(uint32_t)(rand()|(rand()<<16));11check(RadixMap_add(map,key,i)==0,"Failedtoaddkey%u.",12key);13}1415returni;1617error:18return0;19}2021staticintcheck_order(RadixMap*map)22{23RMElementd1,d2;24unsignedinti=0;2526//onlysignalerrorsifany(shouldnotbe)27for(i=0;map->end>0&&i<map->end-1;i++){28d1=map->contents[i];29d2=map->contents[i+1];3031if(d1.data.key>d2.data.key){32debug("FAIL:i=%u,key:%u,value:%u,equalsmax?%d\n",i,33d1.data.key,d1.data.value,34d2.data.key==UINT32_MAX);

35return0;36}37}3839return1;40}4142staticinttest_search(RadixMap*map)43{44unsignedi=0;45RMElement*d=NULL;46RMElement*found=NULL;4748for(i=map->end/2;i<map->end;i++){49d=&map->contents[i];50found=RadixMap_find(map,d->data.key);51check(found!=NULL,"Didn'tfind%uat%u.",d->data.key,i);52check(found->data.key==d->data.key,53"Gotthewrongresult:%p:%ulookingfor%uat%u",found,54found->data.key,d->data.key,i);55}5657return1;58error:59return0;60}6162//testforbignumberofelements63staticchar*test_operations()64{65size_tN=200;6667RadixMap*map=RadixMap_create(N);68mu_assert(map!=NULL,"Failedtomakethemap.");69mu_assert(make_random(map),"Didn'tmakearandomfakeradixmap.");7071RadixMap_sort(map);72mu_assert(check_order(map),73"FailedtoproperlysorttheRadixMap.");7475mu_assert(test_search(map),"Failedthesearchtest.");76mu_assert(check_order(map),77"RadixMapdidn'tstaysortedaftersearch.");7879while(map->end>0){80RMElement*el=RadixMap_find(map,81map->contents[map->end/2].data.key);82mu_assert(el!=NULL,"Shouldgetaresult.");8384size_told_end=map->end;8586mu_assert(RadixMap_delete(map,el)==0,"Didn'tdeleteit.");87mu_assert(old_end-1==map->end,"Wrongsizeafterdelete.");8889//testthattheendisnowtheoldvalue,90//butuint32maxsoittrailsoff91mu_assert(check_order(map),92"RadixMapdidn'tstaysortedafterdelete.");93}9495RadixMap_destroy(map);9697returnNULL;98}

99100char*all_tests()101{102mu_suite_start();103srand(time(NULL));104105mu_run_test(test_operations);106107returnNULL;108}109110RUN_TESTS(all_tests);

Ishouldn’thavetoexplaintoomuchaboutthetest.It’ssimplysimulatingplacingrandomintegersintotheRadixMap,andthenmakingsureitcangetthemoutreliably.Nottoointeresting.Intheradixmap.cfile,mostoftheoperationsareeasytounderstandifyoureadthecode.Here’sadescriptionofwhatthebasicfunctionsaredoingandhowtheywork:

RadixMap_createAsusual,I’mallocatingallofthememoryneededforthestructuresdefinedinradixmap.h.I’llbeusingthetempandcontentslaterwhenItalkaboutradix_sort.

RadixMap_destroyAgain,I’mjustdestroyingwhatwascreated.radix_sortHere’sthemeatofthedatastructure,butI’llexplainwhatit’sdoinginthenextsection.RadixMap_sortThisusestheradix_sortfunctiontoactuallysortthecontents.Itdoesthisbysortingbetweenthecontentsandtempuntilfinallycontentsissorted.You’llseehowthisworkswhenIdescriberadix_sortlater.

RadixMap_findThisisusingabinarysearchalgorithmtofindakeyyougiveit.I’llexplainhowthisworksshortly.

RadixMap_addUsingtheRadixMap_sortfunction,thiswilladdthekeyandvalueyourequestattheend,thensimplysortitagainsothateverythingisintherightplace.Onceeverythingissorted,theRadixMap_findwillworkproperlybecauseit’sabinarysearch.

RadixMap_deleteThisworksthesameasRadixMap_add,exceptitdeleteselementsofthestructurebysettingtheirvaluestothemaxforaunsigned32-bitinteger,UINT32_MAX.Thismeansthatyoucan’tusethatvalueasankeyvalue,butitmakesdeletingelementseasy.Simplysetittothatandthensort,andit’llgetmovedtotheend.Nowit’sdeleted.

StudythecodeforthefunctionsIdescribed.ThatjustleavesRadixMap_sort,radix_sort,andRadixMap_findtounderstand.

RadixMap_findandBinarySearchI’llstartwithhowthebinarysearchisimplemented.Binarysearchisasimplealgorithmthatmostpeoplecanunderstandintuitively.Infact,youcouldtakeadeckofplayingcardsanddothismanually.Here’showthisfunctionworks,andhowabinarysearchisdone,stepbystep:

•Setahighandlowmarkbasedonthesizeofthearray.•Getthemiddleelementbetweenthelowandhighmarks.•Ifthekeyisless-than,thenthekeymustbebelowthemiddle.Sethightoonelessthanmiddle.•Ifthekeyisgreater-than,thenthekeymustbeabovethemiddle.Setthelowmarkonegreaterthanthemiddle.•Ifit’sequal,youfoundit.Stop.

•Keeploopinguntillowandhighpasseachother.Youwon’tfinditifyouexittheloop.Whatyou’reeffectivelydoingisguessingwherethekeymightbebypickingthemiddleandcomparingittothehighandlowmarks.Sincethedataissorted,youknowthatthethekeyhastobeaboveorbelowyourguess.Ifit’sbelow,thenyoujustdividedthesearchspaceinhalf.Youkeepgoinguntilyoueitherfinditoryouoverlaptheboundariesandexhaustthesearchspace.

RadixMap_sortandradix_sortAradixsortiseasytounderstandifyoutrytodoitmanuallyfirst.Whatthisalgorithmdoesisexploitthefactthatnumbersarestoredwithasequenceofdigitsthatgofromleastsignificanttomostsignificant.Itthentakesthenumbersandbucketsthembythedigit,andwhenithasprocessedallofthedigits,thenumberscomeoutsorted.Atfirstitseemslikemagic,andhonestly,lookingatthecodesureseemslikeitis,sotrydoingitmanuallyonce.Todothisalgorithm,writeoutabunchofthree-digitnumbersinarandomorder.Let’ssaywedo223,912,275,100,633,120,and380.

•Placethenumberinbucketsbytheonesdigit:[380,100,120],[912],[633,223],[275].•Inowhavetogothrougheachofthesebucketsinorder,andthensortitbythetensdigit:[100],[912],[120,223],[633],[275],[380].•Noweachbucketcontainsnumbersthataresortedbytheonesdigitandthenthetensdigit.Ineedtothengothroughtheseinorderandfillinthefinalhundredsdigit:[100,120],[223,275],[380],[633],[912].•Atthispointeachbucketissortedbyhundreds,tensandones,andifItakeeachbucketinorder,Igetthefinalsortedlist:100,120,223,275,380,633,912.

Makesureyoudothisafewtimessoyouunderstandhowitworks.Itreallyisaslicklittlealgorithm.Mostimportantly,itwillworkonnumbersofarbitrarysize,soyoucansortreallyhugenumbersbecauseyou’rejustdoingthem1byteatatime.Inmysituation,thedigits(alsocalledplacevalues)areindividual8-bitbytes,soIneed256bucketstostorethedistributionofthenumbersbytheirdigits.IalsoneedawaytostorethemsuchthatIdon’tusetoomuchspace.Ifyoulookatradix_sort,you’llseethatthefirstthingIdoisbuildacounthistogramsoIknowhowmanyoccurrencesofeachdigitthereareforthegivenoffset.OnceIknowthecountsforeachdigit(all256ofthem),Icanthenusethemasdistributionpointsintoatargetarray.Forexample,ifIhave10bytesthatare0x00,thenIknowIcanplacetheminthefirsttenslotsofthetargetarray.Thisgivesmeanindexforwheretheygointhetargetarray,whichisthesecondfor-loopinradix_sort.Finally,onceIknowwheretheycangointhetargetarrayIsimplygothroughallofthedigitsinthesourcearrayforthisoffset,andplacethenumbersintheirslotsinorder.UsingtheByteOfmacrohelpskeepthecodeclean,sincethere’sabitofpointerhackerytomakeitwork.However,theendresultisthatalloftheintegerswillbeplacedinthebucketfortheirdigitwhenthefinalfor-loopisdone.WhatbecomesinterestingishowIusethisinRadixMap_sorttosortthese64-bitintegersbyjustthefirst32bits.RememberhowIhavethekeyandvalueinaunionfortheRMElementtype?Thatmeansthattosortthisarraybythekey,Ionlyneedtosortthefirst4bytes(32bits/8bitsperbyte)ofeveryinteger.

IfyoulookattheRadixMap_sort,youseethatIgrabaquickpointertothecontentsandtempforsourceandtargetarrays,andthenIcallradix_sortfourtimes.EachtimeIcallit,Ialternatesourceandtarget,anddothenextbyte.WhenI’mdone,theradix_sorthasdoneitsjobandthefinalcopyhasbeensortedintothecontents.

HowtoImproveItThereisabigdisadvantagetothisimplementationbecauseithastoprocesstheentirearrayfourtimesoneveryinsertion.Itdoesdoitfast,butit’dbebetterifyoucouldlimittheamountofsortingbythesizeofwhatneedstobesorted.Therearetwowaysyoucanimprovethisimplementation:

•Useabinarysearchtofindtheminimumpositionforthenewelement,thenonlysortfromtheretotheend.Youfindtheminimum,putthenewelementontheend,andthenjustsortfromtheminimumon.Thiswillcutyoursortspacedownconsiderablymostofthetime.•Keeptrackofthebiggestkeycurrentlybeingused,andthenonlysortenoughdigitstohandlethatkey.Youcanalsokeeptrackofthesmallestnumber,andthenonlysortthedigitsnecessaryfortherange.Todothis,you’llhavetostartcaringaboutCPUintegerordering(endianness).

Trytheseoptimizations,butonlyafteryouaugmenttheunittestwithsometiminginformationsoyoucanseeifyou’reactuallyimprovingthespeedoftheimplementation.

ExtraCredit•Implementquicksort,heapsort,andmergesortandthenprovidea#definethatletsyoupickamongthethree,orcreateasecondsetoffunctionsyoucancall.UsethetechniqueItaughtyoutoreadtheWikipediapageforthealgorithm,andthenimplementitwiththepseudo-code.•Comparetheperformanceofyouroptimizationstotheoriginalimplementations.•UsethesesortingfunctionstocreateaDArray_sort_addthataddselementstotheDArraybutsortsthearrayafterward.•WriteaDArray_findthatusesthebinarysearchalgorithmfromRadixMap_findandtheDArray_comparetofindelementsinasortedDArray.

Exercise36.SaferStrings

Thisexerciseisdesignedtogetyouusingbstringfromnowon,explainwhyC’sstringsareanincrediblybadidea,andthenhaveyouchangetheliblcthwcodetousebstring.

WhyCStringsWereaHorribleIdeaWhenpeopletalkaboutproblemswithC,theysayitsconceptofastringisoneofthetopflaws.You’vebeenusingtheseextensively,andI’vetalkedaboutthekindsofflawstheyhave,butthereisn’tmuchthatexplainsexactlywhyCstringsareflawedandalwayswillbe.I’lltrytoexplainthatrightnow,andafterdecadesofusingC’sstrings,there’senoughevidenceformetosaythattheyarejustabadidea.It’simpossibletoconfirmthatanygivenCstringisvalid:

•ACstringisinvalidifitdoesn’tendin'\0'.•AnyloopthatprocessesaninvalidCstringwillloopinfinitely(orjustcreateabufferoverflow).•Cstringsdon’thaveaknownlength,sotheonlywaytocheckifthey’reterminatedcorrectlyistoloopthroughthem.•Therefore,itisn’tpossibletovalidateaCstringwithoutpossiblyloopinginfinitely.

Thisissimplelogic.Youcan’twritealoopthatchecksifaCstringisvalidbecauseinvalidCstringscauseloopstoneverterminate.That’sit,andtheonlysolutionistoincludethesize.Onceyouknowthesize,youcanavoidtheinfiniteloopproblem.IfyoulookatthetwofunctionsIshowedyoufromExercise27,youseethis:

ex36.c


1voidcopy(charto[],charfrom[])2{3inti=0;45//whileloopwillnotendiffromisn't'\0'terminated6while((to[i]=from[i])!='\0'){7++i;8}9}1011intsafercopy(intfrom_len,char*from,intto_len,char*to)12{13inti=0;14intmax=from_len>to_len-1?to_len-1:from_len;1516//to_lenmusthaveatleast1byte17if(from_len<0||to_len<=0)18return-1;1920for(i=0;i<max;i++){21to[i]=from[i];22}2324to[to_len-1]='\0';

2526returni;27}

Imaginethatyouwanttoaddachecktothecopyfunctiontoconfirmthatthefromstringisvalid.Howwouldyoudothat?You’dwritealoopthatcheckedthatthestringendedin'\0'.Ohwait.Ifthestringdoesn’tendin'\0',thenhowdoesthecheckingloopend?Itdoesn’t.Checkmate.Nomatterwhatyoudo,youcan’tcheckthataCstringisvalidwithoutknowingthelengthoftheunderlyingstorage,andinthiscase,thesafercopyincludesthoselengths.Thisfunctiondoesn’thavethesameproblemsinceitsloopswillalwaysterminate,andevenifyoulietoitaboutthesize,youstillhavetogiveitafinitesize.WhattheBetterStringLibrarydoesiscreateastructurethatalwaysincludesthelengthofthestring’sstorage.Becausethelengthisalwaysavailabletoabstring,thenallofitsoperationscanbesafer.Theloopswillterminate,thecontentscanbevalidated,anditwon’thavethismajorflaw.Thelibraryalsocomeswithatonofoperationsyouneedwithstrings,likesplitting,formatting,andsearching,andthey’remostlikelydonerightandaresafer.Therecouldbeflawsinbstring,butit’sbeenaroundalongtime,sothoseareprobablyminimal.Theystillfindflawsinglibc,sowhat’saprogrammertodo,right?

UsingbstrlibTherearequiteafewimprovedstringlibraries,butIlikebstrlibbecauseitfitsinonefileforthebasics,andhasmostofthestuffyouneedtodealwithstrings.Inthisexerciseyou’llneedtogettwofiles,bstrlib.candbstrlib.h,fromtheBetterStringLibrary.Here’smedoingthisintheliblcthwprojectdirectory:

Exercise36Session


$mkdirbstrlib$cdbstrlib/$unzip~/Downloads/bstrlib-05122010.zipArchive:/Users/zedshaw/Downloads/bstrlib-05122010.zip

...

$lsbsafe.cbstraux.cbstrlib.h

bstrwrap.hlicense.txttest.cpp

bsafe.hbstraux.hbstrlib.txt

cpptest.cppporting.txttestaux.c

bstest.cbstrlib.cbstrwrap.cpp

gpl.txtsecurity.txt

$mvbstrlib.hbstrlib.c../src/lcthw/$cd../$rm-rfbstrlib#maketheedits$vimsrc/lcthw/bstrlib.c$makecleanall...

$

Online14,youseemeeditthebstrlib.cfiletomoveittoanewlocationandfixabugonOSX.Here’sthedifffile:

ex36.diff


25c25<#include"bstrlib.h"--->#include<lcthw/bstrlib.h>2759c2759<#ifdef__GNUC__--->#ifdefined(__GNUC__)&&!defined(__APPLE__)

HereIchangetheincludetobe<lcthw/bstrlib.h>,andthenfixoneoftheifdefatline2759.

LearningtheLibraryThisexerciseisshort,andit’smeanttosimplygetyoureadyfortheremainingexercisesthatusetheBetterStringLibrary.Inthenexttwoexercises,I’llusebstrlib.ctocreateahashmapdatastructure.Youshouldnowgetfamiliarwiththislibrarybyreadingtheheaderfileandtheimplementations,andthenwriteatests/bstr_tests.cthattestsoutthefollowingfunctions:

bfromcstrCreateabstringfromaCstyleconstant.blk2bstrDothesamething,butgivethelengthofthebuffer.bstrcpyCopyabstring.bassignSetonebstringtoanother.bassigncstrSetabstringtoaCstring’scontents.bassignblkSetabstringtoaCstringbutgivethelength.bdestroyDestroyabstring.bconcatConcatenateonebstringontoanother.bstricmpComparetwobstringsreturningthesameresultasstrcmp.biseqTestiftwobstringsareequal.binstrTellifonebstringisinanother.bfindreplaceFindonebstringinanother,thenreplaceitwithathird.bsplitSplitabstringintoabstrList.bformatDoaformatstring,whichissuperhandy.blengthGetthelengthofabstring.bdataGetthedatafromabstring.bcharGetacharfromabstring.

Yourtestshouldtryoutalloftheseoperations,andafewmorethatyoufindinterestingfromtheheaderfile.

Exercise37.Hashmaps

Hashmaps(hashmaps,hashes,orsometimesdictionaries)areusedfrequentlyindynamicprogrammingforstoringkey/valuedata.Ahashmapworksbyperformingahashingcalculationonthekeystoproduceaninteger,thenusesthatintegertofindabuckettogetorsetthevalue.It’saveryfast,practicaldatastructurebecauseitworksonnearlyanydataandiseasytoimplement.Here’sanexampleofusingahashmap(aka,dictionary)inPython:

ex37.py


fruit_weights={'Apples':10,'Oranges':100,'Grapes':1.0}

forkey,valueinfruit_weights.items():printkey,"=",value

Almosteverymodernlanguagehassomethinglikethis,somanypeopleendupwritingcodeandneverunderstandhowthisactuallyworks.BycreatingtheHashmapdatastructureinC,I’llshowyouhowthisworks.I’llstartwiththeheaderfilesoIcantalkaboutthedatastructure.

hashmap.h


#ifndef_lcthw_Hashmap_h

#define_lcthw_Hashmap_h

#include<stdint.h>


#defineDEFAULT_NUMBER_OF_BUCKETS100

typedefint(*Hashmap_compare)(void*a,void*b);typedefuint32_t(*Hashmap_hash)(void*key);

typedefstructHashmap{DArray*buckets;Hashmap_comparecompare;Hashmap_hashhash;}Hashmap;

typedefstructHashmapNode{void*key;void*data;uint32_thash;}HashmapNode;

typedefint(*Hashmap_traverse_cb)(HashmapNode*node);

Hashmap*Hashmap_create(Hashmap_comparecompare,Hashmap_hash);voidHashmap_destroy(Hashmap*map);

intHashmap_set(Hashmap*map,void*key,void*data);void*Hashmap_get(Hashmap*map,void*key);

intHashmap_traverse(Hashmap*map,Hashmap_traverse_cbtraverse_cb);

void*Hashmap_delete(Hashmap*map,void*key);

#endif

ThestructureconsistsofaHashmapthatcontainsanynumberofHashmapNodestructs.LookingatHashmap,youcanseethatit’sstructuredlikethis:

DArray*bucketsAdynamicarraythatwillbesettoafixedsizeof100buckets.EachbucketwillinturncontainaDArraythatwillholdHashmapNodepairs.

Hashmap_comparecompareThisisacomparisonfunctionthattheHashmapusestofindelementsbytheirkey.Itshouldworklikealloftheothercomparefunctions,anditdefaultstousingbstrcmpsothatkeysarejustbstrings.

Hashmap_hashhashThisisthehashingfunction,andit’sresponsiblefortakingakey,processingitscontents,andproducingasingleuint32_tindexnumber.You’llseethedefaultonesoon.

Thisalmosttellsyouhowthedataisstored,butthebucketsDArrayhasn’tbeencreatedyet.Justrememberthatit’skindofatwo-levelmapping:

•Thereare100bucketsthatmakeupthefirstlevel,andthingsareinthesebucketsbasedontheirhash.•EachbucketisaDArraythatcontainsHashmapNodestructsthataresimplyappendedtotheendasthey’readded.

TheHashmapNodeisthencomposedofthesethreeelements:void*keyThekeyforthiskey=valuepair.void*valueThevalue.uint32_thashThecalculatedhash,whichmakesfindingthisnodequicker.Wecanjustcheckthehashandskipanythatdon’tmatch,onlycheckingthekeyifit’sequal.

Therestoftheheaderfileisnothingnew,sonowIcanshowyoutheimplementationhashmap.cfile:

hashmap.c


1#undefNDEBUG2#include<stdint.h>3#include<lcthw/hashmap.h>4#include<lcthw/dbg.h>5#include<lcthw/bstrlib.h>67staticintdefault_compare(void*a,void*b)8{9returnbstrcmp((bstring)a,(bstring)b);10}1112/**13*SimpleBobJenkins'shashalgorithmtakenfromthe14*wikipediadescription.15*/16staticuint32_tdefault_hash(void*a)

17{18size_tlen=blength((bstring)a);19char*key=bdata((bstring)a);20uint32_thash=0;21uint32_ti=0;2223for(hash=i=0;i<len;++i){24hash+=key[i];25hash+=(hash<<10);26hash^=(hash>>6);27}2829hash+=(hash<<3);30hash^=(hash>>11);31hash+=(hash<<15);3233returnhash;34}3536Hashmap*Hashmap_create(Hashmap_comparecompare,Hashmap_hashhash)37{38Hashmap*map=calloc(1,sizeof(Hashmap));39check_mem(map);4041map->compare=compare==NULL?default_compare:compare;42map->hash=hash==NULL?default_hash:hash;43map->buckets=DArray_create(44sizeof(DArray*),DEFAULT_NUMBER_OF_BUCKETS);45map->buckets->end=map->buckets->max;//fakeoutexpandingit46check_mem(map->buckets);4748returnmap;4950error:51if(map){52Hashmap_destroy(map);53}5455returnNULL;56}5758voidHashmap_destroy(Hashmap*map)59{60inti=0;61intj=0;6263if(map){64if(map->buckets){65for(i=0;i<DArray_count(map->buckets);i++){66DArray*bucket=DArray_get(map->buckets,i);67if(bucket){68for(j=0;j<DArray_count(bucket);j++){69free(DArray_get(bucket,j));70}71DArray_destroy(bucket);72}73}74DArray_destroy(map->buckets);75}7677free(map);78}79}

8081staticinlineHashmapNode*Hashmap_node_create(inthash,void*key,82void*data)83{84HashmapNode*node=calloc(1,sizeof(HashmapNode));85check_mem(node);8687node->key=key;88node->data=data;89node->hash=hash;9091returnnode;9293error:94returnNULL;95}9697staticinlineDArray*Hashmap_find_bucket(Hashmap*map,void*key,98intcreate,99uint32_t*hash_out)100{101uint32_thash=map->hash(key);102intbucket_n=hash%DEFAULT_NUMBER_OF_BUCKETS;103check(bucket_n>=0,"Invalidbucketfound:%d",bucket_n);104//storeitforthereturnsothecallercanuseit105*hash_out=hash;106107DArray*bucket=DArray_get(map->buckets,bucket_n);108109if(!bucket&&create){110//newbucket,setitup111bucket=DArray_create(112sizeof(void*),DEFAULT_NUMBER_OF_BUCKETS);113check_mem(bucket);114DArray_set(map->buckets,bucket_n,bucket);115}116117returnbucket;118119error:120returnNULL;121}122123intHashmap_set(Hashmap*map,void*key,void*data)124{125uint32_thash=0;126DArray*bucket=Hashmap_find_bucket(map,key,1,&hash);127check(bucket,"Errorcan'tcreatebucket.");128129HashmapNode*node=Hashmap_node_create(hash,key,data);130check_mem(node);131132DArray_push(bucket,node);133134return0;135136error:137return-1;138}139140staticinlineintHashmap_get_node(Hashmap*map,uint32_thash,141DArray*bucket,void*key)142{

143inti=0;144145for(i=0;i<DArray_end(bucket);i++){146debug("TRY:%d",i);147HashmapNode*node=DArray_get(bucket,i);148if(node->hash==hash&&map->compare(node->key,key)==0){149returni;150}151}152153return-1;154}155156void*Hashmap_get(Hashmap*map,void*key)157{158uint32_thash=0;159DArray*bucket=Hashmap_find_bucket(map,key,0,&hash);160if(!bucket)returnNULL;161162inti=Hashmap_get_node(map,hash,bucket,key);163if(i==-1)returnNULL;164165HashmapNode*node=DArray_get(bucket,i);166check(node!=NULL,167"Failedtogetnodefrombucketwhenitshouldexist.");168169returnnode->data;170171error://fallthrough172returnNULL;173}174175intHashmap_traverse(Hashmap*map,Hashmap_traverse_cbtraverse_cb)176{177inti=0;178intj=0;179intrc=0;180181for(i=0;i<DArray_count(map->buckets);i++){182DArray*bucket=DArray_get(map->buckets,i);183if(bucket){184for(j=0;j<DArray_count(bucket);j++){185HashmapNode*node=DArray_get(bucket,j);186rc=traverse_cb(node);187if(rc!=0)188returnrc;189}190}191}192193return0;194}195196void*Hashmap_delete(Hashmap*map,void*key)197{198uint32_thash=0;199DArray*bucket=Hashmap_find_bucket(map,key,0,&hash);200if(!bucket)201returnNULL;202203inti=Hashmap_get_node(map,hash,bucket,key);204if(i==-1)205returnNULL;

206207HashmapNode*node=DArray_get(bucket,i);208void*data=node->data;209free(node);210211HashmapNode*ending=DArray_pop(bucket);212213if(ending!=node){214//alrightlookslikeit'snotthelastone,swapit215DArray_set(bucket,i,ending);216}217218returndata;219}

There’snothingverycomplicatedintheimplementation,butthedefault_hashandHashmap_find_bucketfunctionswillneedsomeexplanation.WhenyouuseHashmap_create,youcanpassinanycompareandhashfunctionsyouwant,butifyoudon’t,itusesthedefault_compareanddefault_hashfunctions.Thefirstthingtolookatishowdefault_hashdoesitsthing.ThisisasimplehashfunctioncalledaJenkinshashafterBobJenkins.Igotthealgorithmfromthe“Jenkinshash”pageonWikipedia.Itsimplygoesthrougheachbyteofthekeytohash(abstring),andthenitworksthebitssothattheendresultisasingleuint32_t.Itdoesthiswithsomeaddingandexclusiveor(XOR)operations.Therearemanydifferenthashfunctions,allwithdifferentproperties,butonceyouhaveone,youneedawaytouseittofindtherightbuckets.TheHashmap_find_bucketdoesitlikethis:

•First,itcallsmap->hash(key)togetthehashforthekey.•Itthenfindsthebucketusinghash%DEFAULT_NUMBER_OF_BUCKETS,soeveryhashwillalwaysfindsomebucketnomatterhowbigitis.•Itthengetsthebucket,whichisalsoaDArray,andifit’snotthere,itwillcreatethebucket.However,thatdependsonifthecreatevariablesaystodoso.•OnceithasfoundtheDArraybucketfortherighthash,itreturnsit,andthehash_outvariableisusedtogivethecallerthehashthatwasfound.

AlloftheotherfunctionsthenuseHashmap_find_buckettodotheirwork:•Settingakey/valueinvolvesfindingthebucket,makingaHashmapNode,andthenaddingittothebucket.•Gettingakeyinvolvesfindingthebucket,andthenfindingtheHashmapNodethatmatchesthehashandkeythatyouwant.•Deletinganitemfindsthebucket,findswheretherequestednodeis,andthenremovesitbyswappingthelastnodeintoitsplace.

TheonlyotherfunctionthatyoushouldstudyistheHashmap_traverse.Thissimplywalksthrougheverybucket,andforanybucketthathaspossiblevalues,itcallsthetraverse_cboneachvalue.ThisishowyouscanawholeHashmapforitsvalues.

TheUnitTestFinally,youhavetheunittesttotestalloftheseoperations:

hashmap_tests.c


1#include"minunit.h"2#include<lcthw/hashmap.h>3#include<assert.h>4#include<lcthw/bstrlib.h>56Hashmap*map=NULL;7staticinttraverse_called=0;8structtagbstringtest1=bsStatic("testdata1");9structtagbstringtest2=bsStatic("testdata2");10structtagbstringtest3=bsStatic("xestdata3");11structtagbstringexpect1=bsStatic("THEVALUE1");12structtagbstringexpect2=bsStatic("THEVALUE2");13structtagbstringexpect3=bsStatic("THEVALUE3");1415staticinttraverse_good_cb(HashmapNode*node)16{17debug("KEY:%s",bdata((bstring)node->key));18traverse_called++;19return0;20}2122staticinttraverse_fail_cb(HashmapNode*node)23{24debug("KEY:%s",bdata((bstring)node->key));25traverse_called++;2627if(traverse_called==2){28return1;29}else{30return0;31}32}3334char*test_create()35{36map=Hashmap_create(NULL,NULL);37mu_assert(map!=NULL,"Failedtocreatemap.");3839returnNULL;40}4142char*test_destroy()43{44Hashmap_destroy(map);4546returnNULL;47}4849char*test_get_set()50{51intrc=Hashmap_set(map,&test1,&expect1);52mu_assert(rc==0,"Failedtoset&test1");53bstringresult=Hashmap_get(map,&test1);54mu_assert(result==&expect1,"Wrongvaluefortest1.");5556rc=Hashmap_set(map,&test2,&expect2);57mu_assert(rc==0,"Failedtosettest2");58result=Hashmap_get(map,&test2);59mu_assert(result==&expect2,"Wrongvaluefortest2.");6061rc=Hashmap_set(map,&test3,&expect3);

62mu_assert(rc==0,"Failedtosettest3");63result=Hashmap_get(map,&test3);64mu_assert(result==&expect3,"Wrongvaluefortest3.");6566returnNULL;67}6869char*test_traverse()70{71intrc=Hashmap_traverse(map,traverse_good_cb);72mu_assert(rc==0,"Failedtotraverse.");73mu_assert(traverse_called==3,"Wrongcounttraverse.");7475traverse_called=0;76rc=Hashmap_traverse(map,traverse_fail_cb);77mu_assert(rc==1,"Failedtotraverse.");78mu_assert(traverse_called==2,"Wrongcounttraverseforfail.");7980returnNULL;81}8283char*test_delete()84{85bstringdeleted=(bstring)Hashmap_delete(map,&test1);86mu_assert(deleted!=NULL,"GotNULLondelete.");87mu_assert(deleted==&expect1,"Shouldgettest1");88bstringresult=Hashmap_get(map,&test1);89mu_assert(result==NULL,"Shoulddelete.");9091deleted=(bstring)Hashmap_delete(map,&test2);92mu_assert(deleted!=NULL,"GotNULLondelete.");93mu_assert(deleted==&expect2,"Shouldgettest2");94result=Hashmap_get(map,&test2);95mu_assert(result==NULL,"Shoulddelete.");9697deleted=(bstring)Hashmap_delete(map,&test3);98mu_assert(deleted!=NULL,"GotNULLondelete.");99mu_assert(deleted==&expect3,"Shouldgettest3");100result=Hashmap_get(map,&test3);101mu_assert(result==NULL,"Shoulddelete.");102103returnNULL;104}105106char*all_tests()107{108mu_suite_start();109110mu_run_test(test_create);111mu_run_test(test_get_set);112mu_run_test(test_traverse);113mu_run_test(test_delete);114mu_run_test(test_destroy);115116returnNULL;117}118119RUN_TESTS(all_tests);

TheonlythingtolearnaboutthisunittestisthatatthetopIuseafeatureofbstringtocreatestaticstringstoworkwithinthetests.IusethetagbstringandbsStatictocreatethemonlines7–13.

HowtoImproveItThisisaverysimpleimplementationofHashmap,asaremostoftheotherdatastructuresinthisbook.Mygoalisn’ttogiveyouinsanelygreat,hyper-speed,well-tuneddatastructures.Usuallythosearemuchtoocomplicatedtodiscussandonlydistractyoufromthereal,basicdatastructureatwork.Mygoalistogiveyouanunderstandablestartingpointtothenimproveuponorbetterunderstandtheimplementation.Inthiscase,therearesomethingsyoucandowiththisimplementation:

•Youcanuseasortoneachbucketsothatthey’realwayssorted.Thisincreasesyourinserttimebutdecreasesyourfindtime,becauseyoucanthenuseabinarysearchtofindeachnode.Rightnow,it’sloopingthroughallofthenodesinabucketjusttofindone.•Youcandynamicallysizethenumberofbuckets,orletthecallerspecifythenumberforeachHashmapcreated.•Youcanuseabetterdefault_hash.Therearetonsofthem.•This(andnearlyeveryHashmap)isvulnerabletosomeonepickingkeysthatwillfillonlyonebucket,andthentrickingyourprogramintoprocessingthem.ThisthenmakesyourprogramrunslowerbecauseitchangesfromprocessingaHashmaptoeffectivelyprocessingasingleDArray.Ifyousortthenodesinthebucket,thishelps,butyoucanalsousebetterhashingfunctions,andforthereallyparanoidprogrammer,addarandomsaltsothatkeyscan’tbepredicted.•Youcouldhaveitdeletebucketsthatareemptyofnodestosavespace,orputemptybucketsintoacachesoyoucansaveontimelostcreatinganddestroyingthem.•Rightnow,itjustaddselementseveniftheyalreadyexist.Writeanalternativesetmethodthatonlyaddsanelementifitisn’tsetalready.

Asusual,youshouldgothrougheachfunctionandmakeitbulletproof.TheHashmapcouldalsouseadebugsettingfordoinganinvariantcheck.

ExtraCredit•ResearchtheHashmapimplementationinyourfavoriteprogramminglanguagetoseewhatfeaturesithas.•FindoutwhatthemajordisadvantagesofaHashmapareandhowtoavoidthem.Forexample,itdoesn’tpreserveorderwithoutspecialchanges,nordoesitworkwhenyouneedtofindthingsbasedonpartsofkeys.•WriteaunittestthatdemonstratesthedefectoffillingaHashmapwithkeysthatlandinthesamebucket,thentesthowthisimpactsperformance.Agoodwaytodothisistojustreducethenumberofbucketstosomethingstupid,likefive.

Exercise38.HashmapAlgorithms

Therearethreehashfunctionsthatyou’llimplementinthisexercise:FNV-1aNamedafterthecreatorsGlennFowler,PhongVo,andLandonCurtNoll,thishashproducesgoodnumbersandisreasonablyfast.

Adler-32NamedafterMarkAdler,thisisahorriblehashalgorithm,butit’sbeenaroundalongtimeandit’sgoodforstudying.

DJBHashThishashalgorithmisattributedtoDanJ.Bernstein(DJB),butit’sdifficulttofindhisdiscussionofthealgorithm.It’sshowntobefast,butpossiblynotgreatnumbers.

You’vealreadyseentheJenkinshashasthedefaulthashfortheHashmapdatastructure,sothisexercisewillbelookingatthesethreenewhashfunctions.Thecodeforthemisusuallysmall,andit’snotoptimizedatall.Asusual,I’mgoingforunderstandingandnotblindingspeed.Theheaderfileisverysimple,soI’llstartwiththat:

hashmap_algos.h


#ifndefhashmap_algos_h

#definehashmap_algos_h

#include<stdint.h>

uint32_tHashmap_fnv1a_hash(void*data);

uint32_tHashmap_adler32_hash(void*data);

uint32_tHashmap_djb_hash(void*data);

#endif

I’mjustdeclaringthethreefunctionsI’llimplementinthehashmap_algos.cfile:

hashmap_algos.c


1#include<lcthw/hashmap_algos.h>2#include<lcthw/bstrlib.h>34//settingstakenfrom5//http://www.isthe.com/chongo/tech/comp/fnv/index.html#FNV-param6constuint32_tFNV_PRIME=16777619;7constuint32_tFNV_OFFSET_BASIS=2166136261;89uint32_tHashmap_fnv1a_hash(void*data)10{11bstrings=(bstring)data;12uint32_thash=FNV_OFFSET_BASIS;13inti=0;1415for(i=0;i<blength(s);i++){16hash^=bchare(s,i,0);

17hash*=FNV_PRIME;18}1920returnhash;21}2223constintMOD_ADLER=65521;2425uint32_tHashmap_adler32_hash(void*data)26{27bstrings=(bstring)data;28uint32_ta=1,b=0;29inti=0;3031for(i=0;i<blength(s);i++){32a=(a+bchare(s,i,0))%MOD_ADLER;33b=(b+a)%MOD_ADLER;34}3536return(b<<16)|a;37}3839uint32_tHashmap_djb_hash(void*data)40{41bstrings=(bstring)data;42uint32_thash=5381;43inti=0;4445for(i=0;i<blength(s);i++){46hash=((hash<<5)+hash)+bchare(s,i,0);/*hash*33+c*/47}4849returnhash;50}

Thisfile,then,hasthethreehashalgorithms.YoushouldnoticethatI’mjustusingabstringforthekey,butI’musingthebcharefunctiontogetacharacterfromthebstring,butreturning0ifthatcharacterisoutsidethestring’slength.Eachofthesealgorithmsarefoundonline,sogosearchforthemandreadaboutthem.Again,IprimarilyusedWikipediaandthenfollowedittoothersources.Ithenhaveaunittestthattestsouteachalgorithm,butitalsotestswhetheritwilldistributewellacrossanumberofbuckets:

hashmap_algos_tests.c


1#include<lcthw/bstrlib.h>2#include<lcthw/hashmap.h>3#include<lcthw/hashmap_algos.h>4#include<lcthw/darray.h>5#include"minunit.h"67structtagbstringtest1=bsStatic("testdata1");8structtagbstringtest2=bsStatic("testdata2");9structtagbstringtest3=bsStatic("xestdata3");1011char*test_fnv1a()12{

13uint32_thash=Hashmap_fnv1a_hash(&test1);14mu_assert(hash!=0,"Badhash.");1516hash=Hashmap_fnv1a_hash(&test2);17mu_assert(hash!=0,"Badhash.");1819hash=Hashmap_fnv1a_hash(&test3);20mu_assert(hash!=0,"Badhash.");2122returnNULL;23}2425char*test_adler32()26{27uint32_thash=Hashmap_adler32_hash(&test1);28mu_assert(hash!=0,"Badhash.");2930hash=Hashmap_adler32_hash(&test2);31mu_assert(hash!=0,"Badhash.");3233hash=Hashmap_adler32_hash(&test3);34mu_assert(hash!=0,"Badhash.");3536returnNULL;37}3839char*test_djb()40{41uint32_thash=Hashmap_djb_hash(&test1);42mu_assert(hash!=0,"Badhash.");4344hash=Hashmap_djb_hash(&test2);45mu_assert(hash!=0,"Badhash.");4647hash=Hashmap_djb_hash(&test3);48mu_assert(hash!=0,"Badhash.");4950returnNULL;51}5253#defineBUCKETS10054#defineBUFFER_LEN2055#defineNUM_KEYSBUCKETS*100056enum{ALGO_FNV1A,ALGO_ADLER32,ALGO_DJB};5758intgen_keys(DArray*keys,intnum_keys)59{60inti=0;61FILE*urand=fopen("/dev/urandom","r");62check(urand!=NULL,"Failedtoopen/dev/urandom");6364structbStream*stream=bsopen((bNread)fread,urand);65check(stream!=NULL,"Failedtoopen/dev/urandom");6667bstringkey=bfromcstr("");68intrc=0;6970//FNV1ahistogram71for(i=0;i<num_keys;i++){72rc=bsread(key,stream,BUFFER_LEN);73check(rc>=0,"Failedtoreadfrom/dev/urandom.");7475DArray_push(keys,bstrcpy(key));

76}7778bsclose(stream);79fclose(urand);80return0;8182error:83return-1;84}8586voiddestroy_keys(DArray*keys)87{88inti=0;89for(i=0;i<NUM_KEYS;i++){90bdestroy(DArray_get(keys,i));91}9293DArray_destroy(keys);94}9596voidfill_distribution(int*stats,DArray*keys,97Hashmap_hashhash_func)98{99inti=0;100uint32_thash=0;101102for(i=0;i<DArray_count(keys);i++){103hash=hash_func(DArray_get(keys,i));104stats[hash%BUCKETS]+=1;105}106107}108109char*test_distribution()110{111inti=0;112intstats[3][BUCKETS]={{0}};113DArray*keys=DArray_create(0,NUM_KEYS);114115mu_assert(gen_keys(keys,NUM_KEYS)==0,116"Failedtogeneraterandomkeys.");117118fill_distribution(stats[ALGO_FNV1A],keys,Hashmap_fnv1a_hash);119fill_distribution(stats[ALGO_ADLER32],keys,Hashmap_adler32_hash);120fill_distribution(stats[ALGO_DJB],keys,Hashmap_djb_hash);121122fprintf(stderr,"FNV\tA32\tDJB\n");123124for(i=0;i<BUCKETS;i++){125fprintf(stderr,"%d\t%d\t%d\n",126stats[ALGO_FNV1A][i],127stats[ALGO_ADLER32][i],stats[ALGO_DJB][i]);128}129130destroy_keys(keys);131132returnNULL;133}134135char*all_tests()136{137mu_suite_start();138

139mu_run_test(test_fnv1a);140mu_run_test(test_adler32);141mu_run_test(test_djb);142mu_run_test(test_distribution);143144returnNULL;145}146147RUN_TESTS(all_tests);

IhavethenumberofBUCKETSinthiscodesetfairlyhigh,sinceIhaveafastenoughcomputer,butifitrunsslow,justloweritandNUM_KEYS.WhatthistestletsmedoisrunthetestandthenlookatthedistributionofkeysforeachhashfunctionusingabitofanalysiswithalanguagecalledR.Idothisbycraftingabiglistofkeysusingthegen_keysfunction.Thesekeysaretakenoutofthe/dev/urandomdeviceandarerandombytekeys.Ithenusethesekeystohavethefill_distributionfunctionfillupthestatsarraywithwherethosekeyswouldhashinatheoreticalsetofbuckets.Allthisfunctiondoesisgothroughallofthekeys,dothehash,thendowhattheHashmapwoulddotofinditsbucket.Finally,I’msimplyprintingoutathree-columntablewiththefinalcountforeachbucket,showinghowmanykeysmanagedtogetintoeachbucketrandomly.Icanthenlookatthesenumberstoseeifthehashfunctionsaredistributingkeysevenly.

WhatYouShouldSeeTeachingyouRisoutsidethescopeofthisbook,butifyouwanttogetitandtrythis,itcanbefoundatwww.r-project.org.Hereisanabbreviatedshellsessionthatshowsmerunningtests/hashmap_algos_testtogetthetableproducedbytest_distribution(notshownhere),andthenusingRtoseewhatthesummarystatisticsare.

Exercise38Session


$tests/hashmap_algos_tests#copy-pastethetableitprintsout$vimhash.txt$R>hash<-read.table("hash.txt",header=T)>summary(hash)FNVA32DJBMin.:945Min.:908.0Min.:9271stQu.:9801stQu.:980.81stQu.:979Median:998Median:1000.0Median:998Mean:1000Mean:1000.0Mean:10003rdQu.:10163rdQu.:1019.23rdQu.:1021Max.:1072Max.:1075.0Max.:1082>

First,Ijustrunthetest,whichonyourscreenwillprintthetable.Then,Ijustcopy-pasteitoutofmyterminalandusevimhash.txttosavethedata.Ifyoulookatthedata,ithastheheaderFNVA32DJBforeachofthethreealgorithms.Secondly,IrunRandloadthedatausingtheread.tablecommand.Thisisasmartfunctionthat

http://www.r-project.org

workswiththiskindoftab-delimiteddata,andIonlyhavetotellitheader=Tforittoknowthatthedatahasaheader.Finally,Ihavethedataloadedandcanusesummarytoprintoutitssummarystatisticsforeachcolumn.Hereyoucanseethateachfunctionactuallydoesalrightwiththisrandomdata.Here’swhateachoftheserowsmeans:

Min.Thisistheminimumvaluefoundforthedatainthatcolumn.FNV-laseemstowinonthisrunsinceithasthelargestnumber,meaningithasatighterrangeatthelowend.

1stQu.Thisisthepointwherethefirstquarterofthedataends.MedianThisisthenumberthat’sinthemiddleifyousortedthem.Medianismostusefulwhencomparedtomean.

MeanMeanistheaveragemostpeoplethinkof,andit’sthesumdividedbythecountofthedata.Ifyoulook,allofthemare1,000,whichisgreat.Ifyoucomparethistothemedian,youseethatallthreehavereallyclosemedianstothemean.Whatthismeansisthedataisn’tskewedinonedirection,soyoucantrustthemean.

3rdQu.Thisisthepointwherethelastquarterofthedatastartsandrepresentsthetailendofthenumbers.

Max.Thisisthemaximumnumberofthedata,andpresentstheupperboundonallofthem.Lookingatthisdata,youseethatallofthesehashesseemtodowellonrandomkeys,andthemeansmatchtheNUM_KEYSsettingthatImade.WhatI’mlookingforisthis:IfImake1,000keysperbucket(BUCKETS×1000),thenonaverageeachbucketshouldhave1,000keysinit.Ifthehashfunctionisn’tworking,thenyou’llseethesesummarystatisticsshowameanthat’snot1,000,andreallyhighrangesatthefirstandthirdquarters.Agoodhashfunctionshouldhaveadead-on1,000mean,andastightarangeaspossible.Youshouldalsoknowthatyou’llgetdifferentnumbersfrommine,andevenbetweendifferentrunsofthisunittest.

HowtoBreakItI’mfinallygoingtohaveyoudosomebreakinginthisexercise.Iwantyoutowritetheworsthashfunctionyoucan,andthenusethedatatoprovethatit’sreallybad.YoucanuseRtodothestatistics,justlikeIdid,butmaybeyouhaveanothertoolthatyoucanusetogiveyouthesamesummarystatistics.Thegoalistomakeahashfunctionthatseemsnormaltoanuntrainedeye,butwhenactuallyrun,ithasabadmeanandisallovertheplace.Thatmeansyoucan’tjusthaveitreturn1.Youhavetogiveastreamofnumbersthatseemalrightbutaren’t,andthey’reloadingupsomebucketstoomuch.ExtrapointsifyoucanmakeaminimalchangetooneofthefourhashalgorithmsthatIgaveyoutodothis.Thepurposeofthisexerciseistoimaginethatsomefriendlycodercomestoyouandofferstoimproveyourhashfunction,butactuallyjustmakesanicelittlebackdoorthatreallyscrewsupyourHashmap.AstheRoyalSocietysays,“Nulliusinverba.”

ExtraCredit•Takethedefault_hashoutofthehashmap.c,makeitoneofthealgorithmsinhashmap_algos.c,andthenmakeallofthetestsworkagain.•Addthedefault_hashtothehashmap_algos_tests.ctestandcompareitsstatisticstotheotherhashfunctions.•Findafewmorehashfunctionsandaddthem,too.Youcanneverhavetoomanyhashfunctions!

Exercise39.StringAlgorithms

Inthisexercise,I’mgoingtoshowyouasupposedlyfasterstringsearchalgorithm,calledbinstr,andcompareittotheonethatexistsinbstrlib.c.Thedocumentationforbinstrsaysthatitusesasimple“bruteforce”stringsearchtofindthefirstinstance.TheonethatI’llimplementwillusetheBoyer-Moore-Horspool(BMH)algorithm,whichissupposedtobefasterifyouanalyzethetheoreticaltime.Assumingmyimplementationisn’tflawed,you’llseethatthepracticaltimeforBMHismuchworsethanthesimplebruteforceofbinstr.Thepointofthisexerciseisn’treallytoexplainthealgorithm,becauseit’ssimpleenoughforyoutoreadthe“Boyer-Moore-Horspoolalgorithm”pageonWikipedia.Thegistofthisalgorithmisthatitcalculatesaskipcharacterslistasafirstoperation,thenitusesthislisttoquicklyscanthroughthestring.It’ssupposedtobefasterthanbruteforce,solet’sgetthecodeintotherightfilesandsee.First,Ihavetheheader:

string_algos.h


#ifndefstring_algos_h

#definestring_algos_h

#include<lcthw/bstrlib.h>


typedefstructStringScanner{bstringin;constunsignedchar*haystack;ssize_thlen;constunsignedchar*needle;ssize_tnlen;size_tskip_chars[UCHAR_MAX+1];}StringScanner;

intString_find(bstringin,bstringwhat);

StringScanner*StringScanner_create(bstringin);

intStringScanner_scan(StringScanner*scan,bstringtofind);

voidStringScanner_destroy(StringScanner*scan);

#endif

Inordertoseetheeffectsofthisskipcharacterslist,I’mgoingtomaketwoversionsoftheBMHalgorithm:

String_findThissimplyfindsthefirstinstanceofonestringinanother,doingtheentirealgorithminoneshot.

StringScanner_scanThisusesaStringScannerstatestructuretoseparatetheskiplistbuildfromtheactualfind.Thiswillletmeseewhatimpactthathasonperformance.Thismodelalsogivesmetheadvantageofincrementallyscanningforonestringinanotherandquicklyfindingallinstances.

Onceyouhavethat,here’stheimplementation:

string_algos.c


1#include<lcthw/string_algos.h>2#include<limits.h>34staticinlinevoidString_setup_skip_chars(size_t*skip_chars,5constunsignedchar*needle,6ssize_tnlen)7{8size_ti=0;9size_tlast=nlen-1;1011for(i=0;i<UCHAR_MAX+1;i++){12skip_chars[i]=nlen;13}1415for(i=0;i<last;i++){16skip_chars[needle[i]]=last-i;17}18}1920staticinlineconstunsignedchar*String_base_search(constunsigned21char*haystack,22ssize_thlen,23constunsigned24char*needle,25ssize_tnlen,26size_t*27skip_chars)28{29size_ti=0;30size_tlast=nlen-1;3132assert(haystack!=NULL&&"Givenbadhaystacktosearch.");33assert(needle!=NULL&&"Givenbadneedletosearchfor.");3435check(nlen>0,"nlencan'tbe<=0");36check(hlen>0,"hlencan'tbe<=0");3738while(hlen>=nlen){39for(i=last;haystack[i]==needle[i];i--){40if(i==0){41returnhaystack;42}43}4445hlen-=skip_chars[haystack[last]];46haystack+=skip_chars[haystack[last]];47}4849error://fallthrough50returnNULL;51}5253intString_find(bstringin,bstringwhat)54{55constunsignedchar*found=NULL;56

57constunsignedchar*haystack=(constunsignedchar*)bdata(in);58ssize_thlen=blength(in);59constunsignedchar*needle=(constunsignedchar*)bdata(what);60ssize_tnlen=blength(what);61size_tskip_chars[UCHAR_MAX+1]={0};6263String_setup_skip_chars(skip_chars,needle,nlen);6465found=String_base_search(haystack,hlen,66needle,nlen,skip_chars);6768returnfound!=NULL?found-haystack:-1;69}7071StringScanner*StringScanner_create(bstringin)72{73StringScanner*scan=calloc(1,sizeof(StringScanner));74check_mem(scan);7576scan->in=in;77scan->haystack=(constunsignedchar*)bdata(in);78scan->hlen=blength(in);7980assert(scan!=NULL&&"fuck");81returnscan;8283error:84free(scan);85returnNULL;86}8788staticinlinevoidStringScanner_set_needle(StringScanner*scan,89bstringtofind)90{91scan->needle=(constunsignedchar*)bdata(tofind);92scan->nlen=blength(tofind);9394String_setup_skip_chars(scan->skip_chars,scan->needle,scan->nlen);95}9697staticinlinevoidStringScanner_reset(StringScanner*scan)98{99scan->haystack=(constunsignedchar*)bdata(scan->in);100scan->hlen=blength(scan->in);101}102103intStringScanner_scan(StringScanner*scan,bstringtofind)104{105constunsignedchar*found=NULL;106ssize_tfound_at=0;107108if(scan->hlen<=0){109StringScanner_reset(scan);110return-1;111}112113if((constunsignedchar*)bdata(tofind)!=scan->needle){114StringScanner_set_needle(scan,tofind);115}116117found=String_base_search(scan->haystack,scan->hlen,118scan->needle,scan->nlen,119scan->skip_chars);

120121if(found){122found_at=found-(constunsignedchar*)bdata(scan->in);123scan->haystack=found+scan->nlen;124scan->hlen-=found_at-scan->nlen;125}else{126//done,resetthesetup127StringScanner_reset(scan);128found_at=-1;129}130131returnfound_at;132}133134voidStringScanner_destroy(StringScanner*scan)135{136if(scan){137free(scan);138}139}

TheentirealgorithmisintwostaticinlinefunctionscalledString_setup_skip_charsandString_base_search.ThesearethenusedintheotherfunctionstoactuallyimplementthesearchingstylesIwant.StudythesefirsttwofunctionsandcomparethemtotheWikipediadescriptionsothatyouknowwhat’sgoingon.TheString_findthenjustusesthesetwofunctionstodoafindandreturnthepositionfound.It’sverysimple,andI’lluseittoseehowthisbuildskip_charsphaseimpactsreal,practicalperformance.Keepinmindthatyoucouldmaybemakethisfaster,butI’mteachingyouhowtoconfirmtheoreticalspeedafteryouimplementanalgorithm.TheStringScanner_scanfunctionthenfollowsthecommonpatternIuseofcreate,scan,anddestroy,andisusedtoincrementallyscanastringforanotherstring.You’llseehowthisisusedwhenIshowyoutheunittestthatwilltestthisout.Finally,Ihavetheunittestthatfirstconfirmsthatthisisallworking,thenitrunssimpleperformancetestsforallthree,findingalgorithmsinacommentedoutsection.

string_algos_tests.c


1#include"minunit.h"2#include<lcthw/string_algos.h>3#include<lcthw/bstrlib.h>4#include<time.h>56structtagbstringIN_STR=bsStatic(7"IhaveALPHAbetaALPHAandorangesALPHA");8structtagbstringALPHA=bsStatic("ALPHA");9constintTEST_TIME=1;1011char*test_find_and_scan()12{13StringScanner*scan=StringScanner_create(&IN_STR);14mu_assert(scan!=NULL,"Failedtomakethescanner.");1516intfind_i=String_find(&IN_STR,&ALPHA);17mu_assert(find_i>0,"Failedtofind'ALPHA'inteststring.");

1819intscan_i=StringScanner_scan(scan,&ALPHA);20mu_assert(scan_i>0,"Failedtofind'ALPHA'withscan.");21mu_assert(scan_i==find_i,"findandscandon'tmatch");2223scan_i=StringScanner_scan(scan,&ALPHA);24mu_assert(scan_i>find_i,25"shouldfindanotherALPHAafterthefirst");2627scan_i=StringScanner_scan(scan,&ALPHA);28mu_assert(scan_i>find_i,29"shouldfindanotherALPHAafterthefirst");3031mu_assert(StringScanner_scan(scan,&ALPHA)==-1,32"shouldn'tfindit");3334StringScanner_destroy(scan);3536returnNULL;37}3839char*test_binstr_performance()40{41inti=0;42intfound_at=0;43unsignedlongfind_count=0;44time_telapsed=0;45time_tstart=time(NULL);4647do{48for(i=0;i<1000;i++){49found_at=binstr(&IN_STR,0,&ALPHA);50mu_assert(found_at!=BSTR_ERR,"Failedtofind!");51find_count++;52}5354elapsed=time(NULL)-start;55}while(elapsed<=TEST_TIME);5657debug("BINSTRCOUNT:%lu,ENDTIME:%d,OPS:%f",58find_count,(int)elapsed,(double)find_count/elapsed);59returnNULL;60}6162char*test_find_performance()63{64inti=0;65intfound_at=0;66unsignedlongfind_count=0;67time_telapsed=0;68time_tstart=time(NULL);6970do{71for(i=0;i<1000;i++){72found_at=String_find(&IN_STR,&ALPHA);73find_count++;74}7576elapsed=time(NULL)-start;77}while(elapsed<=TEST_TIME);7879debug("FINDCOUNT:%lu,ENDTIME:%d,OPS:%f",80find_count,(int)elapsed,(double)find_count/elapsed);81

82returnNULL;83}8485char*test_scan_performance()86{87inti=0;88intfound_at=0;89unsignedlongfind_count=0;90time_telapsed=0;91StringScanner*scan=StringScanner_create(&IN_STR);9293time_tstart=time(NULL);9495do{96for(i=0;i<1000;i++){97found_at=0;9899do{100found_at=StringScanner_scan(scan,&ALPHA);101find_count++;102}while(found_at!=-1);103}104105elapsed=time(NULL)-start;106}while(elapsed<=TEST_TIME);107108debug("SCANCOUNT:%lu,ENDTIME:%d,OPS:%f",109find_count,(int)elapsed,(double)find_count/elapsed);110111StringScanner_destroy(scan);112113returnNULL;114}115116char*all_tests()117{118mu_suite_start();119120mu_run_test(test_find_and_scan);121122//thisisanidiomforcommentingoutsectionsofcode123#if0124mu_run_test(test_scan_performance);125mu_run_test(test_find_performance);126mu_run_test(test_binstr_performance);127#endif128129returnNULL;130}131132RUN_TESTS(all_tests);

Ihaveitwrittenherewith#if0,whichisawaytousetheCPPtocommentoutasectionofcode.Typeitinlikethis,andthenremoveitandthe#endifsothatyoucanseetheseperformancetestsrun.Asyoucontinuewiththebook,simplycommenttheseoutsothatthetestdoesn’twastedevelopmenttime.There’snothingamazinginthisunittest;itjustrunseachofthedifferentfunctionsinloopsthatlastlongenoughtogetafewsecondsofsampling.Thefirsttest(test_find_and_scan)justconfirmsthatwhatI’vewrittenworks,becausethere’snopointintestingthespeedofsomethingthatdoesn’twork.Then,thenextthreefunctionsrunalargenumberofsearches,usingeachofthethree

functions.ThetricktonoticeisthatIgrabthestartingtimeinstart,andthenIloopuntilatleastTEST_TIMEsecondshavepassed.ThismakessurethatIgetenoughsamplestoworkwithwhilecomparingthethree.I’llthenrunthistestwithdifferentTEST_TIMEsettingsandanalyzetheresults.

WhatYouShouldSeeWhenIrunthistestonmylaptop,Igetnumbersthatlooklikethis:

Exercise39.1Session


$./tests/string_algos_testsDEBUGtests/string_algos_tests.c:124:-----RUNNING:

./tests/string_algos_tests

----

RUNNING:./tests/string_algos_tests

DEBUGtests/string_algos_tests.c:116:

-----test_find_and_scan


-----test_scan_performance

DEBUGtests/string_algos_tests.c:105:SCANCOUNT:\

110272000,ENDTIME:2,OPS:55136000.000000


-----test_find_performance

DEBUGtests/string_algos_tests.c:76:FINDCOUNT:\

12710000,ENDTIME:2,OPS:6355000.000000


-----test_binstr_performance

DEBUGtests/string_algos_tests.c:54:BINSTRCOUNT:\

72736000,ENDTIME:2,OPS:36368000.000000

ALLTESTSPASSED

Testsrun:4

$

IlookatthisandIwanttodomorethan2secondsforeachrun.Iwanttorunthismanytimes,andthenuseRtocheckitoutlikeIdidbefore.Here’swhatIgetfortensamplesforabout10secondseach:

scanfindbinstr71195200635370037110200750980006358400374208007491000063513003726360074859600658610037133200733456006365200375497007475440063580003716240075343600663040037075000738048006439900368587007499520063843003681170074781200644950037383000

ThewayIgotthisisusingalittlebitofshellhelp,andtheneditingtheoutput:

Exercise39.2Session


$foriin12345678910>doecho"RUN---$i">>times.log>./tests/string_algos_tests2>&1|grepCOUNT>>times.log>done$lesstimes.log$vimtimes.log

Rightaway,youcanseethatthescanningsystembeatsthepantsoffbothoftheothers,butI’llopenthisinRandconfirmtheresults:

Exercise39.3Session


>times<-read.table("times.log",header=T)>summary(times)scanfindbinstrMin.:71195200Min.:6351300Min.:368117001stQu.:740422001stQu.:63581001stQu.:37083800Median:74820400Median:6374750Median:37147800Mean:74308760Mean:6427680Mean:371768303rdQu.:749739003rdQu.:64471003rdQu.:37353150Max.:75343600Max.:6630400Max.:37549700>

TounderstandwhyI’mgettingthesummarystatistics,Ihavetoexplainsomestatisticsforyou.WhatI’mlookingforinthesenumbersissimplythis:“Arethesethreefunctions(scan,find,bsinter)actuallydifferent?”IknowthateachtimeIrunmytesterfunction,Igetslightlydifferentnumbers,andthosenumberscancoveracertainrange.Youseeherethatthefirstandthirdquartersdothatforeachsample.WhatIlookatfirstisthemean,andIwanttoseeifeachsample’smeanisdifferentfromtheothers.Icanseethat,andclearlythescanbeatsbinstr,whichalsobeatsfind.However,Ihaveaproblem.IfIusejustthemean,there’sachancethattherangesofeachsamplemightoverlap.WhatifIhavemeansthataredifferent,butthefirstandthirdquartersoverlap?Inthatcase,IcouldsaythatifIranthesamplesagainthere’sachancethatthemeansmightnotbedifferent.ThemoreoverlapIhaveintheranges,thehigherprobabilitythatmytwosamples(andmytwofunctions)arenotactuallydifferent.AnydifferencethatI’mseeinginthetwo(inthiscasethree)isjustrandomchance.Therearemanytoolsthatyoucanusetosolvethisproblem,butinourcase,Icanjustlookatthefirstandthirdquartersandthemeanforallthreesamples.Ifthemeansaredifferent,andthequartersarewayoffwithnopossibilityofoverlapping,thenit’salrighttosaythattheyaredifferent.Inmythreesamples,Icansaythatscan,find,andbinstraredifferent,don’toverlapinrange,andIcantrustthesample(forthemostpart).

AnalyzingtheResultsLookingattheresults,IcanseethatString_findismuchslowerthantheothertwo.Infact,it’ssoslowthatI’dthinkthere’ssomethingwrongwithhowIimplementedit.However,whenIcompareittoStringScanner_scan,Icanseethatit’smostlikelythepartthatbuildstheskiplistthat’scostingthetime.Notonlyisfindslower,it’salsodoinglessthanscanbecauseit’sjustfindingthefirststringwhilescanfindsallofthem.

Icanalsoseethatscanbeatsbinstr,aswell,andbyquitealargemargin.Again,notonlydoesscandomorethanbothofthese,butit’salsomuchfaster.Thereareafewcaveatswiththisanalysis:

•Imayhavemessedupthisimplementationorthetest.AtthispointIwouldgoresearchallofthepossiblewaystodoaBMHalgorithmandtrytoimproveit.IwouldalsoconfirmthatI’mdoingthetestright.•Ifyoualterthetimethetestruns,you’llgetdifferentresults.Thereisawarm-upperiodthatI’mnotinvestigating.•Thetest_scan_performanceunittestisn’tquitethesameastheothers,butit’sdoingmorethantheothertests,soit’sprobablyalright.•I’monlydoingthetestbysearchingforonestringinanother.Icouldrandomizethestringstofindtheirpositionandlengthasaconfoundingfactor.•Maybebinstrisimplementedbetterthansimplebruteforce.•Icouldberunningtheseinanunfortunateorder.Mayberandomizingwhichtestrunsfirstwillgivebetterresults.

Onethingtogatherfromthisisthatyouneedtoconfirmrealperformanceevenifyouimplementanalgorithmcorrectly.Inthiscase,theclaimisthattheBMHalgorithmshouldhavebeatenthebinstralgorithm,butasimpletestproveditdidn’t.HadInotdonethis,Iwouldhavebeenusinganinferioralgorithmimplementationwithoutknowingit.Withthesemetrics,Icanstarttotunemyimplementation,orsimplyscrapitandfindanotherone.

ExtraCredit•SeeifyoucanmaketheScan_findfaster.Whyismyimplementationhereslow?•Trysomedifferentscantimesandseeifyougetdifferentnumbers.Whatimpactdoesthelengthoftimethatyourunthetesthaveonthescantimes?Whatcanyousayaboutthatresult?•Altertheunittestsothatitrunseachfunctionforashortburstinthebeginningtoclearoutanywarm-upperiod,andthenstartthetimingportion.Doesthatchangethedependenceonthelengthoftimethetestruns?Doesitchangehowmanyoperationspersecondarepossible?•Maketheunittestrandomizethestringstofindandthenmeasuretheperformanceyouget.Onewaytodothisistousethebsplitfunctionfrombstrlib.htosplittheIN_STRonspaces.Then,youcanusethebstrListstructthatyougettoaccesseachstringitreturns.ThiswillalsoteachyouhowtousebstrListoperationsforstringprocessing.•Trysomerunswiththetestsindifferentorderstoseeifyougetdifferentresults.

Exercise40.BinarySearchTrees

Thebinarytreeisthesimplesttree-baseddatastructure,andeventhoughit’sbeenreplacedbyhashmapsinmanylanguages,it’sstillusefulformanyapplications.Variantsonthebinarytreeexistforveryusefulthingslikedatabaseindexes,searchalgorithmstructures,andevengraphics.I’mcallingmybinarytreeaBSTreeforbinarysearchtree,andthebestwaytodescribeitisthatit’sanotherwaytodoaHashmapstylekey/valuestore.Thedifferenceisthatinsteadofhashingthekeytofindalocation,theBSTreecomparesthekeytonodesinatree,andthenwalksthroughthetreetofindthebestplacetostoreit,basedonhowitcomparestoothernodes.BeforeIreallyexplainhowthisworks,letmeshowyouthebstree.hheaderfilesothatyoucanseethedatastructures,andthenIcanusethattoexplainhowit’sbuilt.

bstree.h


#ifndef_lcthw_BSTree_h

#define_lcthw_BSTree_h

typedefint(*BSTree_compare)(void*a,void*b);

typedefstructBSTreeNode{void*key;void*data;

structBSTreeNode*left;structBSTreeNode*right;structBSTreeNode*parent;}BSTreeNode;

typedefstructBSTree{intcount;BSTree_comparecompare;BSTreeNode*root;}BSTree;

typedefint(*BSTree_traverse_cb)(BSTreeNode*node);

BSTree*BSTree_create(BSTree_comparecompare);voidBSTree_destroy(BSTree*map);

intBSTree_set(BSTree*map,void*key,void*data);void*BSTree_get(BSTree*map,void*key);

intBSTree_traverse(BSTree*map,BSTree_traverse_cbtraverse_cb);

void*BSTree_delete(BSTree*map,void*key);

#endif

ThisfollowsthesamepatternthatI’vebeenusingthiswholetimewhereIhaveabasecontainernamedBSTree,whichhasnodesnamedBSTreeNodethatmakeuptheactualcontents.Boredyet?Good,there’snoreasontobecleverwiththiskindofstructure.TheimportantthingishowtheBSTreeNodeisconfigured,andhowitgetsusedtodoeach

operation:set,get,anddelete.I’llcovergetfirstsinceit’stheeasiestoperation,andI’llpretendI’mdoingitmanuallyagainstthedatastructure:

•Itakethekeyyou’relookingforandIstartattheroot.FirstthingIdoiscompareyourkeywiththatnode’skey.•Ifyourkeyislessthanthenode.key,thenItraversedownthetreeusingtheleftpointer.•Ifyourkeyisgreaterthanthenode.key,thenIgodownwithright.•Irepeatsteps2and3untilIeitherfindamatchingnode.keyorgettoanodethathasnoleftandright.Inthefirstcase,Ireturnthenode.data.Inthesecond,IreturnNULL.

That’sallthereistoget,sonowontoset.It’snearlythesamething,exceptyou’relookingforwheretoputanewnode:

•IfthereisnoBSTree.root,thenIjustmakeitandwe’redone.That’sthefirstnode.•Afterthat,Icompareyourkeytonode.key,startingattheroot.•Ifyourkeyislessthanorequaltothenode.key,thenIwanttogoleft.Ifyourkeyisgreaterthanandnotequaltothenode.key,thenIwanttogoright.•Ikeeprepeatingstep3untilIreachanodewhereleftorrightdoesn’texist,butthat’sthedirectionIneedtogo.•Oncethere,Isetthatdirection(leftorright)toanewnodeforthekeyanddataIwant,andthensetthisnewnode’sparenttothepreviousnodeIcamefrom.I’llusetheparentnodewhenIdodelete.

Thisalsomakessensegivenhowgetworks.Iffindinganodeinvolvesgoingleftorrightdependingonhowthekeycompares,thensettinganodeinvolvesthesamethinguntilIcansettheleftorrightforanewnode.Takesometimetodrawoutafewtreesonpaperandgothroughsettingandgettingnodessoyouunderstandhowthisworks.Afterthat,you’rereadytolookattheimplementation,andIcanexplaindelete.Deletingintreesisamajorpain,andsoit’sbestexplainedbydoingaline-by-linecodebreakdown.

bstree.c


1#include<lcthw/dbg.h>2#include<lcthw/bstree.h>3#include<stdlib.h>4#include<lcthw/bstrlib.h>56staticintdefault_compare(void*a,void*b)7{8returnbstrcmp((bstring)a,(bstring)b);9}1011BSTree*BSTree_create(BSTree_comparecompare)12{13BSTree*map=calloc(1,sizeof(BSTree));14check_mem(map);1516map->compare=compare==NULL?default_compare:compare;17

18returnmap;1920error:21if(map){22BSTree_destroy(map);23}24returnNULL;25}2627staticintBSTree_destroy_cb(BSTreeNode*node)28{29free(node);30return0;31}3233voidBSTree_destroy(BSTree*map)34{35if(map){36BSTree_traverse(map,BSTree_destroy_cb);37free(map);38}39}4041staticinlineBSTreeNode*BSTreeNode_create(BSTreeNode*parent,42void*key,void*data)43{44BSTreeNode*node=calloc(1,sizeof(BSTreeNode));45check_mem(node);4647node->key=key;48node->data=data;49node->parent=parent;50returnnode;5152error:53returnNULL;54}5556staticinlinevoidBSTree_setnode(BSTree*map,BSTreeNode*node,57void*key,void*data)58{59intcmp=map->compare(node->key,key);6061if(cmp<=0){62if(node->left){63BSTree_setnode(map,node->left,key,data);64}else{65node->left=BSTreeNode_create(node,key,data);66}67}else{68if(node->right){69BSTree_setnode(map,node->right,key,data);70}else{71node->right=BSTreeNode_create(node,key,data);72}73}74}7576intBSTree_set(BSTree*map,void*key,void*data)77{78if(map->root==NULL){79//firstsojustmakeitandgetout80map->root=BSTreeNode_create(NULL,key,data);

81check_mem(map->root);82}else{83BSTree_setnode(map,map->root,key,data);84}8586return0;87error:88return-1;89}9091staticinlineBSTreeNode*BSTree_getnode(BSTree*map,92BSTreeNode*node,void*key)93{94intcmp=map->compare(node->key,key);9596if(cmp==0){97returnnode;98}elseif(cmp<0){99if(node->left){100returnBSTree_getnode(map,node->left,key);101}else{102returnNULL;103}104}else{105if(node->right){106returnBSTree_getnode(map,node->right,key);107}else{108returnNULL;109}110}111}112113void*BSTree_get(BSTree*map,void*key)114{115if(map->root==NULL){116returnNULL;117}else{118BSTreeNode*node=BSTree_getnode(map,map->root,key);119returnnode==NULL?NULL:node->data;120}121}122123staticinlineintBSTree_traverse_nodes(BSTreeNode*node,124BSTree_traverse_cbtraverse_cb)125{126intrc=0;127128if(node->left){129rc=BSTree_traverse_nodes(node->left,traverse_cb);130if(rc!=0)131returnrc;132}133134if(node->right){135rc=BSTree_traverse_nodes(node->right,traverse_cb);136if(rc!=0)137returnrc;138}139140returntraverse_cb(node);141}142143intBSTree_traverse(BSTree*map,BSTree_traverse_cbtraverse_cb)

144{145if(map->root){146returnBSTree_traverse_nodes(map->root,traverse_cb);147}148149return0;150}151152staticinlineBSTreeNode*BSTree_find_min(BSTreeNode*node)153{154while(node->left){155node=node->left;156}157158returnnode;159}160161staticinlinevoidBSTree_replace_node_in_parent(BSTree*map,162BSTreeNode*node,163BSTreeNode*new_value)164{165if(node->parent){166if(node==node->parent->left){167node->parent->left=new_value;168}else{169node->parent->right=new_value;170}171}else{172//thisistherootsogottachangeit173map->root=new_value;174}175176if(new_value){177new_value->parent=node->parent;178}179}180181staticinlinevoidBSTree_swap(BSTreeNode*a,BSTreeNode*b)182{183void*temp=NULL;184temp=b->key;185b->key=a->key;186a->key=temp;187temp=b->data;188b->data=a->data;189a->data=temp;190}191192staticinlineBSTreeNode*BSTree_node_delete(BSTree*map,193BSTreeNode*node,194void*key)195{196intcmp=map->compare(node->key,key);197198if(cmp<0){199if(node->left){200returnBSTree_node_delete(map,node->left,key);201}else{202//notfound203returnNULL;204}205}elseif(cmp>0){206if(node->right){

207returnBSTree_node_delete(map,node->right,key);208}else{209//notfound210returnNULL;211}212}else{213if(node->left&&node->right){214//swapthisnodeforthesmallestnodethatisbiggerthanus215BSTreeNode*successor=BSTree_find_min(node->right);216BSTree_swap(successor,node);217218//thisleavestheoldsuccessorwithpossiblyarightchild219//soreplaceitwiththatrightchild220BSTree_replace_node_in_parent(map,successor,221successor->right);222223//finallyit'sswapped,soreturnsuccessorinsteadofnode224returnsuccessor;225}elseif(node->left){226BSTree_replace_node_in_parent(map,node,node->left);227}elseif(node->right){228BSTree_replace_node_in_parent(map,node,node->right);229}else{230BSTree_replace_node_in_parent(map,node,NULL);231}232233returnnode;234}235}236237void*BSTree_delete(BSTree*map,void*key)238{239void*data=NULL;240241if(map->root){242BSTreeNode*node=BSTree_node_delete(map,map->root,key);243244if(node){245data=node->data;246free(node);247}248}249250returndata;251}

BeforegettingintohowBSTree_deleteworks,Iwanttoexplainapatternfordoingrecursivefunctioncallsinasaneway.You’llfindthatmanytree-baseddatastructuresareeasytowriteifyouuserecursion,butformulateasinglerecursivefunction.Partoftheproblemisthatyouneedtosetupsomeinitialdataforthefirstoperation,thenrecurseintothedatastructure,whichishardtodowithonefunction.Thesolutionistousetwofunctions:Onefunctionsetsupthedatastructureandinitialrecursionconditionssothatasecondfunctioncandotherealwork.TakealookatBSTree_getfirsttoseewhatImean.

•Ihaveaninitialcondition:Ifmap->rootisNULL,thenreturnNULLanddon’trecurse.•Ithensetupacalltotherealrecursion,whichisinBSTree_getnode.Icreatetheinitialconditionoftherootnodetostartwiththekeyandthenthemap.•IntheBSTree_getnode,Ithendotheactualrecursivelogic.Icomparethekeyswithmap-

>compare(node->key,key)andgoleft,right,orequaltodependingontheresults.•Sincethisfunctionisself-similaranddoesn’thavetohandleanyinitialconditions(becauseBSTree_getdid),thenIcanstructureitverysimply.Whenit’sdone,itreturnstothecaller,andthatreturnthencomesbacktoBSTree_getfortheresult.•Attheend,theBSTree_gethandlesgettingthenode.dataelementbutonlyiftheresultisn’tNULL.

ThiswayofstructuringarecursivealgorithmmatchesthewayIstructuremyrecursivedatastructures.Ihaveaninitialbasefunctionthathandlesinitialconditionsandsomeedgecases,andthenitcallsacleanrecursivefunctionthatdoesthework.ComparethatwithhowIhaveabasestructureinBStreecombinedwithrecursiveBSTreeNodestructures,whichallreferenceeachotherinatree.Usingthispatternmakesiteasytodealwithrecursionandkeepitstraight.Next,golookatBSTree_setandBSTree_setnodetoseetheexactsamepattern.IuseBSTree_settoconfiguretheinitialconditionsandedgecases.Acommonedgecaseisthatthere’snorootnode,soIhavetomakeonetogetthingsstarted.Thispatternwillworkwithnearlyanyrecursivealgorithmyouhavetofigureout.ThewayIdoitisbyfollowingthispattern:

•Figureouttheinitialvariables,howtheychange,andwhatthestoppingconditionsareforeachrecursivestep.•Writearecursivefunctionthatcallsitself,andhasargumentsforeachstoppingconditionandinitialvariable.•Writeasetupfunctiontosetinitialstartingconditionsforthealgorithmandhandleedgecases,thenhaveitcalltherecursivefunction.•Finally,thesetupfunctionreturnsthefinalresult,andpossiblyaltersitiftherecursivefunctioncan’thandlefinaledgecases.

ThisfinallyleadsmetoBSTree_deleteandBSTree_node_delete.First,youcanjustlookatBSTree_deleteandseethatit’sthesetupfunction.Whatit’sdoingisgrabbingtheresultingnodedataandfreeingthenodethat’sfound.ThingsgetmorecomplexinBSTree_node_delete,becausetodeleteanodeatanypointinthetree,Ihavetorotatethatnode’schildrenuptotheparent.Here’sabreakdownofthisfunctionandthefunctionsituses:

bstree.c:190IrunthecomparefunctiontofigureoutwhichdirectionI’mgoing.bstree.c:192-198Thisistheusualless-thanbranchtousewhenIwanttogoleft.I’mhandlingthecasethatleftdoesn’texisthere,andreturningNULLtosay“notfound.”Thiscoversdeletingsomethingthatisn’tintheBSTree.

bstree.c:199-205Thisisthesamething,butfortherightbranchofthetree.Justkeeprecursingdownintothetreejustlikeintheotherfunctions,andreturnNULLifitdoesn’texist.

bstree.c:206ThisiswhereIhavefoundthenode,sincethekeyisequal(comparereturn0).bstree.c:207Thisnodehasbothaleftandrightbranch,soit’sdeeplyembeddedinthetree.bstree.c:209Toremovethisnode,Ifirstneedtofindthesmallestnodethat’sgreaterthanthisnode,whichmeansIcallBSTree_find_minontherightchild.

bstree.c:210OnceIhavethisnode,I’llswapitskeyanddatawiththecurrentnode’svalues.Thiswilleffectivelytakethisnodethatwasdownatthebottomofthetreeandputitscontents

here,sothatIdon’thavetotryandshufflethenodeoutbyitspointers.bstree.c:214Thesuccessorisnowthisdeadbranchthathasthecurrentnode’svalues.Itcouldjustberemoved,butthere’sachancethatithasarightnodevalue.ThismeansIneedtodoasinglerotatesothatthesuccessor ’srightnodegetsmoveduptocompletelydetachit.

bstree.c:217Atthispoint,thesuccessorisremovedfromthetree,itsvaluesarereplacedthecurrentnode’svalues,andanychildrenithadaremovedupintotheparent.Icanreturnthesuccessorasifitwerethenode.

bstree.c:218Atthisbranch,Iknowthatthenodehasaleftbutnoright,soIwanttoreplacethisnodewithitsleftchild.

bstree.c:219IagainuseBSTree_replace_node_in_parenttodothereplace,rotatingtheleftchildup.

bstree.c:220Thisbranchoftheif-statementmeansIhavearightchildbutnoleftchild,soIwanttorotatetherightchildup.

bstree.c:221Again,Iusethefunctiontodotherotate,butthistime,rotatetherightnode.bstree.c:222Finally,theonlythingthat’sleftistheconditionwhereI’vefoundthenode,andithasnochildren(noleftorright).Inthiscase,IsimplyreplacethisnodewithNULLbyusingthesamefunctionIdidwithalloftheothers.

bstree.c:210Afterallthat,Ihavethecurrentnoderotatedoutofthetreeandreplacedwithsomechildelementthatwillfitinthetree.Ijustreturnthistothecallersoitcanbefreedandmanaged.

Thisoperationisverycomplex,andtobehonest,Ijustdon’tbotherdoingdeletesinsometreedatastructures,andItreatthemlikeconstantdatainmysoftware.IfIneedtodoheavyinsertinganddeleting,IuseaHashmapinstead.Finally,youcanlookattheunittesttoseehowI’mtestingit:

bstree_tests.c


1#include"minunit.h"2#include<lcthw/bstree.h>3#include<assert.h>4#include<lcthw/bstrlib.h>5#include<stdlib.h>6#include<time.h>78BSTree*map=NULL;9staticinttraverse_called=0;10structtagbstringtest1=bsStatic("testdata1");11structtagbstringtest2=bsStatic("testdata2");12structtagbstringtest3=bsStatic("xestdata3");13structtagbstringexpect1=bsStatic("THEVALUE1");14structtagbstringexpect2=bsStatic("THEVALUE2");15structtagbstringexpect3=bsStatic("THEVALUE3");1617staticinttraverse_good_cb(BSTreeNode*node)18{19debug("KEY:%s",bdata((bstring)node->key));20traverse_called++;21return0;

22}2324staticinttraverse_fail_cb(BSTreeNode*node)25{26debug("KEY:%s",bdata((bstring)node->key));27traverse_called++;2829if(traverse_called==2){30return1;31}else{32return0;33}34}3536char*test_create()37{38map=BSTree_create(NULL);39mu_assert(map!=NULL,"Failedtocreatemap.");4041returnNULL;42}4344char*test_destroy()45{46BSTree_destroy(map);4748returnNULL;49}5051char*test_get_set()52{53intrc=BSTree_set(map,&test1,&expect1);54mu_assert(rc==0,"Failedtoset&test1");55bstringresult=BSTree_get(map,&test1);56mu_assert(result==&expect1,"Wrongvaluefortest1.");5758rc=BSTree_set(map,&test2,&expect2);59mu_assert(rc==0,"Failedtosettest2");60result=BSTree_get(map,&test2);61mu_assert(result==&expect2,"Wrongvaluefortest2.");6263rc=BSTree_set(map,&test3,&expect3);64mu_assert(rc==0,"Failedtosettest3");65result=BSTree_get(map,&test3);66mu_assert(result==&expect3,"Wrongvaluefortest3.");6768returnNULL;69}7071char*test_traverse()72{73intrc=BSTree_traverse(map,traverse_good_cb);74mu_assert(rc==0,"Failedtotraverse.");75mu_assert(traverse_called==3,"Wrongcounttraverse.");7677traverse_called=0;78rc=BSTree_traverse(map,traverse_fail_cb);79mu_assert(rc==1,"Failedtotraverse.");80mu_assert(traverse_called==2,"Wrongcounttraverseforfail.");8182returnNULL;83}8485char*test_delete()

86{87bstringdeleted=(bstring)BSTree_delete(map,&test1);88mu_assert(deleted!=NULL,"GotNULLondelete.");89mu_assert(deleted==&expect1,"Shouldgettest1");90bstringresult=BSTree_get(map,&test1);91mu_assert(result==NULL,"Shoulddelete.");9293deleted=(bstring)BSTree_delete(map,&test1);94mu_assert(deleted==NULL,"ShouldgetNULLondelete");9596deleted=(bstring)BSTree_delete(map,&test2);97mu_assert(deleted!=NULL,"GotNULLondelete.");98mu_assert(deleted==&expect2,"Shouldgettest2");99result=BSTree_get(map,&test2);100mu_assert(result==NULL,"Shoulddelete.");101102deleted=(bstring)BSTree_delete(map,&test3);103mu_assert(deleted!=NULL,"GotNULLondelete.");104mu_assert(deleted==&expect3,"Shouldgettest3");105result=BSTree_get(map,&test3);106mu_assert(result==NULL,"Shoulddelete.");107108//testdeletingnon-existentstuff109deleted=(bstring)BSTree_delete(map,&test3);110mu_assert(deleted==NULL,"ShouldgetNULL");111112returnNULL;113}114115char*test_fuzzing()116{117BSTree*store=BSTree_create(NULL);118inti=0;119intj=0;120bstringnumbers[100]={NULL};121bstringdata[100]={NULL};122srand((unsignedint)time(NULL));123124for(i=0;i<100;i++){125intnum=rand();126numbers[i]=bformat("%d",num);127data[i]=bformat("data%d",num);128BSTree_set(store,numbers[i],data[i]);129}130131for(i=0;i<100;i++){132bstringvalue=BSTree_delete(store,numbers[i]);133mu_assert(value==data[i],134"Failedtodeletetherightnumber.");135136mu_assert(BSTree_delete(store,numbers[i])==NULL,137"Shouldgetnothing.");138139for(j=i+1;j<99-i;j++){140bstringvalue=BSTree_get(store,numbers[j]);141mu_assert(value==data[j],142"Failedtogettherightnumber.");143}144145bdestroy(value);146bdestroy(numbers[i]);147}148

149BSTree_destroy(store);150151returnNULL;152}153154char*all_tests()155{156mu_suite_start();157158mu_run_test(test_create);159mu_run_test(test_get_set);160mu_run_test(test_traverse);161mu_run_test(test_delete);162mu_run_test(test_destroy);163mu_run_test(test_fuzzing);164165returnNULL;166}167168RUN_TESTS(all_tests);

I’llpointyoutothetest_fuzzingfunction,whichisaninterestingtechniquefortestingcomplexdatastructures.ItisdifficulttocreateasetofkeysthatcoverallofthebranchesinBSTree_node_delete,andchancesare,Iwouldmisssomeedgecase.Abetterwayistocreateafuzzfunctionthatdoesalloftheoperations,butdoestheminahorribleandrandomway.Inthiscase,I’minsertingasetofrandomstringkeys,andthenI’mdeletingthemandtryingtogettherestaftereachdelete.Doingthispreventsyoufromtestingonlywhatyouknowtowork,andthenmissthingsyoudon’tknow.Bythrowingrandomjunkatyourdatastructures,you’llhitthingsyoudidn’texpectandbeabletoworkoutanybugsyouhave.

HowtoImproveItDonotdoanyoftheseyet.InthenextexerciseI’llbeusingthisunittesttoteachyousomemoreperformance-tuningtricks,andyou’llcomebackanddotheseafteryoucompleteExercise41.

•Asusual,youshouldgothroughallofthedefensiveprogrammingchecksandaddassert``sforconditionsthatshouldn’thappen.Forexample,youshouldn’tbegetting``NULLvaluesfortherecursionfunctions,soassertthat.•Thetraversefunctionwalksthroughthetreeinorderbytraversingleft,thenright,andthenthecurrentnode.Youcancreatetraversefunctionsforthereverseorder,aswell.•Itdoesafullstringcompareoneverynode,butIcouldusetheHashmaphashingfunctionstospeedthisup.Icouldhashthekeys,andthenkeepthehashintheBSTreeNode.Then,ineachofthesetupfunctions,Icanhashthekeyaheadoftimeandpassitdowntotherecursivefunction.Usingthishash,Icanthencompareeachnodemuchquickerinawaythat’ssimilartowhatIdoinHashmap.

ExtraCredit•There’sanalternativewaytodothisdatastructurewithoutusingrecursion.TheWikipediapageshowsalternativesthatdon’tuserecursionbutdothesamething.Whywouldthisbebetterorworse?•Readuponallofthedifferentbutsimilartreesyoucanfind.ThereareAVLtrees(namedafterGeorgyAdelson-VelskyandE.M.Landis),red-blacktrees,andsomenon-treestructureslike

skiplists.

Exercise41.Projectdevpkg

Youarenowreadytotackleanewprojectcalleddevpkg.Inthisprojectyou’regoingtorecreateapieceofsoftwarethatIwrotespecificallyforthisbookcalleddevpkg.You’llthenextenditinafewkeywaysandimprovethecode,mostimportantlybywritingsomeunittestsforit.Thisexercisehasacompanionvideotoit,andalsoaprojectonGitHub(https://github.com)thatyoucanreferenceifyougetstuck.Youshouldattempttodothisexerciseusingthedescriptionbelow,sincethat’showyou’llneedtolearntocodefrombooksinthefuture.Mostcomputersciencetextbooksdon’tincludevideosfortheirexercises,sothisprojectismoreabouttryingtofigureitoutfromthisdescription.Ifyougetstuck,andyoucan’tfigureitout,thengowatchthevideoandlookattheGitHubprojecttoseehowyourcodediffersfrommine.

WhatIsdevpkg?DevpkgisasimpleCprogramthatinstallsothersoftware.Imadeitspecificallyforthisbookasawaytoteachyouhowarealsoftwareprojectisstructured,andalsohowtoreuseotherpeople’slibraries.ItusesaportabilitylibrarycalledtheApachePortableRuntime(APR),whichhasmanyhandyCfunctionsthatworkontonsofplatforms,includingWindows.Otherthanthat,itjustgrabscodefromtheInternet(orlocalfiles)anddoestheusual./configure,make,andmakeinstallthateveryprogramdoes.Yourgoalinthisexerciseistobuilddevpkgfromthesource,finisheachchallengeIgive,andusethesourcetounderstandwhatdevpkgdoesandwhy.

WhatWeWanttoMakeWewantatoolthathasthesecommands:

devpkg-SSetsupanewinstallationonacomputer.devpkg-IInstallsapieceofsoftwarefromaURL.devpkg-LListsallofthesoftwarethat’sbeeninstalled.devpkg-FFetchessomesourcecodeformanualbuilding.devpkg-BBuildsthesourcecodeandinstallsit,evenifit’salreadyinstalled.

WewantdevpkgtobeabletotakealmostanyURL,figureoutwhatkindofprojectitis,downloadit,installit,andregisterthatitdownloadedthatsoftware.We’dalsolikeittoprocessasimpledependencylistsothatitcaninstallallofthesoftwarethataprojectmightneed,aswell.

TheDesignToaccomplishthisgoal,devpkgwillhaveaverysimpledesign:

UseExternalCommandsYou’lldomostoftheworkthroughexternalcommandslikecurl,git,andtar.Thisreducestheamountofcodedevpkgneedstogetthingsdone.

SimpleFileDatabaseYoucouldeasilymakeitmorecomplex,butyou’llstartbymakingjustmakeasinglesimplefiledatabaseat/usr/local/.devpkg/dbtokeeptrackofwhat’sinstalled.

https://github.com

/usr/localAlwaysAgain,youcouldmakethismoreadvanced,butfornowjustassumeit’salways/usr/local,whichisastandardinstallpathformostsoftwareonUNIX.

configure,make,makeinstallIt’sassumedthatmostsoftwarecanbeinstalledwithjustaconfigure,make,andmakeinstall—andmaybeconfigureisoptional.Ifthesoftwareataminimumcan’tdothat,therearesomeoptionstomodifythecommands,butotherwise,devpkgwon’tbother.

TheUserCanBeRootWe’llassumethattheusercanbecomerootusingsudo,butdoesn’twanttobecomerootuntiltheend.

Thiswillkeepourprogramsmallatfirstandworkwellenoughforustogetitgoing,atwhichpointyou’llbeabletomodifyitfurtherforthisexercise.

TheApachePortableRuntimeOnemorethingyou’lldoisleveragetheAPRLibrariestogetagoodsetofportableroutinesfordoingthiskindofwork.APRisn’tnecessary,andyoucouldprobablywritethisprogramwithoutit,butit’dtakemorecodethannecessary.I’malsoforcingyoutouseAPRnowsoyougetusedtolinkingandusingotherlibraries.Finally,APRalsoworksonWindows,soyourskillswithitaretransferabletomanyotherplatforms.Youshouldgogetboththeapr-1.5.2andtheapr-util-1.5.4libraries,aswellasbrowsethroughthedocumentationavailableatthemainAPRsiteathttp://apr.apache.org.Here’sashellscriptthatwillinstallallthestuffyouneed.Youshouldwritethisintoafilebyhand,andthenrunituntilitcaninstallAPRwithoutanyerrors.

Exercise41.1Session


set-e

#gosomewheresafe

cd/tmp

#getthesourcetobaseAPR1.5.2

curl-L-Ohttp://archive.apache.org/dist/apr/apr-1.5.2.tar.gz

#extractitandgointothesource

tar-xzvfapr-1.5.2.tar.gzcdapr-1.5.2

#configure,make,makeinstall

./configuremakesudomakeinstall

#resetandcleanup

cd/tmprm-rfapr-1.5.2apr-1.5.2.tar.gz

#dothesamewithapr-util

curl-L-Ohttp://archive.apache.org/dist/apr/apr-util-1.5.4.tar.gz

#extract

tar-xzvfapr-util-1.5.4.tar.gz

http://apr.apache.org

cdapr-util-1.5.4

#configure,make,makeinstall

./configure--with-apr=/usr/local/apr#youneedthatextraparametertoconfigurebecause

#apr-utilcan'treallyfinditbecause...whoknows.

makesudomakeinstall

#cleanup

cd/tmprm-rfapr-util-1.5.4*apr-1.5.2*

I’mhavingyouwritethisscriptoutbecauseit’sbasicallywhatwewantdevpkgtodo,butwithextraoptionsandchecks.Infact,youcouldjustdoitallinshellwithlesscode,butthenthatwouldn’tbeaverygoodprogramforaCbookwouldit?Simplyrunthisscriptandfixituntilitworks,thenyou’llhavethelibrariesyouneedtocompletetherestofthisproject.

ProjectLayoutYouneedtosetupsomesimpleprojectfilestogetstarted.Here’showIusuallycraftanewproject:

Exercise41.2Session

mkdirdevpkgcddevpkgtouchREADMEMakefile

OtherDependenciesYoushouldhavealreadyinstalledapr-1.5.2andapr-util-1.5.4,sonowyouneedafewmorefilestouseasbasicdependencies:

•dbg.hfromExercise20.•bstrlib.handbstrlib.cfromhttp://bstring.sourceforge.net/.Downloadthe.zipfile,extractit,andcopyjustthosetwofiles.•Typemakebstrlib.o,andifitdoesn’twork,readtheinstructionsforfixingbstringbelow.

http://bstring.sourceforge.net/

Warning!Insomeplatforms,thebstring.cfilewillhaveanerrorlikethis:


bstrlib.c:2762:error:expecteddeclaration\specifiersor'...'beforenumericconstant

Thisisfromabaddefinethattheauthorsadded,whichdoesn’talwayswork.Youjustneedtochangeline2759thatreads#ifdef__GNUC__toread:


#ifdefined(__GNUC__)&&!defined(__APPLE__)

andthenitshouldworkonOSX.

Whenthat’salldone,youshouldhaveaMakefile,README,dbg.h,bstrlib.h,andbstrlib.creadytogo.

TheMakefileAgoodplacetostartistheMakefilesincethislaysouthowthingsarebuiltandwhatsourcefilesyou’llbecreating.

Makefile


PREFIX?=/usr/localCFLAGS=-g-Wall-I${PREFIX}/apr/include/apr-1CFLAGS+=-I${PREFIX}/apr/include/apr-util-1LDFLAGS=-L${PREFIX}/apr/lib-lapr-1-pthread-laprutil-1

all:devpkg

devpkg:bstrlib.odb.oshell.ocommands.o

install:allinstall-d$(DESTDIR)/$(PREFIX)/bin/installdevpkg$(DESTDIR)/$(PREFIX)/bin/

clean:rm-f*.orm-fdevpkgrm-rf*.dSYM

There’snothinginthisthatyouhaven’tseenbefore,exceptmaybethestrange?=syntax,whichsays“setPREFIXequaltothisunlessPREFIXisalreadyset.”

Warning!Ifyou’reonmorerecentversionsofUbuntu,andyougeterrorsaboutapr_off_toroff64_t,thenadd-D_LARGEFILE64_SOURCE=1toCFLAGS.Anotherthingisthatyouneedtoadd/usr/local/apr/libtoafilein/etc/ld.conf.so.d/andthenrunldconfigsothatitcorrectlypicksupthelibraries.

TheSourceFilesFromtheMakefile,weseethattherearefivedependenciesfordevpkg:

bstrlib.oThiscomesfrombstrlib.candtheheaderfilebstlib.h,whichyoualreadyhave.db.oThiscomesfromdb.candheaderfiledb.h,anditwillcontaincodeforourlittledatabaseroutines.

shell.oThisisfromshell.candheadershell.h,aswellasacoupleoffunctionsthatmakerunningothercommandslikecurleasier.

commands.oThisisfromcommand.candheadercommand.h,andcontainsallofthecommandsthatdevpkgneedstobeuseful.

devpkgIt’snotexplicitlymentioned,butit’sthetarget(ontheleft)inthispartoftheMakefile.Itcomesfromdevpkg.c,whichcontainsthemainfunctionforthewholeprogram.

Yourjobistonowcreateeachofthesefiles,typeintheircode,andgetthemcorrect.

Warning!Youmayreadthisdescriptionandthink,“Man!HowisitthatZedissosmartthathejustsatdownandtypedthesefilesoutlikethis!?Icouldneverdothat.”Ididn’tmagicallycraftdevpkginthisformwithmyawesomecodingpowers.Instead,whatIdidisthis:•IwroteaquicklittleREADMEtogetanideaofhowIwantedittowork.•Icreatedasimplebashscript(liketheoneyoudidearlier)tofigureoutallofthepiecesthatwereneeded.•Imadeone.cfileandhackedonitforafewdaysworkingthroughtheideaandfiguringitout.•Igotitmostlyworkinganddebugged,thenIstartedbreakinguptheonebigfileintothesefourfiles.•Aftergettingthesefileslaiddown,Irenamedandrefinedthefunctionsanddatastructuressothatthey’dbemorelogicalandpretty.•Finally,afterIhaditworkingtheexactsamebutwiththenewstructure,Iaddedafewfeatureslikethe-Fand-Boptions.

You’rereadingthisintheorderIwanttoteachittoyou,butdon’tthinkthisishowIalwaysbuildsoftware.SometimesIalreadyknowthesubjectandIusemoreplanning.SometimesIjusthackupanideaandseehowwellit’dwork.SometimesIwriteone,thenthrowitawayandplanoutabetterone.Italldependsonwhatmyexperiencetellsmeisbestorwheremyinspirationtakesme.Ifyourunintoasupposedexpertwhotriestotellyouthatthere’sonlyonewaytosolvea

programmingproblem,they’relyingtoyou.Eithertheyactuallyusemultipletactics,orthey’renotverygood.

TheDBFunctionsTheremustbeawaytorecordURLsthathavebeeninstalled,listtheseURLs,andcheckwhethersomethinghasalreadybeeninstalledsowecanskipit.I’lluseasimpleflatfiledatabaseandthebstrlib.hlibrarytodoit.First,createthedb.hheaderfilesoyouknowwhatyou’llbeimplementing.

db.h


#ifndef_db_h

#define_db_h

#defineDB_FILE"/usr/local/.devpkg/db"

#defineDB_DIR"/usr/local/.devpkg"

intDB_init();intDB_list();intDB_update(constchar*url);intDB_find(constchar*url);

#endif

Then,implementthosefunctionsindb.c,andasyoubuildthis,usemaketogetittocompilecleanly.

db.c


1#include<unistd.h>2#include<apr_errno.h>3#include<apr_file_io.h>45#include"db.h"6#include"bstrlib.h"7#include"dbg.h"89staticFILE*DB_open(constchar*path,constchar*mode)10{11returnfopen(path,mode);12}1314staticvoidDB_close(FILE*db)15{16fclose(db);17}1819staticbstringDB_load()20{21FILE*db=NULL;22bstringdata=NULL;2324db=DB_open(DB_FILE,"r");25check(db,"Failedtoopendatabase:%s",DB_FILE);2627data=bread((bNread)fread,db);28check(data,"Failedtoreadfromdbfile:%s",DB_FILE);2930DB_close(db);31returndata;3233error:34if(db)35DB_close(db);36if(data)37bdestroy(data);38returnNULL;39}4041intDB_update(constchar*url)42{43if(DB_find(url)){44log_info("Alreadyrecordedasinstalled:%s",url);45}4647FILE*db=DB_open(DB_FILE,"a+");48check(db,"FailedtoopenDBfile:%s",DB_FILE);4950bstringline=bfromcstr(url);51bconchar(line,'\n');52intrc=fwrite(line->data,blength(line),1,db);53check(rc==1,"Failedtoappendtothedb.");54

55return0;56error:57if(db)58DB_close(db);59return-1;60}6162intDB_find(constchar*url)63{64bstringdata=NULL;65bstringline=bfromcstr(url);66intres=-1;6768data=DB_load();69check(data,"Failedtoload:%s",DB_FILE);7071if(binstr(data,0,line)==BSTR_ERR){72res=0;73}else{74res=1;75}7677error://fallthrough78if(data)79bdestroy(data);80if(line)81bdestroy(line);8283returnres;84}8586intDB_init()87{88apr_pool_t*p=NULL;89apr_pool_initialize();90apr_pool_create(&p,NULL);9192if(access(DB_DIR,W_OK|X_OK)==-1){93apr_status_trc=apr_dir_make_recursive(DB_DIR,94APR_UREAD|APR_UWRITE95|APR_UEXECUTE|96APR_GREAD|APR_GWRITE97|APR_GEXECUTE,p);98check(rc==APR_SUCCESS,"Failedtomakedatabasedir:%s",99DB_DIR);100}101102if(access(DB_FILE,W_OK)==-1){103FILE*db=DB_open(DB_FILE,"w");104check(db,"Cannotopendatabase:%s",DB_FILE);105DB_close(db);106}107108apr_pool_destroy(p);109return0;110111error:112apr_pool_destroy(p);113return-1;114}115116intDB_list()117{

118bstringdata=DB_load();119check(data,"Failedtoreadload:%s",DB_FILE);120121printf("%s",bdata(data));122bdestroy(data);123return0;124125error:126return-1;127}

Challenge1:CodeReviewBeforecontinuing,readeverylineofthesefilescarefullyandconfirmthatyouhavethementeredinexactlyastheyappearhere.Readthembackwardlinebylinetopracticethat.Also,traceeachfunctioncallandmakesureyou’reusingchecktovalidatethereturncodes.Finally,lookupeveryfunctionthatyoudon’trecognize—eitherintheAPRWebsitedocumentationorinthebstrlib.handbstrlib.csource.

TheShellFunctionsAkeydesigndecisionfordevpkgistohaveexternaltoolslikecurl,tar,andgitdomostofthework.Wecouldfindlibrariestodoallofthisinternally,butit’spointlessifwejustneedthebasefeaturesoftheseprograms.ThereisnoshameinrunninganothercommandinUNIX.Todothis,I’mgoingtousetheapr_thread_proc.hfunctionstorunprograms,butIalsowanttomakeasimplekindoftemplatesystem.I’lluseastructShellthatholdsalloftheinformationneededtorunaprogram,buthasholesintheargumentslistthatIcanreplacewithvalues.Lookattheshell.hfiletoseethestructureandthecommandsthatI’lluse.YoucanseethatI’musingexterntoindicatehowother.cfilescanaccessvariablesthatI’mdefininginshell.c.

shell.h


#ifndef_shell_h

#define_shell_h

#defineMAX_COMMAND_ARGS100

#include<apr_thread_proc.h>

typedefstructShell{constchar*dir;constchar*exe;

apr_procattr_t*attr;apr_proc_tproc;apr_exit_why_eexit_why;intexit_code;

constchar*args[MAX_COMMAND_ARGS];}Shell;

intShell_run(apr_pool_t*p,Shell*cmd);intShell_exec(Shellcmd,...);

externShellCLEANUP_SH;externShellGIT_SH;externShellTAR_SH;externShellCURL_SH;externShellCONFIGURE_SH;externShellMAKE_SH;externShellINSTALL_SH;

#endif

Makesureyou’vecreatedshell.hexactlyasitappearshere,andthatyou’vegotthesamenamesandnumberofexternShellvariables.ThoseareusedbytheShell_runandShell_execfunctionstoruncommands.Idefinethesetwofunctions,andcreatetherealvariablesinshell.c.

shell.c


1#include"shell.h"2#include"dbg.h"3#include<stdarg.h>45intShell_exec(Shelltemplate,...)6{7apr_pool_t*p=NULL;8intrc=-1;9apr_status_trv=APR_SUCCESS;10va_listargp;11constchar*key=NULL;12constchar*arg=NULL;13inti=0;1415rv=apr_pool_create(&p,NULL);16check(rv==APR_SUCCESS,"Failedtocreatepool.");1718va_start(argp,template);1920for(key=va_arg(argp,constchar*);21key!=NULL;key=va_arg(argp,constchar*)){22arg=va_arg(argp,constchar*);2324for(i=0;template.args[i]!=NULL;i++){25if(strcmp(template.args[i],key)==0){26template.args[i]=arg;27break;//foundit28}29}30}3132rc=Shell_run(p,&template);33apr_pool_destroy(p);34va_end(argp);35returnrc;3637error:38if(p){39apr_pool_destroy(p);40}41returnrc;42}43

44intShell_run(apr_pool_t*p,Shell*cmd)45{46apr_procattr_t*attr;47apr_status_trv;48apr_proc_tnewproc;4950rv=apr_procattr_create(&attr,p);51check(rv==APR_SUCCESS,"Failedtocreateprocattr.");5253rv=apr_procattr_io_set(attr,APR_NO_PIPE,APR_NO_PIPE,54APR_NO_PIPE);55check(rv==APR_SUCCESS,"FailedtosetIOofcommand.");5657rv=apr_procattr_dir_set(attr,cmd->dir);58check(rv==APR_SUCCESS,"Failedtosetrootto%s",cmd->dir);5960rv=apr_procattr_cmdtype_set(attr,APR_PROGRAM_PATH);61check(rv==APR_SUCCESS,"Failedtosetcmdtype.");6263rv=apr_proc_create(&newproc,cmd->exe,cmd->args,NULL,attr,p);64check(rv==APR_SUCCESS,"Failedtoruncommand.");6566rv=apr_proc_wait(&newproc,&cmd->exit_code,&cmd->exit_why,67APR_WAIT);68check(rv==APR_CHILD_DONE,"Failedtowait.");6970check(cmd->exit_code==0,"%sexitedbadly.",cmd->exe);71check(cmd->exit_why==APR_PROC_EXIT,"%swaskilledorcrashed",72cmd->exe);7374return0;7576error:77return-1;78}7980ShellCLEANUP_SH={81.exe="rm",82.dir="/tmp",83.args={"rm","-rf","/tmp/pkg-build","/tmp/pkg-src.tar.gz",84"/tmp/pkg-src.tar.bz2","/tmp/DEPENDS",NULL}85};8687ShellGIT_SH={88.dir="/tmp",89.exe="git",90.args={"git","clone","URL","pkg-build",NULL}91};9293ShellTAR_SH={94.dir="/tmp/pkg-build",95.exe="tar",96.args={"tar","-xzf","FILE","--strip-components","1",NULL}97};9899ShellCURL_SH={100.dir="/tmp",101.exe="curl",102.args={"curl","-L","-o","TARGET","URL",NULL}103};104105ShellCONFIGURE_SH={106.exe="./configure",107.dir="/tmp/pkg-build",

108.args={"configure","OPTS",NULL}109,110};111112ShellMAKE_SH={113.exe="make",114.dir="/tmp/pkg-build",115.args={"make","OPTS",NULL}116};117118ShellINSTALL_SH={119.exe="sudo",120.dir="/tmp/pkg-build",121.args={"sudo","make","TARGET",NULL}122};

Readtheshell.cfromthebottomtothetop(whichisacommonCsourcelayout)andyouseehowI’vecreatedtheactualShellvariablesthatyouindicatedwereexterninshell.h.Theylivehere,butareavailabletotherestoftheprogram.Thisishowyoumakeglobalvariablesthatliveinone.ofilebutareusedeverywhere.Youshouldn’tmakemanyofthese,buttheyarehandyforthingslikethis.ContinuingupthefilewegettotheShell_runfunction,whichisabasefunctionthatjustrunsacommandaccordingtowhat’sinaShellstruct.Itusesmanyofthefunctionsdefinedinapr_thread_proc.h,sogolookupeachonetoseehowthebasefunctionworks.Thisseemslikealotofworkcomparedtojustusingthesystemfunctioncall,butitalsogivesyoumorecontrolovertheotherprogram’sexecution.Forexample,inourShellstruct,wehavea.dirattributethatforcestheprogramtobeinaspecificdirectorybeforerunning.Finally,IhavetheShell_execfunction,whichisavariableargumentfunction.You’veseenthisbefore,butmakesureyougraspthestdarg.hfunctions.Inthechallengeforthissection,you’regoingtoanalyzethisfunction.

Challenge2:AnalyzeShell_execThechallengeforthesefiles(inadditiontoafullcodereviewlikeyoudidinChallenge1)istofullyanalyzeShell_execandbreakdownexactlyhowitworks.Youshouldbeabletounderstandeachline,howthetwofor-loopswork,andhowargumentsarebeingreplaced.Onceyouhaveitanalyzed,addafieldtostructShellthatgivesyouthenumberofvariableargsthatmustbereplaced.Updateallofthecommandstohavetherightcountofargs,andhaveanerrorchecktoconfirmthattheseargshavebeenreplaced,andthenerrorexit.

TheCommandFunctionsNowyougettomaketheactualcommandsthatdothework.ThesecommandswillusefunctionsfromAPR,db.h,andshell.htodotherealworkofdownloadingandbuildingthesoftwarethatyouwantittobuild.Thisisthemostcomplexsetoffiles,sodothemcarefully.Asbefore,youstartbymakingthecommands.hfile,thenimplementingitsfunctionsinthecommands.cfile.

commands.h


#ifndef_commands_h

#define_commands_h

#include<apr_pools.h>

#defineDEPENDS_PATH"/tmp/DEPENDS"

#defineTAR_GZ_SRC"/tmp/pkg-src.tar.gz"

#defineTAR_BZ2_SRC"/tmp/pkg-src.tar.bz2"

#defineBUILD_DIR"/tmp/pkg-build"

#defineGIT_PAT"*.git"

#defineDEPEND_PAT"*DEPENDS"

#defineTAR_GZ_PAT"*.tar.gz"

#defineTAR_BZ2_PAT"*.tar.bz2"

#defineCONFIG_SCRIPT"/tmp/pkg-build/configure"

enumCommandType{COMMAND_NONE,COMMAND_INSTALL,COMMAND_LIST,COMMAND_FETCH,COMMAND_INIT,COMMAND_BUILD};

intCommand_fetch(apr_pool_t*p,constchar*url,intfetch_only);

intCommand_install(apr_pool_t*p,constchar*url,constchar*configure_opts,constchar*make_opts,constchar*install_opts);

intCommand_depends(apr_pool_t*p,constchar*path);

intCommand_build(apr_pool_t*p,constchar*url,constchar*configure_opts,constchar*make_opts,constchar*install_opts);

#endif

There’snotmuchincommands.hthatyouhaven’tseenalready.Youshouldseethattherearesomedefinesforstringsthatareusedeverywhere.Thereallyinterestingcodeisincommands.c.

commands.c


1#include<apr_uri.h>2#include<apr_fnmatch.h>3#include<unistd.h>45#include"commands.h"6#include"dbg.h"7#include"bstrlib.h"8#include"db.h"9#include"shell.h"1011intCommand_depends(apr_pool_t*p,constchar*path)12{13FILE*in=NULL;14bstringline=NULL;1516in=fopen(path,"r");17check(in!=NULL,"Failedtoopendownloadeddepends:%s",path);1819for(line=bgets((bNgetc)fgetc,in,'\n');20line!=NULL;

21line=bgets((bNgetc)fgetc,in,'\n'))22{23btrimws(line);24log_info("Processingdepends:%s",bdata(line));25intrc=Command_install(p,bdata(line),NULL,NULL,NULL);26check(rc==0,"Failedtoinstall:%s",bdata(line));27bdestroy(line);28}2930fclose(in);31return0;3233error:34if(line)bdestroy(line);35if(in)fclose(in);36return-1;37}3839intCommand_fetch(apr_pool_t*p,constchar*url,intfetch_only)40{41apr_uri_tinfo={.port=0};42intrc=0;43constchar*depends_file=NULL;44apr_status_trv=apr_uri_parse(p,url,&info);4546check(rv==APR_SUCCESS,"FailedtoparseURL:%s",url);4748if(apr_fnmatch(GIT_PAT,info.path,0)==APR_SUCCESS){49rc=Shell_exec(GIT_SH,"URL",url,NULL);50check(rc==0,"gitfailed.");51}elseif(apr_fnmatch(DEPEND_PAT,info.path,0)==APR_SUCCESS){52check(!fetch_only,"NopointinfetchingaDEPENDSfile.");5354if(info.scheme){55depends_file=DEPENDS_PATH;56rc=Shell_exec(CURL_SH,"URL",url,"TARGET",depends_file,57NULL);58check(rc==0,"Curlfailed.");59}else{60depends_file=info.path;61}6263//recursivelyprocessthedevpkglist64log_info("BuildingaccordingtoDEPENDS:%s",url);65rv=Command_depends(p,depends_file);66check(rv==0,"FailedtoprocesstheDEPENDS:%s",url);6768//thisindicatesthatnothingneedstobedone69return0;7071}elseif(apr_fnmatch(TAR_GZ_PAT,info.path,0)==APR_SUCCESS){72if(info.scheme){73rc=Shell_exec(CURL_SH,74"URL",url,"TARGET",TAR_GZ_SRC,NULL);75check(rc==0,"Failedtocurlsource:%s",url);76}7778rv=apr_dir_make_recursive(BUILD_DIR,79APR_UREAD|APR_UWRITE|80APR_UEXECUTE,p);81check(rv==APR_SUCCESS,"Failedtomakedirectory%s",82BUILD_DIR);8384rc=Shell_exec(TAR_SH,"FILE",TAR_GZ_SRC,NULL);

85check(rc==0,"Failedtountar%s",TAR_GZ_SRC);86}elseif(apr_fnmatch(TAR_BZ2_PAT,info.path,0)==APR_SUCCESS){87if(info.scheme){88rc=Shell_exec(CURL_SH,"URL",url,"TARGET",TAR_BZ2_SRC,89NULL);90check(rc==0,"Curlfailed.");91}9293apr_status_trc=apr_dir_make_recursive(BUILD_DIR,94APR_UREAD|APR_UWRITE95|APR_UEXECUTE,p);9697check(rc==0,"Failedtomakedirectory%s",BUILD_DIR);98rc=Shell_exec(TAR_SH,"FILE",TAR_BZ2_SRC,NULL);99check(rc==0,"Failedtountar%s",TAR_BZ2_SRC);100}else{101sentinel("Don'tnowhowtohandle%s",url);102}103104//indicatesthataninstallneedstoactuallyrun105return1;106error:107return-1;108}109110intCommand_build(apr_pool_t*p,constchar*url,111constchar*configure_opts,constchar*make_opts,112constchar*install_opts)113{114intrc=0;115116check(access(BUILD_DIR,X_OK|R_OK|W_OK)==0,117"Builddirectorydoesn'texist:%s",BUILD_DIR);118119//actuallydoaninstall120if(access(CONFIG_SCRIPT,X_OK)==0){121log_info("Hasaconfigurescript,runningit.");122rc=Shell_exec(CONFIGURE_SH,"OPTS",configure_opts,NULL);123check(rc==0,"Failedtoconfigure.");124}125126rc=Shell_exec(MAKE_SH,"OPTS",make_opts,NULL);127check(rc==0,"Failedtobuild.");128129rc=Shell_exec(INSTALL_SH,130"TARGET",install_opts?install_opts:"install",131NULL);132check(rc==0,"Failedtoinstall.");133134rc=Shell_exec(CLEANUP_SH,NULL);135check(rc==0,"Failedtocleanupafterbuild.");136137rc=DB_update(url);138check(rc==0,"Failedtoaddthispackagetothedatabase.");139140return0;141142error:143return-1;144}145146intCommand_install(apr_pool_t*p,constchar*url,147constchar*configure_opts,constchar*make_opts,

148constchar*install_opts)149{150intrc=0;151check(Shell_exec(CLEANUP_SH,NULL)==0,152"Failedtocleanupbeforebuilding.");153154rc=DB_find(url);155check(rc!=-1,"Errorcheckingtheinstalldatabase.");156157if(rc==1){158log_info("Package%salreadyinstalled.",url);159return0;160}161162rc=Command_fetch(p,url,0);163164if(rc==1){165rc=Command_build(p,url,configure_opts,make_opts,166install_opts);167check(rc==0,"Failedtobuild:%s",url);168}elseif(rc==0){169//noinstallneeded170log_info("Dependssuccessfullyinstalled:%s",url);171}else{172//hadanerror173sentinel("Installfailed:%s",url);174}175176Shell_exec(CLEANUP_SH,NULL);177return0;178179error:180Shell_exec(CLEANUP_SH,NULL);181return-1;182}

Afteryouhavethisenteredinandcompiling,youcananalyzeit.Ifyou’vedonethechallengesthusfar,youshouldseehowtheshell.cfunctionsarebeingusedtorunshells,andhowtheargumentsarebeingreplaced.Ifnot,thengobackandmakesureyoutrulyunderstandhowShell_execactuallyworks.

Challenge3:CritiqueMyDesignAsbefore,doacompletereviewofthiscodeandmakesureit’sexactlythesame.Thengothrougheachfunctionandmakesureyouknowhowtheyworkandwhatthey’redoing.Youshouldalsotracehoweachfunctioncallstheotherfunctionsyou’vewritteninthisfileandotherfiles.Finally,confirmthatyouunderstandallofthefunctionsthatyou’recallingfromAPRhere.Onceyouhavethefilecorrectandanalyzed,gobackthroughandassumethatI’manidiot.Then,criticizethedesignIhavetoseehowyoucanimproveitifyoucan.Don’tactuallychangethecode,justcreatealittlenotes.txtfileandwritedownsomethoughtsaboutwhatyoumightchange.

ThedevpkgMainFunctionThelastandmostimportantfile,butprobablythesimplest,isdevpkg.c,whichiswherethemainfunctionlives.There’sno.hfileforthis,sinceitincludesalloftheothers.Instead,thisjustcreatestheexecutabledevpkgwhencombinedwiththeother.ofilesfromourMakefile.Enterinthecodeforthisfile,andmakesureit’scorrect.

devpkg.c


1#include<stdio.h>2#include<apr_general.h>3#include<apr_getopt.h>4#include<apr_strings.h>5#include<apr_lib.h>67#include"dbg.h"8#include"db.h"9#include"commands.h"1011intmain(intargc,constcharconst*argv[])12{13apr_pool_t*p=NULL;14apr_pool_initialize();15apr_pool_create(&p,NULL);1617apr_getopt_t*opt;18apr_status_trv;1920charch='\0';21constchar*optarg=NULL;22constchar*config_opts=NULL;23constchar*install_opts=NULL;24constchar*make_opts=NULL;25constchar*url=NULL;26enumCommandTyperequest=COMMAND_NONE;2728rv=apr_getopt_init(&opt,p,argc,argv);2930while(apr_getopt(opt,"I:Lc:m:i:d:SF:B:",&ch,&optarg)==31APR_SUCCESS){32switch(ch){33case'I':34request=COMMAND_INSTALL;35url=optarg;36break;3738case'L':39request=COMMAND_LIST;40break;4142case'c':43config_opts=optarg;44break;4546case'm':47make_opts=optarg;48break;4950case'i':51install_opts=optarg;52break;5354case'S':55request=COMMAND_INIT;56break;5758case'F':

59request=COMMAND_FETCH;60url=optarg;61break;6263case'B':64request=COMMAND_BUILD;65url=optarg;66break;67}68}6970switch(request){71caseCOMMAND_INSTALL:72check(url,"YoumustatleastgiveaURL.");73Command_install(p,url,config_opts,make_opts,install_opts);74break;7576caseCOMMAND_LIST:77DB_list();78break;7980caseCOMMAND_FETCH:81check(url!=NULL,"YoumustgiveaURL.");82Command_fetch(p,url,1);83log_info("Downloadedto%sandin/tmp/",BUILD_DIR);84break;8586caseCOMMAND_BUILD:87check(url,"YoumustatleastgiveaURL.");88Command_build(p,url,config_opts,make_opts,install_opts);89break;9091caseCOMMAND_INIT:92rv=DB_init();93check(rv==0,"Failedtomakethedatabase.");94break;9596default:97sentinel("Invalidcommandgiven.");98}99100return0;101102error:103return1;104}

Challenge4:TheREADMEandTestFilesThechallengeforthisfileistounderstandhowtheargumentsarebeingprocessed,whattheargumentsare,andthencreatetheREADMEfilewithinstructionsonhowtousethem.AsyouwritetheREADME,alsowriteasimpletest.shthatruns./devpkgtocheckthateachcommandisactuallyworkingagainstreal,livecode.Usetheset-eatthetopofyourscriptsothatitabortsonthefirsterror.Finally,runtheprogramunderyourdebuggerandmakesureit’sworkingbeforemovingontothefinalchallenge.

TheFinalChallengeYourfinalchallengeisaminiexamanditinvolvesthreethings:

•Compareyourcodetomycodethat’savailableonline.Startingat100%,subtract1%foreachlineyougotwrong.•Takethenotes.txtfilethatyoupreviouslycreatedandimplementyourimprovementstothethecodeandfunctionalityofdevpkg.•Writeanalternativeversionofdevpkgusingyourotherfavoritelanguageortheoneyouthinkcandothisthebest.Comparethetwo,thenimproveyourCversionofdevpkgbasedonwhatyou’velearned.

Tocompareyourcodewithmine,dothefollowing:Clickheretoviewcodeimage

cd..#getonedirectoryaboveyourcurrentonegitclonegit://gitorious.org/devpkg/devpkg.gitdevpkgzeddiff-rdevpkgdevpkgzed

Thiswillclonemyversionofdevpkgintoadirectorycalleddevpkgzedsoyoucanthenusethetooldifftocomparewhatyou’vedonetowhatIdid.Thefilesyou’reworkingwithinthisbookcomedirectlyfromthisproject,soifyougetdifferentlines,that’sanerror.Keepinmindthatthere’snorealpassorfailonthisexercise.It’sjustawayforyoutochallengeyourselftobeasexactandmeticulousaspossible.

Exercise42.StacksandQueues

Atthispointinthebook,youshouldknowmostofthedatastructuresthatareusedtobuildalloftheotherdatastructures.IfyouhavesomekindofList,DArray,Hashmap,andTree,thenyoucanbuildalmostanythingelseoutthere.Everythingelseyourunintoeitherusestheseorsomevariantofthese.Ifitdoesn’t,thenit’smostlikelyanexoticdatastructurethatyouprobablywon’tneed.StacksandQueuesareverysimpledatastructuresthatarereallyvariantsoftheListdatastructure.AlltheydoisuseaListwithadisciplineorconventionthatsaysyoualwaysplaceelementsononeendoftheList.ForaStack,youalwayspushandpop.ForaQueue,youalwaysshifttothefront,butpopfromtheend.IcanimplementbothdatastructuresusingnothingbuttheCPPandtwoheaderfiles.Myheaderfilesare21lineslonganddoalloftheStackandQueueoperationswithoutanyfancydefines.Toseeifyou’vebeenpayingattention,I’mgoingtoshowyoutheunittests,andthenhaveyouimplementtheheaderfilesneededtomakethemwork.Topassthisexercise,youcan’tcreateanystack.corqueue.cimplementationfiles.Useonlythestack.handqueue.hfilestomakethetestsrun.

stack_tests.c


1#include"minunit.h"2#include<lcthw/stack.h>3#include<assert.h>45staticStack*stack=NULL;6char*tests[]={"test1data","test2data","test3data"};78#defineNUM_TESTS3910char*test_create()11{12stack=Stack_create();13mu_assert(stack!=NULL,"Failedtocreatestack.");1415returnNULL;16}1718char*test_destroy()19{20mu_assert(stack!=NULL,"Failedtomakestack#2");21Stack_destroy(stack);2223returnNULL;24}2526char*test_push_pop()27{28inti=0;29for(i=0;i<NUM_TESTS;i++){30Stack_push(stack,tests[i]);31mu_assert(Stack_peek(stack)==tests[i],"Wrongnextvalue.");32}33

34mu_assert(Stack_count(stack)==NUM_TESTS,"Wrongcountonpush.");3536STACK_FOREACH(stack,cur){37debug("VAL:%s",(char*)cur->value);38}3940for(i=NUM_TESTS-1;i>=0;i--){41char*val=Stack_pop(stack);42mu_assert(val==tests[i],"Wrongvalueonpop.");43}4445mu_assert(Stack_count(stack)==0,"Wrongcountafterpop.");4647returnNULL;48}4950char*all_tests()51{52mu_suite_start();5354mu_run_test(test_create);55mu_run_test(test_push_pop);56mu_run_test(test_destroy);5758returnNULL;59}6061RUN_TESTS(all_tests);

Then,thequeue_tests.cisalmostthesame,onlyusingQueue:

queue_tests.c


1#include"minunit.h"2#include<lcthw/queue.h>3#include<assert.h>45staticQueue*queue=NULL;6char*tests[]={"test1data","test2data","test3data"};78#defineNUM_TESTS3910char*test_create()11{12queue=Queue_create();13mu_assert(queue!=NULL,"Failedtocreatequeue.");1415returnNULL;16}1718char*test_destroy()19{20mu_assert(queue!=NULL,"Failedtomakequeue#2");21Queue_destroy(queue);2223returnNULL;24}2526char*test_send_recv()27{

28inti=0;29for(i=0;i<NUM_TESTS;i++){30Queue_send(queue,tests[i]);31mu_assert(Queue_peek(queue)==tests[0],"Wrongnextvalue.");32}3334mu_assert(Queue_count(queue)==NUM_TESTS,"Wrongcountonsend.");3536QUEUE_FOREACH(queue,cur){37debug("VAL:%s",(char*)cur->value);38}3940for(i=0;i<NUM_TESTS;i++){41char*val=Queue_recv(queue);42mu_assert(val==tests[i],"Wrongvalueonrecv.");43}4445mu_assert(Queue_count(queue)==0,"Wrongcountafterrecv.");4647returnNULL;48}4950char*all_tests()51{52mu_suite_start();5354mu_run_test(test_create);55mu_run_test(test_send_recv);56mu_run_test(test_destroy);5758returnNULL;59}6061RUN_TESTS(all_tests);

WhatYouShouldSeeYourunittestshouldrunwithoutyourhavingtochangethetests,anditshouldpassthedebuggerwithnomemoryerrors.Here’swhatitlookslikeifIrunstack_testsdirectly:

Exercise42.1Session


$./tests/stack_testsDEBUGtests/stack_tests.c:60:-----RUNNING:./tests/stack_tests

----

RUNNING:./tests/stack_tests

DEBUGtests/stack_tests.c:53:

-----test_create


-----test_push_pop

DEBUGtests/stack_tests.c:37:VAL:test3data




-----test_destroy

ALLTESTSPASSED

Testsrun:3

$

Thequeue_testisbasicallythesamekindofoutput,soIshouldn’thavetoshowittoyouatthisstage.

HowtoImproveItTheonlyrealimprovementyoucouldmaketothisisswitchingfromaListtoaDArray.TheQueuedatastructureismoredifficulttodowithaDArraybecauseitworksatbothendsofthelistofnodes.Onedisadvantageofdoingthisentirelyinaheaderfileisthatyoucan’teasilyperformancetuneit.Mostly,whatyou’redoingwiththistechniqueisestablishingaprotocolforhowtouseaListinacertainstyle.Whenperformancetuning,ifyoumakeListfast,thenthesetwoshouldimproveaswell.

ExtraCredit•ImplementStackusingDArrayinsteadofList,butwithoutchangingtheunittest.Thatmeansyou’llhavetocreateyourownSTACK_FOREACH.

Exercise43.ASimpleStatisticsEngine

ThisisasimplealgorithmthatIuseforcollectingsummarystatisticsonline,orwithoutstoringallofthesamples.Iusethisinanysoftwarethatneedstokeepsomestatistics,suchasmean,standarddeviation,andsum,butcan’tstoreallthesamplesneeded.Instead,Icanjuststoretherollingresultsofthecalculations,whichisonlyfivenumbers.

RollingStandardDeviationandMeanThefirstthingyouneedisasequenceofsamples.ThiscanbeanythingfromthetimeittakestocompleteatasktothenumberoftimessomeoneaccessessomethingtostarratingsonaWebsite.Itdoesn’treallymatterwhatitis,justsolongasyouhaveastreamofnumbersandyouwanttoknowthefollowingsummarystatisticsaboutthem:

sumThisisthetotalofallthenumbersaddedtogether.sumsquared(sumsq)Thisisthesumofthesquareofeachnumber.count(n)Thisisthenumbersamplesthatyou’vetaken.minThisisthesmallestsampleyou’veseen.maxThisisthelargestsampleyou’veseen.meanThisisthemostlikelymiddlenumber.It’snotactuallythemiddle,sincethat’sthemedian,butit’sanacceptedapproximationforit.

stddevThisiscalculatedusing$sqrt(sumsq–(sum×mean))/(n–1)))$wheresqrtisthesquarerootfunctioninthemath.hheader.

IwillconfirmthiscalculationworksusingR,sinceIknowRgetstheseright:

Exercise43.1Session


>s<-runif(n=10,max=10)>s[1]6.10613349.67832041.27470908.23951310.33334836.97550661.0626275[8]7.65875234.93829739.5788115>summary(s)Min.1stQu.MedianMean3rdQu.Max.0.33332.19106.54105.58508.09409.6780>sd(s)[1]3.547868

>sum(s)[1]55.84602

>sum(s*s)[1]425.1641

>sum(s)*mean(s)[1]311.8778

>sum(s*s)-sum(s)*mean(s)[1]113.2863

>(sum(s*s)-sum(s)*mean(s))/(length(s)-1)[1]12.58737

>sqrt((sum(s*s)-sum(s)*mean(s))/(length(s)-1))[1]3.547868

>

Youdon’tneedtoknowR.JustfollowalongwhileIexplainhowI’mbreakingthisdowntocheckmymath:

Lines1-4Iusethefunctionruniftogetarandomuniformdistributionofnumbers,thenprintthemout.I’llusetheseintheunittestlater.

Lines5-7Here’sthesummary,soyoucanseethevaluesthatRcalculatesforthese.Lines8-9Thisisthestddevusingthesdfunction.Lines10-11NowIbegintobuildthiscalculationmanually,firstbygettingthesum.Lines12-13Thenextpieceofthestdevformulaisthesumsq,whichIcangetwithsum(s*s)thattellsRtomultiplythewholeslistbyitself,andthensumthose.ThepowerofRisbeingabletodomathonentiredatastructureslikethis.

Lines14-15Lookingattheformula,Ithenneedthesummultipliedbymean,soIdosum(s)*mean(s).

Lines16-17Ithencombinethesumsqwiththistogetsum(s*s)-sum(s)*mean(s).Lines18-19Thatneedstobedividedby$n-1$,soIdo(sum(s*s)-sum(s)*mean(s))/(length(s)-1).

Lines20-21Finally,IsqrtthatandIget3.547868,whichmatchesthenumberRgavemeforsdabove.

ImplementationThat’showyoucalculatethestddev,sonowIcanmakesomesimplecodetoimplementthiscalculation.

stats.h


#ifndeflcthw_stats_h

#definelcthw_stats_h

typedefstructStats{doublesum;doublesumsq;unsignedlongn;doublemin;doublemax;}Stats;

Stats*Stats_recreate(doublesum,doublesumsq,unsignedlongn,doublemin,doublemax);

Stats*Stats_create();

doubleStats_mean(Stats*st);

doubleStats_stddev(Stats*st);

voidStats_sample(Stats*st,doubles);

voidStats_dump(Stats*st);

#endif

HereyoucanseethatI’veputthecalculationsIneedtostoreinastruct,andthenIhavefunctionsforsamplingandgettingthenumbers.Implementingthisisthenjustanexerciseinconvertingthemath:

stats.c


1#include<math.h>2#include<lcthw/stats.h>3#include<stdlib.h>4#include<lcthw/dbg.h>56Stats*Stats_recreate(doublesum,doublesumsq,unsignedlongn,7doublemin,doublemax)8{9Stats*st=malloc(sizeof(Stats));10check_mem(st);1112st->sum=sum;13st->sumsq=sumsq;14st->n=n;15st->min=min;16st->max=max;1718returnst;1920error:21returnNULL;22}2324Stats*Stats_create()25{26returnStats_recreate(0.0,0.0,0L,0.0,0.0);27}2829doubleStats_mean(Stats*st)30{31returnst->sum/st->n;32}3334doubleStats_stddev(Stats*st)35{36returnsqrt((st->sumsq-(st->sum*st->sum/st->n))/37(st->n-1));38}3940voidStats_sample(Stats*st,doubles)41{42st->sum+=s;43st->sumsq+=s*s;4445if(st->n==0){46st->min=s;47st->max=s;48}else{49if(st->min>s)50st->min=s;51if(st->max<s)

52st->max=s;53}5455st->n+=1;56}5758voidStats_dump(Stats*st)59{60fprintf(stderr,61"sum:%f,sumsq:%f,n:%ld,"62"min:%f,max:%f,mean:%f,stddev:%f",63st->sum,st->sumsq,st->n,st->min,st->max,Stats_mean(st),64Stats_stddev(st));65}

Here’sabreakdownofeachfunctioninstats.c:Stats_recreateI’llwanttoloadthesenumbersfromsomekindofdatabase,andthisfunctionlet’smerecreateaStatsstruct.

Stats_createThissimplycalledStats_recreatewithall0(zero)values.Stats_meanUsingthesumandn,itgivesthemean.Stats_stddevThisimplementstheformulaIworkedout;theonlydifferenceisthatIcalculatethemeanwithst->sum/st->ninthisformulainsteadofcallingStats_mean.

Stats_sampleThisdoestheworkofmaintainingthenumbersintheStatsstruct.Whenyougiveitthefirstvalue,itseesthatnis0andsetsminandmaxaccordingly.Everycallafterthatkeepsincreasingsum,sumsq,andn.Itthenfiguresoutifthisnewsampleisanewminormax.

Stats_dumpThisisasimpledebugfunctionthatdumpsthestatisticssoyoucanviewthem.ThelastthingIneedtodoisconfirmthatthismathiscorrect.I’mgoingtousenumbersandcalculationsfrommyRsessiontocreateaunittestthatconfirmsthatI’mgettingtherightresults.

stats_tests.c


1#include"minunit.h"2#include<lcthw/stats.h>3#include<math.h>45constintNUM_SAMPLES=10;6doublesamples[]={76.1061334,9.6783204,1.2747090,8.2395131,0.3333483,86.9755066,1.0626275,7.6587523,4.9382973,9.57881159};1011Statsexpect={12.sumsq=425.1641,13.sum=55.84602,14.min=0.333,15.max=9.678,16.n=10,17};1819doubleexpect_mean=5.584602;20doubleexpect_stddev=3.547868;2122#defineEQ(X,Y,N)(round((X)*pow(10,N))==round((Y)*pow(10,N)))

2324char*test_operations()25{26inti=0;27Stats*st=Stats_create();28mu_assert(st!=NULL,"Failedtocreatestats.");2930for(i=0;i<NUM_SAMPLES;i++){31Stats_sample(st,samples[i]);32}3334Stats_dump(st);3536mu_assert(EQ(st->sumsq,expect.sumsq,3),"sumsqnotvalid");37mu_assert(EQ(st->sum,expect.sum,3),"sumnotvalid");38mu_assert(EQ(st->min,expect.min,3),"minnotvalid");39mu_assert(EQ(st->max,expect.max,3),"maxnotvalid");40mu_assert(EQ(st->n,expect.n,3),"maxnotvalid");41mu_assert(EQ(expect_mean,Stats_mean(st),3),"meannotvalid");42mu_assert(EQ(expect_stddev,Stats_stddev(st),3),43"stddevnotvalid");4445returnNULL;46}4748char*test_recreate()49{50Stats*st=Stats_recreate(51expect.sum,expect.sumsq,expect.n,expect.min,expect.max);5253mu_assert(st->sum==expect.sum,"sumnotequal");54mu_assert(st->sumsq==expect.sumsq,"sumsqnotequal");55mu_assert(st->n==expect.n,"nnotequal");56mu_assert(st->min==expect.min,"minnotequal");57mu_assert(st->max==expect.max,"maxnotequal");58mu_assert(EQ(expect_mean,Stats_mean(st),3),"meannotvalid");59mu_assert(EQ(expect_stddev,Stats_stddev(st),3),60"stddevnotvalid");6162returnNULL;63}6465char*all_tests()66{67mu_suite_start();6869mu_run_test(test_operations);70mu_run_test(test_recreate);7172returnNULL;73}7475RUN_TESTS(all_tests);

There’snothingnewinthisunittest,exceptmaybetheEQmacro.Ifeltlazyanddidn’twanttolookupthestandardwaytotelliftwodoublevaluesareclose,soImadethismacro.Theproblemwithdoubleisthatequalityassumestotallyequalresults,butI’musingtwodifferentsystemswithslightlydifferentroundingerrors.ThesolutionistosaythatIwantthenumberstobe“equaltoXdecimalplaces.”IdothiswithEQbyraisingthenumbertoapowerof10,thenusingtheroundfunctiontogetan

integer.ThisisasimplewaytoroundtoNdecimalplacesandcomparetheresultsasaninteger.I’msurethereareabillionotherwaystodothesamething,butthisworksfornow.TheexpectedresultsaretheninaStatsstructandIsimplymakesurethatthenumberIgetisclosetothenumberRgaveme.

HowtoUseItYoucanusethestandarddeviationandmeantodetermineifanewsampleisinteresting,oryoucanusethistocollectstatisticsonstatistics.Thefirstoneiseasyforpeopletounderstand,soI’llexplainthatquicklyusinganexampleforlogintimes.Imagineyou’retrackinghowlongusersspendonaserver,andyou’reusingstatisticstoanalyzeit.Everytimesomeonelogsin,youkeeptrackofhowlongtheyarethere,thenyoucallStats_sample.I’mlookingforpeoplewhoareontoolongandalsopeoplewhoseemtobeontooquickly.Insteadofsettingspecificlevels,whatI’ddoiscomparehowlongsomeoneisonwiththemean(plusorminus)2*stddevrange.Igetthemeanand2*stddev,andconsiderlogintimestobeinterestingiftheyareoutsidethesetworanges.SinceI’mkeepingthesestatisticsusingarollingalgorithm,thisisaveryfastcalculation,andIcanthenhavethesoftwareflagtheuserswhoareoutsideofthisrange.Thisdoesn’tnecessarilypointoutpeoplewhoarebehavingbadly,butinsteaditflagspotentialproblemsthatyoucanreviewtoseewhat’sgoingon.It’salsodoingitbasedonthebehaviorofalloftheusers,whichavoidstheproblemofpickingsomearbitrarynumberthat’snotbasedonwhat’sreallyhappening.Thegeneralruleyoucangetfromthisisthatthemean(plusorminus)2*stddevisanestimateofwhere90%ofthevaluesareexpectedtofall,andanythingoutsidethatrangeisinteresting.ThesecondwaytousethesestatisticsistogometaandcalculatethemforotherStatscalculations.YoubasicallydoyourStats_samplelikenormal,butthenyourunStats_sampleonthemin,max,n,mean,andstddevonthatsample.Thisgivesatwo-levelmeasurement,andletsyoucomparesamplesofsamples.Confusing,right?I’llcontinuemyexampleabove,butlet’ssayyouhave100serversthateachholdadifferentapplication.You’realreadytrackingusers’logintimesforeachapplicationserver,butyouwanttocompareall100applicationsandflaganyusersthatareloggingintoomuchonallofthem.Theeasiestwaytodothatistocalculatethenewloginstatseachtimesomeonelogsin,andthenaddthatStatsstructselementtoasecondStat.Whatyouendupwithisaseriesofstatisticsthatcanbenamedlikethis:

meanofmeansThisisafullStatsstructthatgivesyoumeanandstddevofthemeansofalltheservers.Anyserveroruserwhoisoutsideofthisisworthlookingatonagloballevel.

meanofstddevsAnotherStatsstructthatproducesstatisticsonhowalloftheserversrange.Youcanthenanalyzeeachserverandseeifanyofthemhaveunusuallywide-rangingnumbersbycomparingtheirstddevtothismeanofstddevsstatistic.

Youcoulddothemall,butthesearethemostuseful.Ifyouthenwantedtomonitorserversforerraticlogintimes,you’ddothis:

•UserJohnlogsintoandoutofserverA.GrabserverA’sstatisticsandupdatethem.

•Grabthemeanofmeansstatistics,andthentakeA’smeanandadditasasample.I’llcallthism_of_m.•Grabthemeanofstddevsstatistics,andaddA’sstddevtoitasasample.I’llcallthism_of_s.•IfA’smeanisoutsideofm_of_m.mean+2*m_of_m.stddev,thenflagitaspossiblyhavingaproblem.•IfA’sstddevisoutsideofm_of_s.mean+2*m_of_s.stddev,thenflagitaspossiblybehavingtooerratically.•Finally,ifJohn’slogintimeisoutsideofA’srange,orA’sm_of_mrange,thenflagitasinteresting.

Usingthismeanofmeansandmeanofstddevscalculation,youcanefficientlytrackmanymetricswithaminimalamountofprocessingandstorage.

ExtraCredit•ConverttheStats_stddevandStats_meantostaticinlinefunctionsinthestats.hfileinsteadofinthestats.cfile.•Usethiscodetowriteaperformancetestofthestring_algos_test.c.Makeitoptional,andhaveitrunthebasetestasaseriesofsamples,andthenreporttheresults.•Writeaversionofthisinanotherprogramminglanguageyouknow.ConfirmthatthisversioniscorrectbasedonwhatIhavehere.•Writealittleprogramthatcantakeafilefullofnumbersandspitthesestatisticsoutforthem.•Maketheprogramacceptatableofdatathathasheadersononeline,thenalloftheothernumbersonlinesafteritareseparatedbyanynumberofspaces.Yourprogramshouldthenprintoutthesestatisticsforeachcolumnbytheheadername.

Exercise44.RingBuffer

RingbuffersareincrediblyusefulwhenprocessingasynchronousI/O.Theyallowonesidetoreceivedatainrandomintervalsofrandomsizes,butfeedcohesivechunkstoanothersideinsetsizesorintervals.TheyareavariantontheQueuedatastructurebutfocusonblocksofbytesinsteadofalistofpointers.Inthisexercise,I’mgoingtoshowyoutheRingBuffercode,andthenhaveyoumakeafullunittestforit.

ringbuffer.h


1#ifndef_lcthw_RingBuffer_h2#define_lcthw_RingBuffer_h34#include<lcthw/bstrlib.h>56typedefstruct{7char*buffer;8intlength;9intstart;10intend;11}RingBuffer;1213RingBuffer*RingBuffer_create(intlength);1415voidRingBuffer_destroy(RingBuffer*buffer);1617intRingBuffer_read(RingBuffer*buffer,char*target,intamount);1819intRingBuffer_write(RingBuffer*buffer,char*data,intlength);2021intRingBuffer_empty(RingBuffer*buffer);2223intRingBuffer_full(RingBuffer*buffer);2425intRingBuffer_available_data(RingBuffer*buffer);2627intRingBuffer_available_space(RingBuffer*buffer);2829bstringRingBuffer_gets(RingBuffer*buffer,intamount);3031#defineRingBuffer_available_data(B)(\32((B)->end+1)%(B)->length-(B)->start-1)3334#defineRingBuffer_available_space(B)(\35(B)->length-(B)->end-1)3637#defineRingBuffer_full(B)(RingBuffer_available_data((B))\38-(B)->length==0)3940#defineRingBuffer_empty(B)(\41RingBuffer_available_data((B))==0)4243#defineRingBuffer_puts(B,D)RingBuffer_write(\44(B),bdata((D)),blength((D)))4546#defineRingBuffer_get_all(B)RingBuffer_gets(\

47(B),RingBuffer_available_data((B)))4849#defineRingBuffer_starts_at(B)(\50(B)->buffer+(B)->start)5152#defineRingBuffer_ends_at(B)(\53(B)->buffer+(B)->end)5455#defineRingBuffer_commit_read(B,A)(\56(B)->start=((B)->start+(A))%(B)->length)5758#defineRingBuffer_commit_write(B,A)(\59(B)->end=((B)->end+(A))%(B)->length)6061#endif

Lookingatthedatastructure,youseeIhaveabuffer,start,andend.ARingBufferdoesnothingmorethanmovethestartandendaroundthebuffersothatitloopswheneveritreachesthebuffer ’send.Doingthisgivestheillusionofaninfinitereaddeviceinasmallspace.Ithenhaveabunchofmacrosthatdovariouscalculationsbasedonthis.Here’stheimplementation,whichisamuchbetterexplanationofhowthisworks.

ringbuffer.c


1#undefNDEBUG2#include<assert.h>3#include<stdio.h>4#include<stdlib.h>5#include<string.h>6#include<lcthw/dbg.h>7#include<lcthw/ringbuffer.h>89RingBuffer*RingBuffer_create(intlength)10{11RingBuffer*buffer=calloc(1,sizeof(RingBuffer));12buffer->length=length+1;13buffer->start=0;14buffer->end=0;15buffer->buffer=calloc(buffer->length,1);1617returnbuffer;18}1920voidRingBuffer_destroy(RingBuffer*buffer)21{22if(buffer){23free(buffer->buffer);24free(buffer);25}26}2728intRingBuffer_write(RingBuffer*buffer,char*data,intlength)29{30if(RingBuffer_available_data(buffer)==0){31buffer->start=buffer->end=0;32}3334check(length<=RingBuffer_available_space(buffer),

35"Notenoughspace:%drequest,%davailable",36RingBuffer_available_data(buffer),length);3738void*result=memcpy(RingBuffer_ends_at(buffer),data,length);39check(result!=NULL,"Failedtowritedataintobuffer.");4041RingBuffer_commit_write(buffer,length);4243returnlength;44error:45return-1;46}4748intRingBuffer_read(RingBuffer*buffer,char*target,intamount)49{50check_debug(amount<=RingBuffer_available_data(buffer),51"Notenoughinthebuffer:has%d,needs%d",52RingBuffer_available_data(buffer),amount);5354void*result=memcpy(target,RingBuffer_starts_at(buffer),amount);55check(result!=NULL,"Failedtowritebufferintodata.");5657RingBuffer_commit_read(buffer,amount);5859if(buffer->end==buffer->start){60buffer->start=buffer->end=0;61}6263returnamount;64error:65return-1;66}6768bstringRingBuffer_gets(RingBuffer*buffer,intamount)69{70check(amount>0,"Needmorethan0forgets,yougave:%d",71amount);72check_debug(amount<=RingBuffer_available_data(buffer),73"Notenoughinthebuffer.");7475bstringresult=blk2bstr(RingBuffer_starts_at(buffer),amount);76check(result!=NULL,"Failedtocreategetsresult.");77check(blength(result)==amount,"Wrongresultlength.");7879RingBuffer_commit_read(buffer,amount);80assert(RingBuffer_available_data(buffer)>=081&&"Errorinreadcommit.");8283returnresult;84error:85returnNULL;86}

ThisisallthereistoabasicRingBufferimplementation.Youcanreadandwriteblocksofdatatoit.Youcanaskhowmuchisinitandhowmuchspaceithas.TherearesomefancierringbuffersthatusetricksontheOStocreateanimaginaryinfinitestore,butthosearen’tportable.SincemyRingBufferdealswithreadingandwritingblocksofmemory,I’mmakingsurethatanytimeend==start,Iresetthemto0(zero)sothattheygotothebeginningofthebuffer.IntheWikipediaversionitisn’twritingblocksofdata,soitonlyhastomoveendandstartaroundinacircle.Tobetterhandleblocks,youhavetodroptothebeginningoftheinternalbufferwheneverthe

dataisempty.

TheUnitTestForyourunittest,you’llwanttotestasmanypossibleconditionsasyoucan.TheeasiestwaytodothatistopreconstructdifferentRingBufferstructs,andthenmanuallycheckthatthefunctionsandmathworkright.Forexample,youcouldmakeonewhereendisrightattheendofthebufferandstartisrightbeforethebuffer,andthenseehowitfails.

WhatYouShouldSeeHere’smyringbuffer_testsrun:

Exercise44.1Session


$./tests/ringbuffer_testsDEBUGtests/ringbuffer_tests.c:60:-----RUNNING:./tests/ringbuffer_tests

----

RUNNING:./tests/ringbuffer_tests

DEBUGtests/ringbuffer_tests.c:53:

-----test_create


-----test_read_write


-----test_destroy

ALLTESTSPASSED

Testsrun:3

$

Youshouldhaveatleastthreeteststhatconfirmallofthebasicoperations,andthenseehowmuchmoreyoucantestbeyondwhatI’vedone.

HowtoImproveItAsusual,youshouldgobackandadddefensiveprogrammingcheckstothisexercise.Hopefullyyou’vebeendoingthis,becausethebasecodeinmostofliblcthwdoesn’thavethecommondefensiveprogrammingchecksthatI’mteachingyou.Ileavethistoyousothatyoucangetusedtoimprovingcodewiththeseextrachecks.Forexample,inthisringbuffer,there’snotalotofcheckingthatanaccesswillactuallybeinsidethebuffer.Ifyoureadthe“Circularbuffer”pageonWikipedia,you’llseethe“OptimizedPOSIXimplementation”thatusesPortableOperatingSystemInterface(POSIX)-specificcallstocreateaninfinitespace.StudythatandI’llhaveyoutryitintheExtraCreditsection.

ExtraCredit•CreateanalternativeimplementationofRingBufferthatusesthePOSIXtrickandthencreateaunittestforit.•Addaperformancecomparisontesttothisunittestthatcomparesthetwoversionsbyfuzzingthemwithrandomdataandrandomread/writeoperations.Makesurethatyousetupthisfuzzingsothatthesameoperationsaredonetoeachversion,andyoucancomparethembetweenruns.

Exercise45.ASimpleTCP/IPClient

ImgoingtousetheRingBuffertocreateaverysimplisticnetworktestingtoolthatIcallnetclient.Todothis,IhavetoaddsomestufftotheMakefiletohandlelittleprogramsinthebin/directory.

AugmenttheMakefileFirst,addavariablefortheprogramsjustliketheunittest’sTESTSandTEST_SRCvariables:Clickheretoviewcodeimage

PROGRAMS_SRC=$(wildcardbin/*.c)PROGRAMS=$(patsubst%.c,%,$(PROGRAMS_SRC))

Then,youwanttoaddthePROGRAMStothealltarget:Clickheretoviewcodeimage

all:$(TARGET)$(SO_TARGET)tests$(PROGRAMS)

Then,addPROGRAMStothermlineinthecleantarget:Clickheretoviewcodeimage

rm–rfbuild$(OBJECTS)$(TESTS)$(PROGRAMS)

Finally,youjustneedatargetattheendtobuildthemall:Clickheretoviewcodeimage

$(PROGRAMS):CFLAGS+=$(TARGET)

Withthesechanges,youcandropsimple.cfilesintobin,andmakewillbuildthemandlinkthemtothelibraryjustlikeunittestsdo.

ThenetclientCodeThecodeforthelittlenetclientlookslikethis:

netclient.c


1#undefNDEBUG2#include<stdlib.h>3#include<sys/select.h>4#include<stdio.h>5#include<lcthw/ringbuffer.h>6#include<lcthw/dbg.h>7#include<sys/socket.h>8#include<sys/types.h>9#include<sys/uio.h>10#include<arpa/inet.h>11#include<netdb.h>12#include<unistd.h>13#include<fcntl.h>1415structtagbstringNL=bsStatic("\n");

16structtagbstringCRLF=bsStatic("\r\n");1718intnonblock(intfd)19{20intflags=fcntl(fd,F_GETFL,0);21check(flags>=0,"Invalidflagsonnonblock.");2223intrc=fcntl(fd,F_SETFL,flags|O_NONBLOCK);24check(rc==0,"Can'tsetnonblocking.");2526return0;27error:28return-1;29}3031intclient_connect(char*host,char*port)32{33intrc=0;34structaddrinfo*addr=NULL;3536rc=getaddrinfo(host,port,NULL,&addr);37check(rc==0,"Failedtolookup%s:%s",host,port);3839intsock=socket(AF_INET,SOCK_STREAM,0);40check(sock>=0,"Cannotcreateasocket.");4142rc=connect(sock,addr->ai_addr,addr->ai_addrlen);43check(rc==0,"Connectfailed.");4445rc=nonblock(sock);46check(rc==0,"Can'tsetnonblocking.");4748freeaddrinfo(addr);49returnsock;5051error:52freeaddrinfo(addr);53return-1;54}5556intread_some(RingBuffer*buffer,intfd,intis_socket)57{58intrc=0;5960if(RingBuffer_available_data(buffer)==0){61buffer->start=buffer->end=0;62}6364if(is_socket){65rc=recv(fd,RingBuffer_starts_at(buffer),66RingBuffer_available_space(buffer),0);67}else{68rc=read(fd,RingBuffer_starts_at(buffer),69RingBuffer_available_space(buffer));70}7172check(rc>=0,"Failedtoreadfromfd:%d",fd);7374RingBuffer_commit_write(buffer,rc);7576returnrc;7778error:79return-1;

80}8182intwrite_some(RingBuffer*buffer,intfd,intis_socket)83{84intrc=0;85bstringdata=RingBuffer_get_all(buffer);8687check(data!=NULL,"Failedtogetfromthebuffer.");88check(bfindreplace(data,&NL,&CRLF,0)==BSTR_OK,89"FailedtoreplaceNL.");9091if(is_socket){92rc=send(fd,bdata(data),blength(data),0);93}else{94rc=write(fd,bdata(data),blength(data));95}9697check(rc==blength(data),"Failedtowriteeverythingtofd:%d.",98fd);99bdestroy(data);100101returnrc;102103error:104return-1;105}106107intmain(intargc,char*argv[])108{109fd_setallreads;110fd_setreadmask;111112intsocket=0;113intrc=0;114RingBuffer*in_rb=RingBuffer_create(1024*10);115RingBuffer*sock_rb=RingBuffer_create(1024*10);116117check(argc==3,"USAGE:netclienthostport");118119socket=client_connect(argv[1],argv[2]);120check(socket>=0,"connectto%s:%sfailed.",argv[1],argv[2]);121122FD_ZERO(&allreads);123FD_SET(socket,&allreads);124FD_SET(0,&allreads);125126while(1){127readmask=allreads;128rc=select(socket+1,&readmask,NULL,NULL,NULL);129check(rc>=0,"selectfailed.");130131if(FD_ISSET(0,&readmask)){132rc=read_some(in_rb,0,0);133check_debug(rc!=-1,"Failedtoreadfromstdin.");134}135136if(FD_ISSET(socket,&readmask)){137rc=read_some(sock_rb,socket,0);138check_debug(rc!=-1,"Failedtoreadfromsocket.");139}140141while(!RingBuffer_empty(sock_rb)){142rc=write_some(sock_rb,1,0);

143check_debug(rc!=-1,"Failedtowritetostdout.");144}145146while(!RingBuffer_empty(in_rb)){147rc=write_some(in_rb,socket,1);148check_debug(rc!=-1,"Failedtowritetosocket.");149}150}151152return0;153154error:155return-1;156}

Thiscodeusesselecttohandleeventsfrombothstdin(filedescriptor0)andsocket,whichitusestotalktoaserver.ThecodeusesRingBufferstostorethedataandcopyitaround.Youcanconsiderthefunctionsread_someandwrite_someearlyprototypesforsimilarfunctionsintheRingBufferlibrary.Thislittlebitofcodecontainsquiteafewnetworkingfunctionsthatyoumaynotknow.Asyoucomeacrossafunctionthatyoudon’tknow,lookitupinthemanpagesandmakesureyouunderstandit.ThisonelittlefilemightinspireyoutothenresearchalloftheAPIsrequiredtowritealittleserverinC.

WhatYouShouldSeeIfyouhaveeverythingbuilding,thenthequickestwaytotestthecodeisseeifyoucangetaspecialfileoffofhttp://learncodethehardway.org.

Exercise45.1Session


$$./bin/netclientlearncodethehardway.org80GET/ex45.txtHTTP/1.1

Host:learncodethehardway.org

HTTP/1.1200OK

Date:Fri,27Apr201200:41:25GMT

Content-Type:text/plain

Content-Length:41

Last-Modified:Fri,27Apr201200:42:11GMT

ETag:4f99eb63-29

Server:Mongrel2/1.7.5

LearnCTheHardWay,Exercise45works.

^C

$

WhatIdohereistypeinthesyntaxneededtomaketheHTTPrequestforthefile/ex45.txt,thentheHost:headerline,andthenIpressENTERtogetanemptyline.Ithengettheresponse,withheadersandthecontent.Afterthat,IjusthitCTRL-Ctoexit.

http://learncodethehardway.org

HowtoBreakItThiscodecoulddefinitelyhavebugs,andcurrentlyinthedraftofthisbook,I’mgoingtohavetokeepworkingonit.Inthemeantime,tryanalyzingthecodeIhavehereandthrashingitagainstotherservers.There’satoolcallednetcatthat’sgreatforsettingupthesekindsofservers.AnotherthingtodoisusealanguagelikePythonorRubytocreateasimplejunkserverthatspewsoutjunkandbaddata,randomlyclosesconnections,anddoesothernastythings.Ifyoufindbugs,reporttheminthecomments,andI’llfixthemup.

ExtraCredit•AsImentioned,therearequiteafewfunctionsyoumaynotknow,solookthemup.Infact,lookthemallupevenifyouthinkyouknowthem.•Runthisunderthedebuggerandlookforerrors.•Gobackthroughandaddvariousdefensiveprogrammingcheckstothefunctionstoimprovethem.•Usethegetoptfunctiontoallowtheusertheoptionnottotranslate\nto\r\n.Thisisonlyneededonprotocolsthatrequireitforlineendings,likeHTTP.Sometimesyoudon’twantthetranslation,sogivetheusertheoption.

Exercise46.TernarySearchTree

ThefinaldatastructurethatI’llshowyouiscalledtheTSTree,whichissimilartotheBSTree,exceptithasthreebranches:low,equal,andhigh.It’sprimarilyusedjustlikeBSTreeandHashmaptostorekey/valuedata,butitworksoffoftheindividualcharactersinthekeys.ThisgivestheTSTreesomeabilitiesthatneitherBSTreenorHashmaphas.InaTSTree,everykeyisastring,andit’sinsertedbywalkingthroughandbuildingatreebasedontheequalityofthecharactersinthestring.Itstartsattheroot,looksatthecharacterforthatnode,andifit’slower,equalto,orhigherthanthat,thenitgoesinthatdirection.Youcanseethisintheheaderfile:

tstree.h


#ifndef_lcthw_TSTree_h

#define_lcthw_TSTree_h

#include<stdlib.h>


typedefstructTSTree{charsplitchar;structTSTree*low;structTSTree*equal;structTSTree*high;void*value;}TSTree;

void*TSTree_search(TSTree*root,constchar*key,size_tlen);

void*TSTree_search_prefix(TSTree*root,constchar*key,size_tlen);

typedefvoid(*TSTree_traverse_cb)(void*value,void*data);

TSTree*TSTree_insert(TSTree*node,constchar*key,size_tlen,void*value);

voidTSTree_traverse(TSTree*node,TSTree_traverse_cbcb,void*data);

voidTSTree_destroy(TSTree*root);

#endif

TheTSTreehasthefollowingelements:splitcharThecharacteratthispointinthetree.lowThebranchthat’slowerthansplitchar.equalThebranchthat’sequaltosplitchar.highThebranchthat’shigherthansplitchar.valueThevaluesetforastringatthatpointwithsplitchar.

Youcanseethatthisimplementationhasthefollowingoperations:

searchAtypicaloperationtofindavalueforthiskey.search_prefixThisoperationfindsthefirstvaluethathasthisasaprefixofitskey.Thisistheanoperationthatyoucan’teasilydoinaBSTreeorHashmap.

insertThisbreaksthekeydownbyeachcharacterandinsertsthemintothetree.traverseThiswalksthroughthetree,allowingyoutocollectoranalyzeallthekeysandvaluesitcontains.

TheonlythingmissingisaTSTree_delete,andthat’sbecauseit’sahorriblyexpensiveoperation,evenmoreexpensivethanBSTree_delete.WhenIuseTSTreestructures,ItreatthemasconstantdatathatIplanontraversingmanytimes,andnotremovinganythingfromthem.Theyareveryfastforthis,butaren’tgoodifyouneedtoinsertanddeletethingsquickly.Forthat,IuseHashmap,sinceitbeatsbothBSTreeandTSTree.TheimplementationfortheTSTreeisactuallysimple,butitmightbehardtofollowatfirst.I’llbreakitdownafteryouenteritin:

tstree.c


1#include<stdlib.h>2#include<stdio.h>3#include<assert.h>4#include<lcthw/dbg.h>5#include<lcthw/tstree.h>67staticinlineTSTree*TSTree_insert_base(TSTree*root,TSTree*node,8constchar*key,size_tlen,9void*value)10{11if(node==NULL){12node=(TSTree*)calloc(1,sizeof(TSTree));1314if(root==NULL){15root=node;16}1718node->splitchar=*key;19}2021if(*key<node->splitchar){22node->low=TSTree_insert_base(23root,node->low,key,len,value);24}elseif(*key==node->splitchar){25if(len>1){26node->equal=TSTree_insert_base(27root,node->equal,key+1,len-1,value);28}else{29assert(node->value==NULL&&"Duplicateinsertintotst.");30node->value=value;31}32}else{33node->high=TSTree_insert_base(34root,node->high,key,len,value);35}3637returnnode;

38}3940TSTree*TSTree_insert(TSTree*node,constchar*key,size_tlen,41void*value)42{43returnTSTree_insert_base(node,node,key,len,value);44}4546void*TSTree_search(TSTree*root,constchar*key,size_tlen)47{48TSTree*node=root;49size_ti=0;5051while(i<len&&node){52if(key[i]<node->splitchar){53node=node->low;54}elseif(key[i]==node->splitchar){55i++;56if(i<len)57node=node->equal;58}else{59node=node->high;60}61}6263if(node){64returnnode->value;65}else{66returnNULL;67}68}6970void*TSTree_search_prefix(TSTree*root,constchar*key,size_tlen)71{72if(len==0)73returnNULL;7475TSTree*node=root;76TSTree*last=NULL;77size_ti=0;7879while(i<len&&node){80if(key[i]<node->splitchar){81node=node->low;82}elseif(key[i]==node->splitchar){83i++;84if(i<len){85if(node->value)86last=node;87node=node->equal;88}89}else{90node=node->high;91}92}9394node=node?node:last;9596//traverseuntilwefindthefirstvalueintheequalchain97//thisisthenthefirstnodewiththisprefix98while(node&&!node->value){99node=node->equal;100}101

102returnnode?node->value:NULL;103}104105voidTSTree_traverse(TSTree*node,TSTree_traverse_cbcb,void*data)106{107if(!node)108return;109110if(node->low)111TSTree_traverse(node->low,cb,data);112113if(node->equal){114TSTree_traverse(node->equal,cb,data);115}116117if(node->high)118TSTree_traverse(node->high,cb,data);119120if(node->value)121cb(node->value,data);122}123124voidTSTree_destroy(TSTree*node)125{126if(node==NULL)127return;128129if(node->low)130TSTree_destroy(node->low);131132if(node->equal){133TSTree_destroy(node->equal);134}135136if(node->high)137TSTree_destroy(node->high);138139free(node);140}

ForTSTree_insert,I’musingthesamepatternforrecursivestructureswhereIhaveasmallfunctionthatcallstherealrecursivefunction.I’mnotdoinganyadditionalcheckshere,butyoushouldaddtheusualdefensiveprogrammingcheckstoit.Onethingtokeepinmindisthatit’susingaslightlydifferentdesignthatdoesn’thaveaseparateTSTree_createfunction.However,ifyoupassitaNULLforthenode,thenitwillcreateitandreturnthefinalvalue.ThatmeansIneedtobreakdownTSTree_insert_basesothatyouunderstandtheinsertoperation:

tstree.c:10-18AsImentioned,ifgivenaNULL,thenIneedtomakethisnodeandassignthe*key(currentcharacter)toit.Thisisusedtobuildthetreeasweinsertkeys.

tstree.c:20-21Ifthe*keyislessthanthis,thenrecurse,butgotothelowbranch.tstree.c:22Thissplitcharisequal,soIwanttogoanddealwithequality.Thiswillhappenifwejustcreatethisnode,sowe’llbebuildingthetreeatthispoint.

tstree.c:23-24Therearestillcharacterstohandle,sorecursedowntheequalbranch,butgotothenext*keycharacter.

tstree.c:26-27Thisisthelastcharacter,soIsetthevalueandthat’sit.Ihaveanassertherein

caseofaduplicate.tstree.c:29-30Thelastconditionisthatthis*keyisgreaterthansplitchar,soIneedtorecursedownthehighbranch.

ThekeytothisdatastructureisthefactthatI’monlyincrementingthecharacterwhenasplitcharisequal.Fortheothertwoconditions,IjustwalkthroughthetreeuntilIhitanequalcharactertorecurseintonext.Whatthisdoesismakeitveryfastnottofindakey.Icangetabadkey,andsimplywalkthroughafewhighandlownodesuntilIhitadeadendbeforeIknowthatthiskeydoesn’texist.Idon’tneedtoprocesseverycharacterofthekeyoreverynodeofthetree.Onceyouunderstandthat,thenmoveontoanalyzinghowTSTree_searchworks.

tstree.c:46Idon’tneedtoprocessthetreerecursivelyintheTSTree.Icanjustuseawhile-loopandanodeforwhereIcurrentlyam.

tstree.c:47-48Ifthecurrentcharacterislessthanthenodesplitchar,thengolow.tstree.c:49-51Ifit’sequal,thenincrementiandgoequalaslongasit’snotthelastcharacter.That’swhytheif(i<len)isthere,sothatIdon’tgotoofarpastthefinalvalue.

tstree.c:52-53Otherwise,Igohigh,sincethecharacterisgreater.tstree.c:57-61IfIhaveanodeaftertheloop,thenreturnitsvalue,otherwisereturnNULL.

Thisisn’ttoodifficulttounderstand,andyoucanseethatit’salmostexactlythesamealgorithmfortheTSTree_search_prefixfunction.TheonlydifferenceisthatI’mnottryingtofindanexactmatch,butfindthelongestprefixIcan.Todothat,Ikeeptrackofthelastnodethatwasequal,andthenafterthesearchloop,walkthroughthatnodeuntilIfindavalue.LookingatTSTree_search_prefix,youcanstarttoseethesecondadvantageaTSTreehasovertheBSTreeandHashmapforfindingstrings.GivenanykeyofXlength,youcanfindanykeyinXmoves.YoucanalsofindthefirstprefixinXmoves,plusNmoredependingonhowbigthematchingkeyis.Ifthebiggestkeyinthetreeistencharacterslong,thenyoucanfindanyprefixinthatkeyintenmoves.Moreimportantly,youcandoallofthisbycomparingeachcharacterofthekeyonce.Incomparison,todothesamewithaBSTree,youwouldhavetochecktheprefixesofeachcharacterineverypossiblematchingnodeintheBSTreeagainstthecharactersintheprefix.It’sthesameforfindingkeysorseeingifakeydoesn’texist.YouhavetocompareeachcharacteragainstmostofthecharactersintheBSTreetofindornotfindamatch.AHashmapisevenworseforfindingprefixes,becauseyoucan’thashjusttheprefix.Basically,youcan’tdothisefficientlyinaHashmapunlessthedataissomethingyoucanparse,likeaURL.Eventhen,thatusuallyrequireswholetreesofHashmaps.Thelasttwofunctionsshouldbeeasyforyoutoanalyzesincethey’rethetypicaltraversinganddestroyingoperationsthatyou’vealreadyseeninotherdatastructures.Finally,Ihaveasimpleunittestforthewholethingtomakesureitworksright:

tstree_tests.c


1#include"minunit.h"

2#include<lcthw/tstree.h>3#include<string.h>4#include<assert.h>5#include<lcthw/bstrlib.h>67TSTree*node=NULL;8char*valueA="VALUEA";9char*valueB="VALUEB";10char*value2="VALUE2";11char*value4="VALUE4";12char*reverse="VALUER";13inttraverse_count=0;1415structtagbstringtest1=bsStatic("TEST");16structtagbstringtest2=bsStatic("TEST2");17structtagbstringtest3=bsStatic("TSET");18structtagbstringtest4=bsStatic("T");1920char*test_insert()21{22node=TSTree_insert(node,bdata(&test1),blength(&test1),valueA);23mu_assert(node!=NULL,"Failedtoinsertintotst.");2425node=TSTree_insert(node,bdata(&test2),blength(&test2),value2);26mu_assert(node!=NULL,27"Failedtoinsertintotstwithsecondname.");2829node=TSTree_insert(node,bdata(&test3),blength(&test3),reverse);30mu_assert(node!=NULL,31"Failedtoinsertintotstwithreversename.");3233node=TSTree_insert(node,bdata(&test4),blength(&test4),value4);34mu_assert(node!=NULL,35"Failedtoinsertintotstwithsecondname.");3637returnNULL;38}3940char*test_search_exact()41{42//tstreturnsthelastoneinserted43void*res=TSTree_search(node,bdata(&test1),blength(&test1));44mu_assert(res==valueA,45"Gotthewrongvalueback,shouldgetAnotB.");4647//tstdoesnotfindifnotexact48res=TSTree_search(node,"TESTNO",strlen("TESTNO"));49mu_assert(res==NULL,"Shouldnotfindanything.");5051returnNULL;52}5354char*test_search_prefix()55{56void*res=TSTree_search_prefix(57node,bdata(&test1),blength(&test1));58debug("result:%p,expected:%p",res,valueA);59mu_assert(res==valueA,"GotwrongvalueAbyprefix.");6061res=TSTree_search_prefix(node,bdata(&test1),1);62debug("result:%p,expected:%p",res,valueA);63mu_assert(res==value4,"Gotwrongvalue4forprefixof1.");6465res=TSTree_search_prefix(node,"TE",strlen("TE"));

66mu_assert(res!=NULL,"Shouldfindforshortprefix.");6768res=TSTree_search_prefix(node,"TE--",strlen("TE--"));69mu_assert(res!=NULL,"Shouldfindforpartialprefix.");7071returnNULL;72}7374voidTSTree_traverse_test_cb(void*value,void*data)75{76assert(value!=NULL&&"ShouldnotgetNULLvalue.");77assert(data==valueA&&"ExpectingvalueAasthedata.");78traverse_count++;79}8081char*test_traverse()82{83traverse_count=0;84TSTree_traverse(node,TSTree_traverse_test_cb,valueA);85debug("traversecountis:%d",traverse_count);86mu_assert(traverse_count==4,"Didn'tfind4keys.");8788returnNULL;89}9091char*test_destroy()92{93TSTree_destroy(node);9495returnNULL;96}9798char*all_tests()99{100mu_suite_start();101102mu_run_test(test_insert);103mu_run_test(test_search_exact);104mu_run_test(test_search_prefix);105mu_run_test(test_traverse);106mu_run_test(test_destroy);107108returnNULL;109}110111RUN_TESTS(all_tests);

AdvantagesandDisadvantagesThereareotherinteresting,practicalthingsyoucandowithaTSTree:

•Inadditiontofindingprefixes,youcanreverseallofthekeysyouinsert,andthenfindthingsbysuffix.Iusethistolookuphostnames,sinceIwanttofind*.learncodethehardway.com.IfIgobackward,Icanmatchthemquickly.•Youcandoapproximatematching,bygatheringallofthenodesthathavemostofthesamecharactersasthekey,orusingotheralgorithmstofindaclosematch.•Youcanfindallofthekeysthathaveapartinthemiddle.

I’vealreadytalkedaboutsomeofthethingsTSTreescando,buttheyaren’tthebestdatastructureallthetime.HerearethedisadvantagesoftheTSTree:

•AsImentioned,deletingfromthemismurder.Theyarebetterusedfordatathatneedstobelookedupfastandrarelyremoved.Ifyouneedtodelete,thensimplydisablethevalueandthenperiodicallyrebuildthetreewhenitgetstoobig.•ItusesatonofmemorycomparedtoBSTreeandHashmapsforthesamekeyspace.Thinkaboutit.It’susingafullnodeforeachcharacterineverykey.Itmightworkbetterforsmallerkeys,butifyouputalotinaTSTree,itwillgethuge.•Theyalsodon’tworkwellwithlargekeys,butlargeissubjective.Asusual,testitfirst.Ifyou’retryingtostore10,000-characterkeys,thenuseaHashmap.

HowtoImproveItAsusual,gothroughandimprovethisbyaddingthedefensiveprogrammingpreconditions,asserts,andcheckstoeachfunction.Therearesomeotherpossibleimprovements,butyoudon’tnecessarilyhavetoimplementallofthese:

•YoucouldallowduplicatesbyusingaDArrayinsteadofthevalue.•AsImentionedearlier,deletingishard,butyoucouldsimulateitbysettingthevaluestoNULLsothattheyareeffectivelygone.•Therearenowaystocollectallofthepossiblematchingvalues.I’llhaveyouimplementthatinanExtraCreditexercise.•Thereareotheralgorithmsthataremorecomplexbuthaveslightlybetterproperties.Takealookatsuffixarray,suffixtree,andradixtreestructures.

ExtraCredit•ImplementaTSTree_collectthatreturnsaDArraycontainingallofthekeysthatmatchthegivenprefix.•ImplementTSTree_search_suffixandaTSTree_insert_suffixsoyoucandosuffixsearchesandinserts.•UsethedebuggertoseehowthisstructurestoresdatacomparedtotheBSTreeandHashmap.

Exercise47.AFastURLRouter

ImnowgoingtoshowyouhowIusetheTSTreetodofastURLroutinginWebserversthatI’vewritten.ThisworksforsimpleURLroutingthatyoumightuseattheedgeofanapplication,butitdoesn’treallyworkforthemorecomplex(andsometimesunnecessary)routingfoundinmanyWebapplicationframeworks.Toplaywithrouting,I’mgoingtomakealittlecommandlinetoolthatI’mcallingurlor,whichreadsasimplefileofroutes,andthenpromptstheusertoenterinURLs.

urlor.c


1#include<lcthw/tstree.h>2#include<lcthw/bstrlib.h>34TSTree*add_route_data(TSTree*routes,bstringline)5{6structbstrList*data=bsplit(line,'');7check(data->qty==2,"Line'%s'doesnothave2columns",8bdata(line));910routes=TSTree_insert(routes,11bdata(data->entry[0]),12blength(data->entry[0]),13bstrcpy(data->entry[1]));1415bstrListDestroy(data);1617returnroutes;1819error:20returnNULL;21}2223TSTree*load_routes(constchar*file)24{25TSTree*routes=NULL;26bstringline=NULL;27FILE*routes_map=NULL;2829routes_map=fopen(file,"r");30check(routes_map!=NULL,"Failedtoopenroutes:%s",file);3132while((line=bgets((bNgetc)fgetc,routes_map,'\n'))!=NULL){33check(btrimws(line)==BSTR_OK,"Failedtotrimline.");34routes=add_route_data(routes,line);35check(routes!=NULL,"Failedtoaddroute.");36bdestroy(line);37}3839fclose(routes_map);40returnroutes;4142error:43if(routes_map)fclose(routes_map);44if(line)bdestroy(line);45

46returnNULL;47}4849bstringmatch_url(TSTree*routes,bstringurl)50{51bstringroute=TSTree_search(routes,bdata(url),blength(url));5253if(route==NULL){54printf("Noexactmatchfound,tryingprefix.\n");55route=TSTree_search_prefix(routes,bdata(url),blength(url));56}5758returnroute;59}6061bstringread_line(constchar*prompt)62{63printf("%s",prompt);6465bstringresult=bgets((bNgetc)fgetc,stdin,'\n');66check_debug(result!=NULL,"stdinclosed.");6768check(btrimws(result)==BSTR_OK,"Failedtotrim.");6970returnresult;7172error:73returnNULL;74}7576voidbdestroy_cb(void*value,void*ignored)77{78(void)ignored;79bdestroy((bstring)value);80}8182voiddestroy_routes(TSTree*routes)83{84TSTree_traverse(routes,bdestroy_cb,NULL);85TSTree_destroy(routes);86}8788intmain(intargc,char*argv[])89{90bstringurl=NULL;91bstringroute=NULL;92TSTree*routes=NULL;9394check(argc==2,"USAGE:urlor<urlfile>");9596routes=load_routes(argv[1]);97check(routes!=NULL,"Yourroutefilehasanerror.");9899while(1){100url=read_line("URL>");101check_debug(url!=NULL,"goodbye.");102103route=match_url(routes,url);104105if(route){106printf("MATCH:%s==%s\n",bdata(url),bdata(route));107}else{108printf("FAIL:%s\n",bdata(url));109}

110111bdestroy(url);112}113114destroy_routes(routes);115return0;116117error:118destroy_routes(routes);119return1;120}

I’llthenmakeasimplefilewithsomefakeroutestoplaywith:/MainApp

/helloHello

/hello/Hello

/signupSignup

/logoutLogout

/album/Album

WhatYouShouldSeeOnceyouhaveurlorworking,andaroutesfile,youcantryitouthere:

Exercise47Session


$./bin/urlorurls.txtURL>/

MATCH:/==MainApp

URL>/hello

MATCH:/hello==Hello

URL>/hello/zed

Noexactmatchfound,tryingprefix.

MATCH:/hello/zed==Hello

URL>/album


MATCH:/album==Album

URL>/album/12345


MATCH:/album/12345==Album

URL>asdfasfdasfd


FAIL:asdfasfdasfd

URL>/asdfasdfasf


MATCH:/asdfasdfasf==MainApp

URL>

$

Youcanseethattheroutingsystemfirsttriesanexactmatch,andifitcan’tfindone,itwillgiveaprefixmatch.Thisismostlydonetotryoutthedifferencebetweenthetwo.DependingonthesemanticsofyourURLs,youmaywanttoalwaysmatchexactly,alwaystoprefixes,ordobothandpickthebestone.

HowtoImproveItURLsareweirdbecausepeoplewantthemtomagicallyhandlealloftheinsanethingstheirWebapplicationsdo,evenifthat’snotverylogical.InthissimpledemonstrationofhowtousetheTSTreetodorouting,therearesomeflawsthatpeoplewouldn’tbeabletoarticulate.Forexample,theTSTreewillmatch/altoAlbum,whichgenerallyisn’twhattheywant.Theywant/album/*tomatchAlbum,and/altobea404error.Thisisn’tdifficulttoimplement,though,sinceyoucouldchangetheprefixalgorithmtomatchanywayyouwant.Ifyouchangethematchingalgorithmtofindallmatchingprefixes,andthenpickthebestone,you’llbeabletodoiteasily.Inthiscase,/alcouldmatchMainApporAlbum.Takethoseresults,andthendoalittlelogictodeterminewhichisbetter.AnotherthingyoucandoinarealroutingsystemisusetheTSTreetofindallpossiblematches,butthesematchesareasmallsetofpatternstocheck.InmanyWebapplications,there’salistofregularexpressions(regex)thathastobematchedagainstURLsoneachrequest.Runningalloftheregexcanbetimeconsuming,soyoucanuseaTSTreetofindallofthepossiblematchesbytheirprefixes.Thatwayyounarrowdownthepatternstotrytoafewveryquickly.Usingthismethod,yourURLswillmatchexactlysinceyou’reactuallyrunningrealregexpatterns,andthey’llmatchmuchfastersinceyou’refindingthembypossibleprefixes.Thiskindofalgorithmalsoworksforanythingelsethatneedstohaveflexibleuser-visibleroutingmechanisms:domainnames,IPaddresses,registriesanddirectories,files,orURLs.

ExtraCredit•Insteadofjuststoringthestringforthehandler,createanactualenginethatusesaHandlerstructtostoretheapplication.ThestructurewouldstoretheURLtowhichit’sattached,thename,andanythingelseyou’dneedtomakeanactualroutingsystem.•InsteadofmappingURLstoarbitrarynames,mapthemto.sofilesandusethedlopensystemtoloadhandlersontheflyandcallcallbackstheycontain.PutthesecallbacksinyourHandlerstruct,andthenyouhaveyourselfafullydynamiccallbackhandlersysteminC.

Exercise48.ASimpleNetworkServer

Wenowstartthepartofthebookwhereyoudoalong-running,moreinvolvedprojectinaseriesofexercises.Thelastfiveexerciseswillpresenttheproblemofcreatingasimplenetworkserverinasimilarfashionasyoudidwiththelogfindproject.I’lldescribeeachphaseoftheproject,you’llattemptit,andthenyou’llcompareyourimplementationtominebeforecontinuing.Thesedescriptionsarepurposefullyvaguesothatyouhavethefreedomtoattempttosolvetheproblemsonyourown,butI’mstillgoingtohelpyou.Includedwitheachoftheseexercisesaretwovideos.Thefirstvideoshowsyouhowtheprojectfortheexerciseshouldwork,soyoucanseeitinactionandtrytoemulateit.ThesecondvideoshowsyouhowIsolvedtheproblem,soyoucancompareyourattempttomine.Finally,you’llhaveaccesstoallofthecodeintheGitHubproject,soyoucanseerealcodebyme.Youshouldattempttheproblemfirst,thenafteryougetitworking(orifyougettotallystuck),gowatchthesecondvideoandtakealookatmycode.Whenyou’redone,youcaneitherkeepusingyourcode,orjustusemineforthenextexercise.

TheSpecificationInthisfirstsmallprogramyou’lllaythefirstfoundationfortheremainingprojects.You’llcallthisprogramstatserve,eventhoughthisspecificationdoesn’tmentionstatisticsoranything.Thatwillcomelater.Thespecificationforthisprojectisverysimple:

1.Createasimplenetworkserverthatacceptsaconnectiononport7899fromnetclientorthenccommand,andthatechoesbackanythingyoutype.

2.You’llneedtolearnhowtobindaport,listenonthesocket,andanswerit.Useyourresearchskillstostudyhowthisisdoneandattempttoimplementityourself.

3.Themoreimportantpartofthisprojectislayingouttheprojectdirectoryfromthec-skeleton,andmakingsureyoucanbuildeverythingandgetitworking.

4.Don’tworryaboutthingslikedaemonsoranythingelse.Yourserverjusthastorunfromthecommandlineandkeeprunning.

Theimportantchallengeforthisprojectisfiguringouthowtocreateasocketserver,buteverythingyou’velearnedsofarmakesthispossible.WatchthefirstlecturevideowhereIteachyouaboutthisifyoufindthatit’stoohardtofigureoutonyourown.

Exercise49.AStatisticsServer

Thenextphaseofyourprojectistoimplementtheveryfirstfeatureofthestatserveserver.YourprogramfromExercise48shouldbeworkingandnotcrashing.Remembertothinkdefensivelyandattempttobreakanddestroyyourprojectasbestyoucanbeforecontinuing.WatchbothExercise48videostoseehowIdothis.Thepurposeofstatserveisforclientstoconnecttoitandsubmitcommandsformodifyingstatistics.Ifyouremember,welearnedalittlebitaboutdoingincrementalbasicstatistics,andyouknowhowtousedatastructureslikehashmaps,dynamicarrays,binarysearchtrees,andternarysearchtrees.Thesearegoingtobeusedinstatservetoimplementthisnextfeature.

SpecificationYouhavetoimplementaprotocolthatyournetworkclientcanusetostorestatistics.IfyourememberfromExercise43,youhavethreesimpleoperationsyoucandotointhestats.hAPI:

createCreateanewstatistic.meanGetthecurrentmeanofastatistic.sampleAddanewsampletoastatistic.dumpGetalloftheelementsofastatistic(sum,sumsq,n,min,andmax).

Thiswillmakethebeginningofyourprotocol,butyou’llneedtodosomemorethings:1.You’llneedtoallowpeopletonamethesestatistics,whichmeansusingoneofthemapstyledatastructurestomapnamestoStatsstructs.

2.You’llneedtoaddtheCRUDstandardoperationsforeachname.CRUDstandsforcreatereadupdatedelete.Currently,thelistofcommandsabovehascreate,mean,anddumpforreading;andsampleforupdating.Youneedadeletecommandnow.

3.Youmayalsoneedtohavealistcommandforlistingoutalloftheavailablestatisticsintheserver.

Giventhatyourstatserveshouldhandleaprotocolthatallowstheaboveoperations,let’screatestatistics,updatetheirsample,deletethem,dumpthem,getthemean,andfinally,listthem.Doyourbesttodesignasimple(andImeansimple)protocolforthisusingplaintext,andseewhatyoucomeupwith.Dothisonpaperfirst,thenwatchthelecturevideoforthisexercisetofindouthowtodesignaprotocolandgetmoreinformationabouttheexercise.Ialsorecommendusingunitteststotestthattheprotocolisparsingseparatelyfromtheserver.Createseparate.cand.hfilesforjustprocessingstringswithprotocolinthem,andthentestthoseuntilyougetthemright.Thiswillmakethingsmucheasierwhenyouaddthisfeaturetoyourserver.

Exercise50.RoutingtheStatistics

Onceyou’vesolvedtheproblemoftheprotocolandputtingstatisticsintoadatastructure,you’llwanttomakethismuchricher.Thisexercisemayrequirethatyouredesignandrefactorsomeofyourcode.That’sonpurpose,asthisisanabsoluterequirementwhenwritingsoftware.Youmustfrequentlythrowoutoldcodetomakeroomfornewcode.Nevergettooattachedtosomethingyou’vewritten.Inthisexercise,you’regoingtousetheURLroutingfromExercise47toaugmentyourprotocol,allowingstatisticstobestoredatarbitraryURLpaths.Thisisallthehelpyouget.It’sasimplerequirementthatyouhavetoattemptonyourown,modifyingyourprotocol,updatingyourdatastructures,andchangingyourcodetomakeitwork.WatchthelecturevideotoseewhatIwant,andthentryyourbestbeforewatchingthesecondvideotoseehowIimplementedit.

Exercise51.StoringtheStatistics

Thenextproblemtosolveishowtostorethestatistics.Thereisanadvantagetohavingthestatisticsinmemory,becauseit’smuchfasterthanstoringthem.Infact,therearelargedatastoragesystemsthatdothisverything,butinourcase,wewantasmallerserverthatcanstoresomeofthedatatoaharddrive.

TheSpecificationForthisexercise,you’lladdtwocommandsforstoringtoandloadingstatisticsfromaharddrive:

storeIfthere’saURL,storeittoaharddrive.loadIftherearetwoURLs,loadthestatisticfromtheharddrivebasedonthefirstURL,andthenputitintothesecondURLthat’sinmemory.

Thismayseemsimple,butyou’llhaveafewbattleswhenimplementingthisfeature:1.IfURLshave/charactersinthem,thenthatconflictswiththefilesystem’suseofslashes.Howwillyousolvethis?

2.IfURLshave/charactersinthem,thensomeonecanuseyourservertooverwritefilesonaharddrivebygivingpathstothem.Howwillyousolvethis?

3.Ifyouchoosetousedeeplynesteddirectories,thentraversingdirectoriestofindfileswillbeveryslow.Whatwillyoudohere?

4.IfyouchoosetouseonedirectoryandhashURLs(oops,Igaveahint),thendirectorieswithtoomanyfilesinthemareslow.Howwillyousolvethis?

5.WhathappenswhensomeoneloadsastatisticfromaharddriveintoaURLthatalreadyexists?6.Howwillsomeonerunningstatserveknowwherethestorageshouldbe?

AnalternativetousingafilesystemtostorethedataisusingsomethinglikeSQLiteandSQL.AnotheroptionistouseasystemlikeGNUdbm(GDBM)tostoretheminasimplerdatabase.Researchallofyouroptionsandwatchthelecturevideo,andthenpickthesimplestoptionandtryit.Takeyourtimefiguringoutthisfeaturebecausethenextexercisewillinvolvefiguringouthowtodestroyyourserver.

Exercise52.HackingandImprovingYourServer

Thefinalexerciseconsistsofthreevideos.Thefirstvideoisalectureonhowtohackyourserverandattempttodestroyit.Inthevideo,Ishowyouagreatmanytoolsandtricksforbreakingprotocols,usingmyownimplementationtodemonstrateflawsinthedesign.Thisvideoisfun,andifyou’vebeenfollowingalongwithyourowncode,youcancompetewithmetoseewhomadethemorerobustserver.ThesecondvideothendemonstrateshowI’daddimprovementstotheserver.Youshouldattemptyourownimprovementsfirst,beforewatchingthisvideo,andthenseeifyourimprovementsaresimilartomine.Thethirdandfinalvideoteachesyouhowtomakefurtherimprovementsanddesigndecisionsintheproject.ItcoverseverythingI’dthinkabouttocompletetheprojectandrefineit.Typically,tocompleteaproject,I’ddothefollowing:

1.Getitonlineandaccessibletopeople.2.Documentitandimprovetheusabilitytomakesurethatthedocumentsareeasytoread.3.Doasmuchtestcoverageaspossible.4.ImproveanycornercasesandadddefensesagainstanyattacksthatIcanfind.

Thesecondvideodemonstrateseachoftheseandexplainshowyoucandothemyourself.

NextSteps

Thisbookismostlikelyamonumentalundertakingforabeginnerprogrammer,orevenaprogrammerwithnoexperiencewithmanyofthetopicscoveredinside.YouhavesuccessfullylearnedanintroductoryamountofknowledgeofC,testing,securecoding,algorithms,datastructures,unittesting,andgeneralappliedproblemsolving.Congratulations.Youshouldbeamuchbetterprogrammernow.IrecommendthatyounowgoreadotherbooksontheCprogramminglanguage.Youcan’tgowrongwithTheCProgrammingLanguage(PrenticeHall1988)byBrianW.KernighanandDennisM.Ritchie,thecreatorsoftheClanguage.Mybookteachesyouaninitial,practicalversionofCthatgetsthejobdone,mostlyasameansofteachingyouothertopics.TheirbookwillteachyoudeeperCasdefinedbythecreatorsandtheCstandard.Ifyouwanttocontinueimprovingasaprogrammer,Irecommendthatyoulearnatleastfourprogramminglanguages.Ifyoualreadyknewonelanguage,andnowyouknowC,thenIrecommendyoutrylearninganyoftheseprogramminglanguagesasyournextones:

•Python,withmybookLearnPythonTheHardWay,ThirdEdition(Addison-Wesley,2014)•Ruby,withmybookLearnRubyTheHardWay,ThirdEdition(Addison-Wesley,2015)•Go,withitslistofdocumentationathttp://golang.org/doc,anotherlanguagebytheauthorsoftheClanguage,andfrankly,amuchbetterone•Lua,whichisaveryfunlanguagethathasadecentAPIforCthatyoumightenjoynow•JavaScript,althoughI’mnotsurewhichbookisbestforthislanguage

Therearemanyprogramminglanguagesavailable,sochoosewhicheverlanguageinterestsyouandlearnit.Irecommendthisbecausetheeasiestwaytobecomeadeptatprogrammingandbuildconfidenceistostrengthenyourabilitytolearnmultiplelanguages.Fourlanguagesseemstobethebreakingpointwhereabeginnertransitionstobeingacapableprogrammer.It’salsojustalotoffun.

http://golang.org/doc

Index

Operators–(minussign)negativenumber(unary),108subtract(binary),108subtractionoperator,21

––(minussigns)decrement,thenread(prefix),108read,thendecrement(postfix),108

()(parentheses)Coperator,23functioncalloperator,108

!(exclamationpoint)BooleanNOToperator,109logicalNOToperator,22

?:(questionmark,colon)Booleanternaryoperator,109logicalternaryoperator,22

.(period)structurereferenceoperator,23structurevalueaccess,108

[](squarebrackets)arrayindex,108arraysubscriptoperator,23

{}(curlybraces)Coperator,23enclosingfunctions,7

*(asterisk)multiplicationoperator,21multiplyoperator(binary),108valueof(unary),108value-ofoperator,23

&(ampersand)addressof(unary),108address-ofoperator,23bitwiseANDoperator,22BooleanANDoperator,109

^(caret)assignXOR-equal,109

bitwiseXORoperator,22^=(caret,equal)assignXOR-equaloperator,23bitwiseXORandassignoperator,110

+(plussign)addoperator(binary),108addoperator(unary),108additionoperator,21

++(plussigns)increment,thenread(prefix),108read,thenincrement(postfix),108

|=(verticalbar,equal)assignor-equal,23bitwiseORandassignoperator,110

,(comma),Coperator,23––(minussigns),decrementoperator,21/(slash),divideoperator,21,108//(slashes),commentindicator,7,24;(semicolon),statementterminator,7:(colon),Coperator,23!=(exclamationpoint,equal),notequaloperator,22,109?=(questionmark,equal),devpkgsyntax,278*/(asteriskslash),multi-linecommentend,6,24/*(slashasterisk),multi-linecommentstart,6,24*=(asterisk,equal),assignmultiply-equal,23,110&&(ampersands),logicalANDoperator,22&=(ampersand,equal),assignand-equal,23,110++(plussigns),incrementoperator,21+=(plussign,equal),assignplus-equal,23,110<(leftanglebracket),lessthanoperator,22,109<<(leftanglebrackets),bitwiseshiftleftoperator,22,109<<=(leftanglebrackets,equal),assignshift-left-equal,23,110<=(leftanglebracket,equal),lessthanorequaloperator,22,109=(equalsign),assignequal,23,110–=(minus,equal),assignminus-equal,23,110/=(slash,equal),assigndivide-equal,23,110==(equalsigns),equalsoperator,22,109|(verticalbar),bitwiseORoperator,22,109||(verticalbars),BooleanORoperator,109~(tilde),complementoperator,109

%(percentsign)modulusoperator,21printingasaliteral,33

%=(percentsign,equal),assignmodulus-equal,23,110

Symbols->(dash,rightanglebracket)structuredereferenceoperator,23structurepointeraccess,108

>(rightanglebracket),greaterthanoperator,22,109>=(rightanglebracket,equal),greaterthanorequaloperator,22,109>>(rightanglebrackets),bitwiseshiftrightoperator,22,109>>=(rightanglebrackets,equal),assignshift-left-equal,23,110'0'(nul)byte,arrayterminator,46–49

AAdler,Mark,240Adler-32function,240–247Alphabeticalcharacters,identifying,60Ampersand(&)addressof(unary),108address-ofoperator,23bitwiseANDoperator,22BooleanANDoperator,109

Ampersand,equal(&=),assignand-equal,23,110Ampersands(&&),logicalANDoperator,22APR(ApachePortableRuntime),274,275–276Arguments,passingtoaprogramGDB,18LLDB,19

Arguments,printing,54Arithmeticoperators,21Arrays’0’(nul)byte,arrayterminator,46–49description,46–49dynamic,198–206indexinginto,65–66multipledimensions,57vs.pointers,67sizing,50–53ofstrings,54–57

Assignmentoperators,23,109–110Asterisk(*)multiplicationoperator,21multiplyoperator(binary),108valueof(unary),108value-ofoperator,23

Asterisk,equal(*=),assignmultiply-equal,23,110Asteriskslash(*/),multi-linecommentend,6,24attachpidcommandGDB,19LLDB,19

Attachingto/detachingfromaprocessGDB,19LLDB,19

autooperator,26Automateeverything,148Automatedtestingdescription,166samplecode,166wiringthetestframework,167–171

BBacktrace,dumpingGDB,18–19LLDB,19

backtracecommand,GDB,18bassignfunction,227bassignblkfunction,227bassigncstrfunction,227bcharfunction,227bconcatfunction,227bdatafunction,227bdestroyfunction,227Bernstein,DanJ.,240BetterStringLibrary,225–227bfindreplacefunction,227bformatfunction,227bfromcstrfunction,227Binarysearch,211–220Binarysearchtrees,260–273.SeealsoHashmaps;Ternarysearchtrees.binstrfunction,227

biseqfunction,227Bitoperators,109Bitwiseoperators,22,109Blanks,detecting,60blengthfunction,227blk2bstrfunction,227BMH(Boyer-Moore-Horspool)algorithm,248–257Booleanexpressionsswitchstatements,42–44while-loopstatements,40–41

Booleanoperators,109.SeealsoTrue/falsebranching.breakcommandflowcontrol,110GDB,18

breakoperator,26breakpointsetcommand,LLDB,19Breakpoints,GDBclearing,19setting,18showinginformationabout,19

Breakpoints,LLDBclearing,19setting,19showinginformationabout,19

bsplitfunction,227bstrcpyfunction,227BSTree,260–273.SeealsoTSTree.bstricmpfunction,227bstrliblibrary,225–227bstrlib.ofile,278Bubblesort,190–197Buildingcode.Seealsomakecommand;Makefile.inGDB,18inLLDB,19

CClanguage,compilerscheckingversion,2confirming,6–8errorhandling,90–91samplecode,6–8

supportunderWindows,3Clanguage,operators.Seealsospecificoperators.arithmetic,21assignment,23,109–110bitwise,22,109Boolean,109data,23,108logical,22,109math,108memorizing,20–21relational,22

Clanguage,syntaxkeywords,26–27lexemes,26–29lexicalanalysis,26–29memorizing,26–29syntaxstructures,27–30

Cpreprocessor(CPP)conditionallycompilingcode,98expandingmacros,96–98

CUB(commonundefinedbehavior)definition,xvdescription,172–173top20undefinedbehaviors,174–177

Cunions,212–213Caret(^)assignXOR-equal,109bitwiseXORoperator,22

Caret,equal(^=)assignXOR-equaloperator,23bitwiseXORandassignoperator,110

caseoperator,26cc-Wall-g-DNDEBUG-ldlex29.c-oex29command,164cdcommandGDB,18LLDB,19

ChangingdirectoryGDB,18LLDB,19

charoperator,26,104Characterdatatypes,104Charactertypedata,defining,26

Characters,detecting,60Clang’sGettingStartedinstructions,2clearcommand,19Clearingbreakpoints,19Code,building.Seemakecommand;Makefile.Colon(:),Coperator,23Comma(,),Coperator,23Commandlinearguments,printing,54commands.cfile,288–291commands.hfile,287–288Comparingstrings,227Compilers,Clanguage.SeeClanguage,compilers.Concatenatingstrings,227constoperator,26constqualifier,105continuecommandflowcontrol,110GDB,18LLDB,19

continueoperator,27ContinuerunningtheprogramGDB,18LLDB,19

Controlstructures,110Copyingstrings,227Countingstatisticalsamples,300,340–341CPP(Cpreprocessor)conditionallycompilingcode,98expandingmacros,96–98

Creatingdoublelinkedlistlibraries,178–179strings,227variables,32–34

Creating,datatypesfrommultiplevariables,27,32–34fornewtypes,29asstructs,30

Creativeprogrammermind-set,140–141Curlybraces({})Coperator,23enclosingfunctions,7

Cygwinsystem,3

DDArrayprogram,198–206Dash,rightanglebracket(->)structuredereferenceoperator,23structurepointeraccess,108

Dataoperators,23,108Datasize,determining,27Datastructuresdefinition,178fuzzing,272testing,272

Datatypes.Seealsospecificdatatypes.character,26,104combiningintoasinglerecord.SeeStructs.conversion,105declareempty,27doublefloatingpoint,27,104enumeratedtypes,104floatingpoint,27,104modifiers,104–107qualifiers,105sizes,106–107void,104

Datatypes,andflowcontrolassignmentoperators,109–110bitoperators,109Booleanoperators,109controlstructures,110dataoperators,108logicoperators,109mathoperators,108operators,107–110typeconversion,105typemodifiers,104–107typequalifiers,105typesizes,106–107types,104–107

Datatypes,creatingfrommultiplevariables,27,32–34fornewtypes,29

asstructs,30Datatypes,integerdeclaring,27integerconstants,29,104shortinteger,27signedmodifierfor,27unsignedmodifierfor,27

DBfunctions,devpkgprogram,279–287db.cfile,280–282dbg.hmacro,91–95db.hfile,280db.ofile,278Debugmacros,91–95Debugprintingvs.GDB,100–102Debugging.SeealsoGDB(GNUDebugger);LLDBDebugger.advancedtechniques,100–102avoidingstackbugs,118–119strategy,101–102withvarargfunctions,132–136

defaultoperator,27Defensiveprogrammermind-set,141Defensivestrategiesautomateeverything,148documentassumptions,147failearlyandopenly,146importanceoforder,149nevertrustinput,142–145overview,141–142preventerrors,145–146preventionoverdocumentation,147–148questionauthority,149simplifyandclarify,148–149

Destroyingstrings,227detachcommand,19Detachingfrom/attachingtoaprocessGDB,19LLDB,19

devpkgfile,279devpkgprogram?=(questionmark,equal),278apr_off_terror,278

apr_thread_proc.hfunctions,283checkingforinstalledURLs,279–287commandfunctions,287–288commands,274commands.cfile,288–291commands.hfile,287–288DBfunctions,279–287dependencies,277description,274devpkg.cfile,292–294externaltools,283Mainfunction,292–294Makefile,277–278off64_terror,278READMEfile,294recordingandlistinginstalledURLs,279–287shellfunctions,283–287shell.cfile,284–286Shell_execfile,287shell.hfile,283–284Shell.runfunction,286sourcefiles,278–279

devpkg.cfile,292–294Dictionaries.SeeHashmaps.DJBHashfunction,240–247dooperator,27Documentassumptions,147Doublefloatingpointdatatypesdeclaring,27description,104

Doublelinkedlistscreatingalibrary,178–179datastructures,definition,178description,179–181implementing,181–185testing,185–187

doubleoperator,27,104do-whileloopexample,29flowcontrol,110starting,27

Duff,Tom,120Duff’sdevice,120–125DumpingabacktraceGDB,18–19LLDB,19

Dynamicarrays,198–206Dynamiclibraries,160

Eelseoperator,27else-ifstatement,36–38else-statement,36–38Emacstexteditor,4enumoperator,27,29,104Enumerateddatatypes,104Equalsign(=),assignequal,23,110Equalsigns(==),equalsoperator,22,109Equalitytesting.SeeLogicoperators.Errorcodes,90–91Errorhandling.SeeClanguage,errorhandling.ex22.cfile,112–114ex22.hfile,112–114ex22_main.cfile,114–118Exclamationpoint(!)BooleanNOToperator,109logicalNOToperator,22

Exclamationpoint,equal(!=),notequaloperator,22,109Exitoutofacompoundstatement,26externoperator,27

FFailearlyandopenly,146fclosefunction,129fcloseallfunction,130fdopenfunction,129fgetposfunction,130fgetsfunction,126–130Findandreplacestrings,227floatoperator,27,104Floatingpointdatatypesdeclaring,27

description,104FNV-1afunction,240–247fopenfunction,129foroperator,27for-loopsarraysofstrings,54–57example,29starting,27

Formattedprinting,14–16Formattingstrings,227Fowler,Glenn,240fprintffunction,130freadfunction,130freopenfunction,129fscanffunction,126–130fseekfunction,130ftellfunction,130Functioncalls,steppingintoGDB,18LLDB,19

Functioncalls,steppingoverGDB,18LLDB,19

Functions.Seealsospecificfunctions.bad,checkingfor,158–159defining,29I/Ohandling,126–130returningfrom,27using,58–60vararg,132–136withvariablearguments,132–136writing,58–60

Functions,pointerstodescription,84–85format,84samplecode,84–85

Fuzzingdatastructures,272fwritefunction,130

GGDB(GNUDebugger).SeealsoLLDBDebugger.

attachingto/detachingfromarunningprocess,19buildcode,18changedirectory,18continuerunningtheprogram,18vs.debugprinting,100–102dumpingabacktrace,18–19help,18listtensourcelines,19passingargumentstotheprogram,18quit,18startaprogram,18startashell,19stepintofunctioncalls,18stepoverfunctioncalls,18watchpoints,showinginformationabout,19

GDB(GNUDebugger),breakpointsclearing,19setting,18showinginformationabout,19

GDB(GNUDebugger),commandsattachpid,19backtrace,18break,18cd,18clear,19continue,18detach,19help,18infobreak,19infowatch,19list,19make,18next,18printexpr,18pwd,18quickreference,18–19quit,18run,18shell,19step,18

GDB(GNUDebugger),printing

valueofanexpression,18workingdirectory,18

gedittexteditor,3Gettingstarted.SeeSettingupyourcomputer.GNUDebugger(GDB).SeeGDB(GNUDebugger).gotooperator,27,29,110

HHashmapsAdler-32function,240–247algorithms,240–247vs.binaryorternarysearchtrees,323definition,228DJBHashfunction,240–247example,228–235findingprefixes,327FNV-1afunction,240–247scanning,235unittesting,235–237

Hashmap_traversefunction,235Headers,Makefileexample,154–155Heapsort,208–210Heapspotentialproblems,81samplecode,74–79vs.stackmemoryallocation,74–79

HelpGDB,18LLDB,19

helpcommandGDB,18LLDB,19

IIDE(IntegratedDevelopmentEnvironment),4Identifiers,declaringasexternal,27ifoperator,27if-statementelsebranch,27example,28starting,27true/falsebranching,36–38

Indexingintoarrays,65–66Inequalitytesting.SeeLogicoperators.infobreakcommand,19infowatchcommand,19Input/output.SeeI/O.InstallingaMakefile,example,158software.Seedevpkgprogram.

Intconstants,definingasetof,27intoperator,27,104int8_ttypesize,106int16_ttypesize,106int32_ttypesize,106int64_ttypesize,106Integerconstants,datatypesfor,104Integerdatatypedeclaring,27signedmodifierfor,27unsignedmodifierfor,27

IntegratedDevelopmentEnvironment(IDE),4INT_FAST(N)_MAXtypesize,107int_fast(N)_ttypesize,107INT_FAST(N)_MAXtypesize,107int_fast(N)_ttypesize,107INT_LEAST(N)_MAXtypesize,107INT_LEAST(N)_MINtypesize,107int_least(N)_ttypesize,107INT_LEAST(N)_MAXtypesize,107INT_LEAST(N)_MINtypesize,107int_least(N)_ttypesize,107INTMAX_MAXtypesize,107INTMAX_MINtypesize,107intmax_ttypesize,107INTPTR_MAXtypesize,107INTPTR_MINtypesize,107intptr_ttypesize,107I/O,readingfromfiles,126–130isalphafunction,60isblankfunction,60

JJumptables,42–44Jumpingtoalabel,27,29

KKeywords,C,26–27

LLeftanglebracket,equal(<=),lessthanorequaloperator,22,109Leftanglebracket(<),lessthanoperator,22,109Leftanglebrackets,equal(<<=),assignshift-left-equal,23,110Leftanglebrackets(<<),bitwiseshiftleftoperator,22,109Lengthofstrings,getting,227Letters,identifying,60Lexemes,Csyntax,26–29Lexicalanalysis,Csyntax,26–29LibrariesBetterStringLibrary,225–227bstrlib,225–227doublelinkedlists,178–179dynamic,160linkingto,160–164shared,dynamicloading,161–164static,160

Linkedlistalgorithms,sortingwith,190–197Linkingtolibraries,160–164Linuxcompilerversion,checking,2runningunderWindows,3settingupyourcomputer,2

listcommand,19ListnexttensourcelinesinGDBandLLDB,19list_algos.cfile,193–195list_algos.hfile,193list_algos_tests.ffile,191–193LLDBDebugger.SeealsoGDB(GNUDebugger).attachingto/detachingfromaprocess,19buildingcode,19changedirectory,19continuerunningtheprogram,19dumpingabacktrace,19

help,19listingnexttenlines,19quitting,19startingashell,19startingtheprogram,19steppingintofunctioncalls,19steppingoverfunctioncalls,19watchpoints,showinginformationabout,19

LLDBDebugger,breakpointsclearing,19setting,19showinginformationabout,19

LLDBDebugger,commandsattachpid,19breakpointset,19cd,19clear,19continue,19detach,19help,19infobreak,19infowatch,19list,19make,19next,19printexpr,19pwd,19quickreference,19quit,19runcommand,19shell,19step,19threadbacktrace,19

LLDBDebugger,printingexpressions,19workingdirectory,19

loadcommand,344Localvariables,givingalocallifetime,26Logfiles,finding,138Logfindproject,138

Logicoperators,22,109longmodifier,104Loops.Seealsospecificloops.breakingtoexit,28continuingtothetopof,27infinite,144

MMacOSXcompilerversion,checking,2settingupyourcomputer,2–3

Macrosdbg.h,91–95fordebugging,91–95expanding,96–98

MacVimtexteditor,4Mainfunction,devpkgprogram,292–294makecleancommand,10–12makecommandbuildingcode,10–12GDB,18LLDB,19

Makefileasautomationtool,11buildingcode,10–12devpkgprogram,277–278

Makefile,examplesbasicstructure,152–154checkingforbadfunctions,158–159cleanup,157–158header,154–155installing,158targetbuild,155–156unittests,156–157

Mathoperators,108Max/minsamples,identifying,300Mean,calculating,300,340–341MemorizingCoperators,20–21Csyntax,26–29

Memoryformatconversion,212–213

leaks,shownbythedebugger,80stackallocation,80–81

Mergesort,190–197,208–210Middlenumber,calculating,300,340–341Mind-setforprogramming.SeeProgrammermind-set.MinGwsystem,3Min/maxsamples,identifying,300,340–341Minussign(-)negativenumber(unary),108subtract(binary),108subtractionoperator,21

Minussign,equal(–=),assignminus-equal,23,110Minussigns(––)decrement,thenread(prefix),108read,thendecrement(postfix),108

Minussigns(––),decrementoperator,21

NNanotexteditor,4netclient.cfile,316–320Networkserverprogram,338Nevertrustinput,142–145nextcommandGDB,18LLDB,19

Noll,LeonardCurt,240Nulbyte,arrayterminator,46–49

Ooff64_terror,278Operators.SeeClanguage,operators.Output.SeeI/O.

PPercentsign(%)modulusoperator,21printingasaliteral,33

Percentsign,equal(%=),assignmodulus-equal,23,110Period(.)structurereferenceoperator,23structurevalueaccess,108

Plussign(+)

addoperator(binary),108addoperator(unary),108additionoperator,21

Plussign,equal(+=),assignplus-equal,23,110Plussigns(++)increment,thenread(prefix),108read,thenincrement(postfix),108

Plussigns(++),incrementoperator,21Pointersvs.arrays,67definition,65description,65indexingintoarrays,65–66lexiconof,66–67samplecode,62–64tostructures,68–71usesfor,66

Pointers,tofunctionsdescription,84–85format,84samplecode,84–85

Preventerrors,145–146Preventionoverdocumentation,147–148printexprcommandGDB,18LLDB,19

printffunction,14–16Printing%(percentsigns),asliterals,33commandlinearguments,54expressionvalues,18,19formatting,14–16fromGDB,18fromLLDB,19inscientificnotation,33workingdirectory,18,19

Programmermind-setcreative,140–141defensive,141

Programmermind-set,defensivestrategiesautomateeverything,148documentassumptions,147

failearlyandopenly,146importanceoforder,149nevertrustinput,142–145overview,141–142preventerrors,145–146preventionoverdocumentation,147–148questionauthority,149simplifyandclarify,148–149

PTRDIFF_MAXtypesize,107PTRDIFF_MINtypesize,107pwdcommandGDB,18LLDB,19

QQuestionauthority,149Questionmark,colon(?:)Booleanternaryoperator,109logicalternaryoperator,22

Questionmark,equal(?=),devpkgsyntax,278

Queues,296–299Quicksort,208–210quitcommandGDB,18LLDB,19

QuittingGDB,18LLDB,19

RRadixsort,211–221RadixMap_addfunction,219RadixMap_createfunction,219RadixMap_deletefunction,219RadixMap_destroyfunction,219RadixMap_findfunction,219–220RadixMap_sortfunction,219,220–221radix_sortfunction,219,220–221Readingfromfiles,126–130READMEfile,devpkgprogram,294

registeroperator,27registerqualifier,105Relationaloperators,22Returnfromafunction,27returnoperator,27rewindfunction,130Rightanglebracket,equal(>=),greaterthanorequaloperator,22,109Rightanglebracket(>),greaterthanoperator,22,109Rightanglebrackets,equal(>>=),assignshift-left-equal,23,110Rightanglebrackets(>>),bitwiseshiftrightoperator,22,109Ringbuffers,310–314runcommandGDB,18LLDB,19

SScanninghashmaps,235Scopeex22.cfile,112–114ex22.hfile,112–114ex22_main.cfile,114–118andstacks,118–119

Searchalgorithms,248–257Searchingbinarysearch,211–220binarysearchtrees,260–273ternarysearchtrees,322–330

Semicolon(;),statementterminator,7Serverimprovements,346Serverlogintimes,summarystatistics,307Settingbreakpoints,GDB,18breakpoints,LLDB,19strings,227

SettingupyourcomputerClang’sGettingStartedinstructions,2Linux,2MacOSX,2–3texteditors,3–4.Seealsospecifictexteditors.Windows,3

Sharedlibraries,dynamicloading,161–164

Shaw,ZedA.,contactinformation,xvshellcommand,19shell.cfile,284–286Shell_execfile,287shell.hfile,283–284shell.ofile,278Shell.runfunction,286Shells,starting,19Shortintegerdatatype,declaring,27shortmodifier,104shortoperator,27Showinginformationaboutbreakpoints,19signedmodifierdescription,104forintegerdatatypes,27

Simplifyandclarify,148–149SIZE_MAXtypesize,107sizeofoperatordataaccess,108description,23,27sizingarrays,50–53

Slash,equal(/=),assigndivide-equal,23,110Slash(/),divideoperator,21,108Slashasterisk(/*),multi-linecommentstart,6,24Slashes(//),commentindicator,7,24Sortingbubblesort,190–197heapsort,208–210withlinkedlistalgorithms,190–197mergesort,190–197,208–210quicksort,208–210radixsort,211–221statistics,344

Splittingstrings,227Squarebrackets([])arrayindex,108arraysubscriptoperator,23

Squaresofnumbers,calculating,300Stacksavoidingbugs,118–119definition,81

description,296–299memoryallocation,80–81potentialproblems,81

Standarddeviation,calculating,300–304StartingaprogramwithargumentsGDB,18LLDB,19

Startingashell,19Staticlibraries,160staticoperator,27Statistics.SeeSummarystatistics.Statisticsserver,340–341stats.cfile,302–304Stats_createfunction,304Stats_dumpfunction,304statserveprogram,338,340–341stats.hAPI,340–341Stats_meanfunction,304Stats_recreatefunction,304Stats_samplefunction,304Stats_stddevfunction,304stats_tests.cfile,304–306stepcommandGDB,18LLDB,19

SteppingintofunctioncallsGDB,18LLDB,19

SteppingoverfunctioncallsGDB,18LLDB,19

storecommand,344String_base_searchfunction,252–255String_findfunction,249–255,257Stringsarraysof,54–57BetterStringLibrary,225–227BMH(Boyer-Moore-Horspool)algorithm,248–257checkingforvalidity,224–225comparing,227concatenating,227

copying,227creating,227destroying,227disadvantagesof,224–225findandreplace,227formatting,227functionsfor,227gettingcharactersfrom,227gettingdatafrom,227gettinglengthof,227searchalgorithms,248–257setting,227splitting,227storingasarrays,46–49testingforequality,227

StringScanner_scanfunction,249–255,257String_setup_skip_charsfunction,252–255structoperator,27,30Structs,68–71Sum,calculating,300,340–341Sumofsquares,calculating,340–341Summarystatisticscountingsamples,300,340–341loadcommand,344loadingfromaharddrive,344mean,calculating,300,340–341middlenumber,calculating,300,340–341min/maxsamples,300,340–341routing,342forserverlogintimes,307sorting,344standarddeviation,calculating,300–304statisticsonstatistics,306–307statisticsserver,340–341stats.cfile,302–304Stats_createfunction,304Stats_dumpfunction,304stats.hAPI,340–341Stats_meanfunction,304Stats_recreatefunction,304Stats_samplefunction,304

Stats_stddevfunction,304stats_tests.cfile,304–306storecommand,344storingtoaharddrive,344sum,calculating,300,340–341sumofsquares,calculating,300,340–341unittest,304

switchoperator,27switch-statementsbranchingina,26defaultbranch,27description,42–44example,28starting,27

Syntaxstructures,Csyntax,27–30

TTCP/IPclient,316–321Ternarysearchtrees,322–330.SeealsoBinarysearchtrees;TSTree.Testingautomated.SeeAutomatedtesting.datastructures,272doublelinkedlists,185–187stringsforequality,227

Texteditors,3–4.Seealsospecifictexteditors.TextWranglertexteditor,3threadbacktracecommand,LLDB,19Tilde(~),complementoperator,109True/falsebranching,36–38TSTree.SeealsoBSTree.fastURLrouting,332–336searchingwith,322–330

typedefoperator,27,30

UUB(undefinedbehavior).SeeCUB(commonundefinedbehavior).uint8_ttypesize,106uint16_ttypesize,106uint32_ttypesize,106uint64_ttypesize,106UINT_FAST(N)_MAXtypesize,107

uint_fast(N)_ttypesize,107UINT_FAST(N)_MAXtypesize,107uint_fast(N)_ttypesize,107UINT_LEAST(N)_MAXtypesize,107uint_least(N)_ttypesize,107UINT_LEAST(N)_MAXtypesize,107uint_least(N)_ttypesize,107UINTMAX_MAXtypesize,107uintmax_ttypesize,107UINTPTR_MINtypesize,107uintptr_ttypesize,107unionoperator,27,30Unions,212–213union-statement,starting,27,30unsignedoperator,27,104URLrouting,332–336Urlortool,332–336

VValiditycheckingstrings,224–225Variablescombiningintoasinglerecord,27.SeealsoStructs.creating,32–34declaringasmodifiable,27declaringtobestoredinaCPUregister,27makeunmodifiable,26preservingvalueafterscopeexits,27

Verticalbar,equal(|=)assignor-equal,23bitwiseORandassignoperator,110

Verticalbar(|),bitwiseORoperator,22,109Verticalbars(||),BooleanORoperator,109Vimtexteditor,4VirtualBox,3Vo,Phong,240Voiddatatypes,104voidoperator,27,104volatileoperator,27volatiletypequalifier,105

W

Watchpoints,showinginformationaboutGDBandLLDB,19whileoperator,27while-loopBooleanexpressions,40–41example,28starting,27

WindowsCsupport,3runningLinuxunder,3settingupyourcomputer,3

WherearetheCompanionContentFiles?

RegisterthisdigitalversionofLearnCtheHardWaytoaccessimportantdownloads.Registerthisdigitalversiontounlockthecompanionfilesthatareincludedonthediscthataccompaniestheprintedition.Followthestepsbelow.

1.Gotohttp://www.informit.com/registerandloginorcreateanewaccount.2.EnterthisISBN:9780321884923NOTE:ThisistheISBNoftheprintbookwhichmustbeusedtoregistertheeBookedition.

3.Answerthechallengequestionasproofofpurchase.4.Clickonthe“AccessBonusContent”linkinthe“RegisteredProducts”sectionofyouraccountpage,whichwilltakeyoutothepagewhereyourdownloadablecontentisavailable.

TheProfessionalandPersonalTechnologyBrandsofPearson

http://www.informit.com/register

CodeSnippets

Learn C the Hard Way

Documents

Transcript of Learn C the Hard Way