COMP61511(Fall2018)SoftwareEngineeringConcepts

InPracticeWeek3

BijanParsia&ChristosKotselidis< , @manchester.ac.uk>

(bugreportswelcome!)bijan.parsia christos.kotselidis

ReflectingOnPersonalQualities

3Of5We'reatWeek3!

3/5sdoneaftertoday!Whereareyou?

Andwhereareyougoing?Whatarethenextsteps?

Thisisagoodtimetoreflect

ReflectionReflection is the process of examining one own thoughts,beliefs, experiences, concepts, etc. in order to gain self-knowledgeandinsight

Refectiondoesn'tneedtobejudgementalYouaren'tlookingforflawsYouaretryingtounderstandyourselfThisincludesgoodthings!

Maybetheycangetbetter

ReflectionExample1ForCW1somepeoplehandedin

arararchive(e.g.,mbassbp2_cw1.rar)Thisisinspite

mymentioningitinlectureitbeingdescribedintheassignmentLab0andLab1havingthesamerequirment

AndbeinggradedandreturnedbeforeCW1wasdue!Whatwentwrongwiththeirprocess?Howtofixit?

ReflectionExample2LastThursafternoon

severalpeopleassertedthattheircode100%shouldpasssomesimpletestsandthattheircodereviewfound"nobugs"

Forsome,IfoundtheycouldnotpassanytestForothers,Ilookedatthesourcecodeandimmediatelyspied2-3"obvious"bugsWhatwentwrong?

Andhowdowefixit

ReflectionExample3Lookonthediscussionboard!

Alotofmyrepliesarealongthelines:"Ontheonehand,youhavewhatwcdoes.Ontheother,youhaveaideaofhowtodoitdifferently.Whichdoyouthinkshouldwin?"

Andthatwasenough!Howtogetthatwithoutanexternalprompt?

ReflectionExample4Peopleasked:

Ifixedmyminiwc.pywillyouretestittocheckthatit'srightIwanttoknowthatmyinterpretationisrightIjustwanttoconfirmmyunderstandingsoIdon'tlosepoints

Reflect!Doweunderstandwhypeopleaskedthese?What'stherightresponse?What'swouldbeaproblemwithansweringthesedirectly?

Arepointsthepointofthecoursework?

ReflectionExample4Lookonthediscussionboard!

AlotofpeoplegotstuckHowandwhy?Didyougetstuck?Didyougetunstuck?

How?

Stuck?

BijanicReflection!Notsobad,actually!

Aremygoalscorrect?Aremymethodseffective?

Whatwouldprovemewrong?Isitworththeeffort?

I'mprettytired!

How'dIdo?

MetacognitionMetacognitionisthinkingaboutthinking

ReflectionisoneexampleIngeneral,animportantskill

Forexample,whenyouarestuckonaproblemithelpstocheckwhetheryouareinarutthatis,justtryingthesamethingoverandover

Beingawarethatyougotstuckcanhelpyougetunstuck!

CognitiveBiasesAcognitivebiasisasystematicdeparturefromrationality.

Weallhavethem,and ofthem:lots

CognitiveBiases

Self-EfficacyPerceivedself-efficacyisdefinedaspeople'sbeliefsabouttheircapabilities to produce designated levels of performance thatexerciseinfluenceovereventsthataffecttheirlives.

Toomuchself-efficacyisoverconfidenceisrelatedtoDunning-Krugar

Toolittleself-efficacyparalysesyouleadstounderachievement

TwoKeyBiases"Thetendencytoseeoneselfaslessbiasedthanotherpeople,ortobeabletoidentifymorecognitivebiasesinothersthaninoneself."

"Thetendencyforunskilledindividualstooverestimatetheirownabilityandthetendencyforexpertstounderestimatetheirownability."

Bias-BlindSpot

Dunning-KrugarEffect

Beverycarefulhere!

GoldilocksSelf-EfficacyBothtoomuchandtoolittlearebad!

Toomuch==boredToolittle==dauntedanduninterested

Aimforthesweetspot!

GoldilocksScience!

Awesomestpapertitle:"Self-efficacyandinterest:Experimentalstudiesofoptimalincompetence"

Trajectory!Trajectoryovercurrentlevel

CurrentlevelisstaticItinformstrajectory

Butdoesn'tdetermineitReflection!

AreyoulearningquicklyorslowlyAreyoulearninghowtolearn

AGoalA student who has mastered the [Core Body of Knowledge(CBOK) will be able to develop a modest-sized softwaresystemofafewthousandlinesofcodefromscratch,beabletomodify a pre-existing large-scale software systemexceeding 1,000,000 lines of code, and be able to integratethird-partycomponentsthatarethemselvesthousandsof lines of code. Development and modification includeanalysis,design,andverification,andshouldyieldhigh-qualityartefacts,includingthefinalsoftwareproduct.

IEEE&ACMACM2009.

SoftwareEngineeringCurriculumRecommendations

AStudentWill...beabletodevelop

amodest-sizedsoftwaresystemofafewthousandlinesofcodefromscratch,

beabletomodifya...large-scalesoftwaresystemexceeding1,000,000linesofcode,andbeabletointegrate(1000sLOC)third-partycomponents

Developmentandmodificationincludeanalysis,design,andverification,andshouldyieldhigh-qualityartefacts,includingthefinalsoftwareproduct.

Wc?Wheredoeswc.pygetus?

Foraproperclone≈hundredsofLOCOTOH,maybeunder100!wcgolf?

Withextensionsmaybe1000s?

NotcountinginfrastructureTests,etc.

Does100spredict1000s?Goodquestion!

LookAround!Modestsizesoftwaresystems?

Whatdotheylooklike?Whatdotheydo?Collectsomeexamples!

RememberreverseengineeringPortfromadifferentlanguage!Rewritefromscratch

Createsomethingnew!

SomeInternalQualities

SoftwareQualityLandscape20.1.CharacteristicsofSoftwareQuality,CodeComplete

ThusFarWeLookedAt...External

FunctionalCorrectness(thefunctionalquality)

Non-functional(abit)Efficiency(thenon-functionalquality)

Now,someinternalTestabilityForModification

Maintainability

Internal:ForModificationMaintainability

easeto"changeoraddcapabilities,improveperformance,orcorrectdefects"

Flexibilityeasetomodifyfornewsituations("internal"versionofadaptability)

Portabilityeasetomodifyfornewenvironments

Reusabilityeasetoextractpartsforuseinothersystems

Internal:ForComprehensionReadability

easeofcomprehendingthesourcecode,espatthestatementlevelUnderstandability

easeofcomprehendingthesoftwaresystemasawholefromthesynoptic("bird'seye")viewtothemyopic("worm'seye")view

Readabilityispartofunderstandability.Butyoucanhavereadablemethodsorfunctionsandanimpossibletograsparchitecture.

Internal:TestabilityAcriticalproperty!

RelativetoatargetqualityAsystemcouldbe

highlytestableforcorrectensslowlytestableforefficiency

PartlydeterminedbytestinfrastructureHavinggreathooksfortestspointlesswithouttests

PracticallyspeakingLowtestabilityblocksknowingqualitiesTest-basedevidenceisessential

ProblemIndicators"asurfaceindicationthatusuallycorrespondstoadeeperproblem"(KentBeckvia )Quicktospot(ifyouhaveexperience)Doesn'talwayscorrespondtoaproblemSomewhatsubjectiveThe

PainPointsApartofthesystemthatrecurrentlycausesproblems

HardtouseRevistoften

CodeSmell

MartinFowler

"WTFtest"

TestabilitySmell def get_file_list(): # Get list of arguments from the command line, minus "wc.py" args_list = sys.argv[1:] ...

or def get_max_width(): max_val_list = []

for rec in file_log: max_val_list.append(rec.get_max_value())

return max(max_val_list)

Thankstothebravestudentwhovolunteeredtheircode!

TestabilitySmellFIXED def get_file_list(**args**): # Get list of arguments from the command line, minus "wc.py" args_list = args[1:] ...

sowecantestby: >>> import wc >>> wc.get_file_list(['wc.py', '-l', 'filename.txt'])

Thankstothebravestudentwhovolunteeredtheircode!

TestabilitySmell2Whatabout:

#all the module's code wc()

Thankstothebravestudentwhovolunteeredtheircode!

TestabilitySmell2FIXEDWewanttoimportthemodulewithoutrunninganything!

#all the module's code if __name__ = "__main__": wc()

Now,import wcdoesn'trunwc()

Thankstothebravestudentwhovolunteeredtheircode!

RefactoringNotice

NoneofthesemoveschangedfunctionalityOrprettymuchanyexternalquality

Butweimprovedtestabilitymaybereadabilityandmaintainabilityreusability!

Werefactoredthecode

WhyDidTheseThingsHappen?HasteLackofunderstanding

OftheproblemOfthetools

E.g.,PythonSomeoneElse'sCodeJustanaccident!

Seemedlikeagoodideaatthetime!

AConsequenceRemembertheTestableFunction'stests

Whatdotestslooklikefor:

>>> import wc>>> wc.get_file_list(['wc.py', '-l', 'filename.txt'])

def get_file_list(): # Get list of arguments from the command line, minus "wc.py" args_list = sys.argv[1:] ...

TestConsequencesMaybe

or

Aretheseeasytomaintain?

>>> import subprocess>>> subprocess.check_output('python wc -l filename.txt')

>>> import sys, wc>>> sys.argv = ['wc.py', '-l', 'filename.txt']>>> wc.get_file_list()

TechnicalDebtOurlackoftestabilityislikeadebt

WemighthavetopayitoffinthefutureByrefactoring

IfwewaititincursinterestThemoretestswewrite,themoretestswehavetochangelaterOurcurrenttestsare

HardtowriteHardtoreadMaybebuggy!

Debtbreedsmoredebt!

ABitOnGitSlides

TechnicalDebt

TechnicalDebt is "the obligations incurred by a software

organization when it chooses an expedient design orconstruction approach that increases complexity and is morecostlyinthelongterm."

Technical debt

Typically,lower(internal)qualitylevelItmaybuyanexternalqualityeffect

Morefunctionality(correctness)Moreefficiency

ItmayhavenegativeexternaleffectsItmayjustbuyprojecteffects

E.g.,developereffort

DebtTaxonomy

IntentionalVsUnintentionalDebtUnintentionaldebt==accidentalorincidental

Wemightnotknowweincurredit!Wemightnotknowtheinterest!Resultsofpoorpractice

Seethetestabilityissuesfromthismorning!Intentionaldebt==deliberate,knowinglyincurred

NeedsanidentifiablerationaleWithascope

Ifyoudon'tknowthescope,it'sprobablynot(fully)intentional

WhyGoIntoDebt?2.AShort-TermDebt

Tacticalreasons2.A.1"Big"Debt

Significantshortcuts2.A.2"Little"(individual)Debt

Tinyshortcuts2.BLong-TermDebt

Strategicreasons

PayingDownDebtDebtcanbecomeunmanageable

EvenmanageabledebtcanbecostlyPayingdowndebtcosts

Debtshiftscoststothefuture(Butmightaddsomecostsnow)

RefactoringistheusualapproachButalsothingslikeaddingtests

Doyoualwayshavetopaydownyourdebt?

GoodDebtVs.BadDebtGooddebt

HasaclearbenefitIsworththecostIsmanageable

BaddebtSkewedcost/benefitratioLessorun-manageable

Debtcan"spoil"Toomuchgooddebtcanbecomebad

ProgramEquivalence

ManyEquivalencesSourcecodeequivalent

CharacterequivalentASTequivalentNon-comment/namesASTequivalent

TranslationequivalentE.g.,aftercompilation

All-behaviorequivalentBisimilarFunctionallyequivalent

FunctionallyEquivalentTwoprogramsarefunctionallyequivalent just incase theyimplementexactlythesamefunctionality.

Functionalityistypicallycharacterisedby"Input-Output"behaviourInternalstructuredoesn'tmatter

Programminglanguage,algorithm,etc.FEPscandiffer"solely"inexecutionpaths

Therecanbebehaviourdifferences(e.g.,performance!)Strongbutnotmaximallystrong!

WhatBehaviourIs"Functionality"?The functionality of a software system is the requiredbehaviourofthesystem

Notideal,asnon-functionalbehaviourmayberequired

Thefunctionalityofasoftwaresystemisthebehaviourofthesystemthatperformssomeusertask

Ineithercause,thefunctionalityisasubsetofallbehaviour

FunctionalityEquivalent(Reprise)Twoprogramsarefunctionallyequivalent just incase theyimplementexactlythesamefunctionality

The functionality of a program are those behaviours whichperformsausertask

Functionalitymaybechanging,unknown,ormisunderstood

Thesetoffunctionallyequivalentprogramsdependsonthefunctionalityparameter

FizzBuzzExampleComparea"normal"FizzBuzzsolutionwithagolfversion:

FunctionallyEquivalentGivenasetofrequiredfunctionalitiesF,andtwosystems,S1&S2,S1isfunctionalityequivalent(withrespecttoF)toS2ifS1andS2enactF.

So,iftwoprogramsarebehaviourlyequivalentthentheyarefunctionallyequivalent.

WhathappensifS1andS2don'tquiteenactthesameF?

"Sufficiently"FunctionallyEquivalentGivenasetofrequiredfunctionalitiesF,andtwosystems,S1 & S2, which enact functionality sets F1 & F2 (respectively),where, F1 ≠ F2 ≠ F, S1 is sufficiently functionallyequivalent to S2 wrt F if F1 and F2 share "enough" of anoverlapwithF.

Obviously,"enough"isaparameter!

WcExample!GNUwchasmorefunctionality(anduser-notablebehaviour)thanminiwc.py

Orotherwcs!Differentflagoptions,findlongestline,etc.

Somebehaviourisuservisiblebutnot"functional"(orinteresting)wc --help --versionvs.wc --version --help

Non-equivalentinGNUwcDowecaretopreservethis?!

Howaboutspacing?!

Refactoring

WhatIsRefactoringRefactoring is a transformation of code into sufficientlyfunctionally equivalent code that has "better" internalproperties.

"MartinFowlerdefinesas"achangemadetotheinternalstructureofthesoftwaretomakeiteasiertounderstandandcheapertomodifywithoutchangingitsobservablebehavior"(Fowler1999)"—McConnell,24.2

"Sufficientlyfunctionallyequivalent"Userobservable/desirablebehaviourispreservedUptosomepoint

ExamplesForexample,amonolithicscript

haslowtestability(onlysystemtests!)replaceitwithasetoffunctions

e.g.,forarghandling,counting,andprintingresultsresult:easytotestscript

Forexample,hardcodedvaluesgreatforgettinggoing(techdebt!)refactorbymakingthemconfigurable

easiertotweakoreventuallymakeaparameterresult:moreflexibility!

CodeSmellsProblemsigns(selectsample,McConnell )

Codeisduplicated.Aroutineistoolong.Aloopistoolongortoodeeplynested.Inheritancehierarchieshavetobemodifiedinparallel.Aclassdoesn'tdoverymuch.Aroutinehasapoorname.Commentsareusedtoexplaindifficultcode.Aprogramcontainscodethatseemslikeitmightbeneededsomeday.

24.2

KnownDebtCodesmellsindicate(potentially)unknowndebtButthere'sexplicit,knowndebt

HacksdonefortimepressureIncompletetransitionsfromearlierdesignsLearningcodeTechnologyworkaroundsCodefordiscardedfeaturesOverengineeredcode

WhatRefactoringIsNotCodecreation

RefactoringmightenableorfacilitatenewfunctionalityButyoushouldn'taddwhilerefactoring

Bugfixing!Again,mayfacilitateRefactoringmayrevealor"fix"bugs

PerformancetuningSeeaboveCleancodemaybefaster...ornot!

DesignchangesorrearchitectingPrescursoractivity!

RefactoringPreconditionsTests,tests,tests

Evenwhenapplying"automatic"refactoringsRemember,nochangeinbehavior

Uptoapointatleast!Forsimplerefactorings

useatool!e.g.,renamingaroutine

Forcomplexrefactoringshaveaplan!

andtest!

TechnicalDebtCaseStudySlides