Requirements are King – Better Requirements = Better Software

World®’16

RequirementsareKing– BetterRequirements=BetterSoftwareBenjaminJohnson-Ward– SeniorConsultant,Presales– CATechnologies

DO5X23E

DEVOPS

ForInformationalPurposesOnlyTermsofthisPresentation

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

Abstract

Countlessresearchandcasestudieshavelongpointedtoacold,hardtruth:therequirements,notdevelopersortesters,introducethevastmajorityofdefectsthatwreakhavocinproduction.Ratherthansolvethisissue,thedevelopmentpracticesthathavearisenaround“Agile”haveintroducedanewchallenge,astestershavetopiecetogetherandreacttoaconstantbarrageofdisparateuserstoriesandchangerequests.What’smore,therequirementsremainstatic,sothattestershavetomanuallyconvertthemintotests.Thisisaslow,unsystematicprocess,whichcreatesdefectswhentheambiguousrequirementsaremisinterpreted.Inorderfortestinganddevelopmenttokeepupwithconstantlychanginguserneeds,requirementsmustbecapturedinaformatwhichminimizesmiscommunication,canreacttochange,andenablessubsequenttestassetstobederivedautomatically.§ Thissessionwillsetouthow“active”flowchartmodellingcan:§ Provideaformatalreadyfamiliartobusinessanalyststocaptureuserneeds§ Provideaformatfortesterstogenerateoptimizedtestcases,testscripts,dataandexpectedresults

automatically.§ AutomaticallyreflectchangesmadetotherequirementsinthetestsassetsInthisrequirements-drivenapproach,testingtherebybecomesalargelyautomatedcomparisonofhowasystemshouldfunctiontothesystemthathasbeendelivered.

BenjaminJohnson-WardCATechnologiesSeniorConsultant,Presales

Agenda

WHATISAREQUIREMENT?

WHYFOCUSONREQUIREMENTS?

HOWABOUTSOFTWAREENGINEERING?

HOWABOUTREQUIREMENTSENGINEERING?

INTRODUCINGTHEREQUIREMENT-DRIVENAPPROACH

WhatisaRequirement?

Definition:Requirement

Requirement:singulardocumentedphysicalandfunctionalneed thataparticulardesign,product

orprocessmustbeabletoperform.

ISO/IEC/IEEE29148:2011

Wikipedia

Requirement:statementwhichtranslatesorexpressesaneed anditsassociatedconstraintsand

conditions.

WhatisaRequirement?

ErikSimmons,nuCognitive,“21st CenturyRequirements”,11/08/11

Arequirementisastatementof:

1. Whatasystemmustdo(functionalrequirement)

2. Howwellthesystemmustdowhatitdoes(qualityorperformancerequirement)

3. Aknownresourceordesignlimitation (constraint)

Moregenerally,

Arequirementisanythingthatdrivesadesign choice

ThePurposeofRequirements

Thepurposeofrequirementsistohelpestablishaclear,common,andcoherentunderstandingofwhatthesystemmustaccomplish.

Sowhyfocusonrequirements?ErikSimmons,nuCognitive,“21stCenturyRequirements”,11/08/11

Allstatementsareunambiguous,complete,

andconcise

Common

Allstakeholderssharethesameunderstanding

Coherent

Allstatementsareconsistentandforma

logicalwhole

WhyFocusonRequirements?

WhataretheMainSoftwareChallengesYouareFacing?

Pre-webinarsurvey,“TestingImperativesintheWorldofAgileDevelopmentandContinuousDelivery”,HuwPrice&JonathonWright,28/07/15

0% 10% 20% 30% 40% 50% 60% 70%

Time/resourcesintestdatacompliance(PII)

Defectsstemmingfromambiguousrequirements

Testinginnefficienciesleadingtohighercost

Lackoftestcoveragecreatingdefects/rework

Difficultyfindingtherightdataforaparticulartest

Manualtestingleadingtoprojectdelays

VAGUENESS

ExamplesofAmbiguityandIncompleteness

Poor ScopingAmbiguous Reference

Omission Acronyms

Aliasing

Sentence Structure

DanglingElse

Incomplete Glossary

Implicit Constraints

I.E. or E.g.?

LexicalAmbiguity

Polysemy

Passive Sentences

WhatMakesaGoodRequirement?

§ complete§ consistent§ correct§ modifiable§ ranked§ traceable§ unambiguous§ understandable§ testableNASA,AutomatedRequirementsMeasurementTool,1999,http://satc.gsfc.nasa.gov/tools/arm (archive.org)

WhatistheProblemwithRequirements?

§ Thoughaplethoraoftechniquesexist,mostarewritteninambiguousnaturallanguage

§ Therequirementsare“static”–theyoffernowaytoderivetestsdirectlyfromthem…

§ …nowaytoupdatetestswhentherequirementschange– thishastobedonemanually

BenderRBT,2009

UserTheenduserknowswhattheywant

TheSDC:What’sSupposedtoHappen

BusinessAnalystTheanalystspecifieswhatthatis

DeveloperTheprogrammerwritesthecode

TesterThetesterteststheprogram

TheFundamentalIssueWithThisModel

§ Ateachstagethatinformationispassedonandtransformedtoanotherform,thereisanaturaldegradationinquality

§ ThetypicalSDLChasseveralofthese“hops”:1. Requirementsareconvertedtocode2. Requirementsareconvertedtotestcases,tobeappliedtothecode3. Testcasesareconvertedintoautomatedtests4. Datahastobefoundorcreated,basedontherequirements,for

developmentandtesting

InformationHops:Enterprise-Wide“ChineseWhispers”

Theimpactonqualityiscreatedbyuncertainty.Some(inherent)uncertaintyisunavoidable.Someisavoidable,andshouldbeavoided.

“RumsfeldMatrix”– AvoidablevsUnavoidableUncertainty§ InformationintheSDLCcanbebrokendowninto:

– KnownKnowns– UnknownKnowns– KnownUnknowns– UnknownUnknowns

Knowns Unknowns

Known Information AvoidableUncertaintyàObservableDefects

Unknown AvoidableUncertaintyàObservableDefects

Unavoidable Uncertainty

Inanidealworld,testerswillbeprovidedwithalltheinformationtodiscoverobservabledefects.Whateverpreviouslyunknowableinformationtheydiscoverwilladdtothe“knownknowns”

Unambiguous,IncompleteRequirements

§ Poorqualityrequirementsintroduceuncertaintyatthebeginning,whichisthenperpetuatedthroughouttheSDLC

§ Asystemcanbebuildaroundmisunderstanding,sothat:– Atleast56%ofdefectsstemfromambiguityinrequirements1 – some

placethisashighas59%2 oreven65%3

– 64%oftotaldefectcost3originateintherequirementsanalysisanddesignphase– someplacethisashighas80%1

– 40-50%ofprojectcosts areexpendedinrework4

– Onaverage,only69%ofdesiredfunctionalityisactuallydelivered5

Introduceavoidableuncertaintyattheverybeginning

1– BenderRBT,2009

2– ITUniveristy,Copenhagen,2001

3- HyderabadBusinessSchool,20124– Critical-Logic,20145– StandishGroup’sChaosManifesto2014

Unambiguous,IncompleteRequirementsAnexampleambiguityreview:

AmbiguousNaturalLanguage

§ “Timeflieslikeanarrow;fruitflieslikeanapple”

§ “BuffalobuffaloBuffalobuffalobuffalobuffaloBuffalobuffalo”– Agrammaticallycorrectsentence,whichmeans“InthecityofBuffalo,

bisonwhoarebulliedbybison,arethemselvesbullyingbison”

§ Wesawherduck

Sentencestructure,homonyms,polysemy,lexicalambiguity,etc.

TheDanglingElse

§ IFATHENIFBTHENCELSED– IFATHEN(IFBTHEN(C)ELSED)– IFATHEN(IFBTHEN(C))ELSED

§ IFRAINTHENIFWINDYTHENCOATELSET-Shirt– Ifitisraining,thenwearacoatifitisalsowindy.Ifitisrainingbutnot

alsowindy,wearaT-Shirt.– Ifitisraining,andifitiswindy,thenwearacoat.OtherwisewearaT-

Incompletesystemlogicstemmingfromsyntacticalambiguity

ThoughaPlethoraofTechniquesExist,MostAreWritteninAmbiguousNaturalLanguage

WhatAretheChallengesWithRequirements?

1– BenderRBT,20092– ITUniversity,Copenhagen,20013- HyderabadBusinessSchool,20124– Critical-Logic,20145– StandishGroup’sChaosManifesto2014

§ Asystemcanbebuildaroundmisunderstanding,sothat:– Atleast56%ofdefectsstemfromambiguityinrequirements1 – some

placethisashighas59%2 oreven65%3

– 64%oftotaldefectcost3originateintherequirementsanalysisanddesignphase– someplacethisashighas80%1

– 40-50%ofprojectcosts areexpendedinrework4

– Onaverage,only69%ofdesiredfunctionalityisactuallydelivered5

TheDanglingElse

§ IFATHENIFBTHENCELSED– IFATHEN(IFBTHEN(C)ELSED)– IFATHEN(IFBTHEN(C))ELSED

§ IFRAINTHENIFWINDYTHENCOATELSET-Shirt– Ifitisraining,thenwearacoatifitisalsowindy.Ifitisrainingbut

notalsowindy,wearaT-Shirt.– Ifitisraining,andifitiswindy,thenwearacoat.Otherwisewear

aT-Shirt

Incompletesystemlogicstemmingfromsyntacticalambiguity

TheMissingElseAcommonmenacewhenusingBehaviour-DrivenDevelopment,GherkinandRSPEC

GIVENCurrentAccount

AndGIVENnegativebalance

WHENcustomerattemptspayment

THENgointooverdraft

Requirements

Asmanyas56%ofdefectsareavoidedindevelopment

§ TestcasescanbeAuto-generatedandre-generatedfromrequirements

§ Theyareoptimizedtoprovide100%coverageinfewertests

§ Changescanbequicklyandeasilymade

§ Requirementsandtestcasesarelinkedtodatawith100%coverage,includingnegativeandfuturescenarios § Automatedtests

aregeneratedfromtestcases

§ Areupdatedifrequirementschange

§ Exhaustivelytestapplication

§ Pallet-basedUIAutomation

Requirementsaremodelledtoacomplete,unambiguous,“Active”Flowchart

ReduceUncertainty– ClearRequirementsandFewer“Hops”Code TestCases TestData Automation

APossibleSolution:Model-BasedTesting

1. Mappingrequirementstoanunambiguous,activeflowchart– Codersknowwhatneedstobecoded

2. Auto-generatetestcasesandAutomatedtests–nomore“Hops”– Executethesetestsagainstthecode,testingthatitmatches

thefunctionalityspecifiedintherequirements

3. Updatethemodelastherequirementschange,orastestinguncoversfurther,previously“unknown”information

Testasystemagainstanagreedupon,cleardefinitionofwhatthefunctionalityshouldbe

HowAboutSoftwareEngineering?

WhatisEngineering?

Engineeringistheapplicationofheuristicsunderuncertaintytocausethebestpossiblechangewithintheavailableresources(BillyVaughnKoen*)§ Aheuristicisanythingthatprovidesaplausibleaidordirectioninthe

solutionofaproblem§ Useofheuristicsdoesnotguaranteeasolutiontoaproblem,anddifferent

heuristicscanofferconflictingadviceforanygivensituation§ Heuristicsworkbyexploitingthestructureofanenvironmenttosimply

decisionmakingandnarrowthesolutionsearchspace

ErikSimmons,nuCognitive,11/09/16adaptedfrom“adiscussionofthemethod”,BillyVaughnKoenOxfordUniversityPress2003

HowDoesThisApplytoSoftwareEngineering(SDLC)?Simpleexampleoflackofclarityandvisionwithinthesoftwaredevelopmentlifecycle

BusinessAnalyst Programmer TesterUser

TheUserKnowswhattheywantTheAnalystspecifieswhatthatis

TheProgrammerwritesthecode

TheTesterteststheprogram

Clarity&Vision

HowAboutWithinLean&Agile?

DeanLeffingwell,ScaledAgileFrameworkFoundations(v4.0.6),“EmbracingLean-Agilevalues”,2016

HouseofLean AgileManifesto

LEADERSHIP

Valueinthesustainablyshortestleadtime

Thatis,whilethereisvalueintheitemsontheright,wevaluetheitemsontheleftmore.

Weareuncoveringbetterwaysofdevelopingsoftwarebydoingitandhelpingothersdoit.Throughthisworkwehavecometovalue:

Individualsandinteractionsoverprocessesandtools

Workingsoftwareovercomprehensivedocumentation

Customercollaborationovercontractnegotiation

Respondingtochangeoverfollowingaplan

HowAboutRequirementsEngineering?

Definition:RequirementEngineeringISO/IEC/IEEE29148:2011

Wikipedia

Referstotheprocessofdefining,documentingandmaintainingrequirementstothesub-fieldsof

systemsengineeringandsoftwareengineeringconcernedwiththisprocess.

Interdisciplinaryfunctionthatmediatesbetweenthedomainsoftheacquirerandsuppliertoestablishand

maintaintherequirementstobemetbythesystem,softwareorserviceofinterest.Requirementsengineering

isconcernedwithdiscovering,eliciting,developing,analysing,determiningverificationmethods,validating,

communicating,documenting,andmanagingrequirements.

WhatisRequirementsEngineering?

Requirementsengineeringistheuseofheuristicsfordiscovering,documenting,andmaintaining asetofrequirementsforasystemorserviceAtthehighestlevel,requirementsengineeringseeksto:§ Definethenecessaryandsufficientsystemscopeandfeaturesto

enablesystemdevelopmentatanacceptablelevelofrisk§ Enablerequirements-drivenarchitecture,design,construction,

andverificationactivities§ Provideinformationaboutsystemdevelopmentprogressandstatus§ Provideinformationtoorganizationallearningandcontinuous

improvementeffortsErikSimmons,nuCognitive,11/09/16adaptedfrom“adiscussionofthemethod”,BillyVaughnKoen,OxfordUniversityPress2003

HowtoManageRequirementsEngineering?

ToGenerateTestCases

DesignandprovisionTestDataVirtualisation

ToManageChangeinTestCases

ToGenerateAutomation

ToEstimateComplexity

PopulateStoryBoards&Backlogs

ToBuildBetterRequirements

ToImproveExisting

TestCases DesignandprovisionServiceVirtualization

ImpactandDifferences

built by

CAAgileRequirementsDesigner

NativeSupport

IntroducingtheRequirement-drivenApproach

DesignOpsistheNewDeliveryParadigm:FromIdeationtoDesign,DevelopmentandTesting

ModelRequirementsasan“Active”Flowchart

§ AformalmodelthatisaccessibletothebusinesswhoalreadyuseVISIO,BPM,etc.

§ Whichisalsoamathematicallyprecisemodelofasystem,sothatiteliminatesambiguityandincompleteness

§ Itcanbeusedbytestersanddevelopers–itbringstheend-user,businessandITintoclosealignment

TestingWiththe“Active”Flowchart

§ Testerscanoverlaytheflowchartwithallthefunctionallogicanddatainvolvedinasystem

§ Testscanthereforebeautomaticallyderivedfromit

GeneratingTestCases

§ Exhaustivelytestamodel– extractallpossibleroutesfromstart->end

§ Eachroute/pathbecomesatestcase

§ Numberofroutesgrowsexponentiallywitheveryaddeddecision32nodes+62edges=1,073,741,824possibleroutes

2145Paths

Optimization/Coverage

§ Coverage,inmostcases,isrelatedtohowmuchfunctionalityisbeingcoveredinatestcase.

§ Traversethemodelbysatisfyingcoverageconstraints:19PathsEdgePairs

9PathsInOutEdges

5PathsEdges

3PathsNodes

GenerateTestData

NotReadyforTesting!

CATestDataManager+Requireddatacharacteristics

Provisionfitforpurpose dataanytimeandeverytimeProvisiondatawithorwithoutaccesstoproductionsystems

ReadyforTesting!

§ Automaticallyprofiledata,modelit,andaccuratelymeasureitscoverage§ Generaterichsyntheticdatawhichprovides100%coverage§ Covereveryoutlier,unexpectedresult,boundaryconditionandnegativepath

ProvideFullTraceabilitywithRequirements

§ Knowwhatneedstobere-testedandwhentheintegrityofasystemisatrisk…“IfIchangethis,whatwillIbreak?”

§ Theimpactofachangemadetoanindividualcomponentisidentifiedsystemwide

§ Theimpactontestcasesanduserstoriesupanddownasystemcanalsobeidentifiedautomatically

ButAllofThisStillDependsontheRequirementsBeingRight!§ Yes,youcanreaphugerewardsfrommaintainingonevery

highqualityartefact,ratherthanthousands.

§ No,nosilverbullet,modelscanbewrong,peoplecanstillmakemistakes

§ TheideabehindMBTandARDistomakethosepotentialmistakesmoreobviousandeasiertofixoncediscovered

§ Andofcourseleveragethemodel,whilstacknowledgingtheresidualrisk.

FunExample:PokémonGo

PokémonGo,HuwPrice,22/07/16,(www.grid-tools-downloads.com/huw/POKEMONGO.zip)

Summary:Heuristics,RiskandRequirementsEngineering§ Valueiskeytodeliveringsoftwarewhichmatterstousers:

– asolutionmustbefrequentlyanditerativelymeasuredforvalue

§ PoorRequirementsdegradequalitybutalsointerruptthevaluestream:– Testerscannottestrigorouslyorquicklyagainstcustomerneeds

§ Requirementscanalsothereforebekeytoimprovingthedeliveryrateofvaluablesoftware

§ Enablingrequirements-drivendesign,constructionandverification:– Modelrequirementsas“active”flowcharts– Testfromtheflowchartwithauto-generationoftests/scripts– Generatetestdatatomatchthetestcases– Updatetestswhentherequirementschange

Questions?

RecommendedSessions

SESSION# TITLE DATE/TIME

DO5T05TTechTalk:GoingFullCircle- LinkingCodetoTeststoRequirementsandBackAroundAgain

11/16/2016at12:45pm

DO5X27S What'sNewinCAAgileRequirementsDesigner? 11/17/2016at12:45pm

DO5X40S CaseStudy:WhyRabobankWasInspired 11/17/2016at3:45pm

MustSeeDemos

UseModelBasedTesting

CAARDDevOpsTheatre5

DevOps5Theatre

AchieveMaxTestCoverage

CAARDDevOpsTheatre5

ImproveDataCompliance

CATestDataManagerDevOpsTheatre5

DeliverTestDataFaster

CATestDataManagerDevOpsTheatre5

Stayconnectedatcommunities.ca.com

Thankyou.

DevOps– ContinuousDelivery

FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu

Requirements are King – Better Requirements = Better Software

Technology

Transcript of Requirements are King – Better Requirements = Better Software

Writing Better Requirements - University of Ottawabochmann/SEG3101/Notes/SEG3101-ch3-3... · Writing Better Requirements. Martha can’t write requirements because ... •Each requirement

How to prioritize requirements - better and faster (workshop), Razvan Radulian

Would you have been a “better” King than Charles I.

KING III Impact on Government. Contents Introduction Key Principles of King III Governance Framework and Application New Requirements Chapters 1 to 11.

CONCEPT OF OPERATIONS AND REQUIREMENTS DEVELOPMENT … · CONCEPT OF OPERATIONS AND REQUIREMENTS DEVELOPMENT ... provide better travel information to travelers, who can make better

Aspire 3610 Series - s-manuals.com filedecided for Acer's "global" product offering. To better fit local market requirements and enhance product To better fit local market requirements

Toward Better Password Requirements

“We couldn’t have asked for a better locationclose to Jacó ... · Bedroom 1 1 king bed Bedroom 2 1 king bed Bedroom 3 1 king bed Bedroom 4 1 king bed Private transportation We

Industrial Revolution Why you live better than a king!

Achieving better requirements in agile

MSN STUDENT H - King University concentration offered in the King University MSN program. Educational and experience requirements for ...

Writing Better Requirements

Knowledge Factors and Models of Requirements Change ...requirements-to-code traces for better managing requirements change[16]. Ernst et al sought solutions for managing requirements

Technology for a better society TDT4140 Software Engineering 29.01.2013: 1 Experiences from Requirements Engineering: Coping with requirements in large,

INNOVATION FOR A BETTER WORLD U13X - KING LONG

Special freight requirements demand specializedFreight Delivery Services Special freight requirements demand specialized freight experts. Faster. Better. Smarter. IntelliQuick Delivery

Find Requirements Defects to Build Better Software

Innovations in Requirements Management for Better Feedback and Better Software

Data Collection and Analysis for Better Requirements

WITH CROWD-RE TO BETTER REQUIREMENTS