TechVision: Test Data on Demand: Delivering the Right Data, to the Right Place, at the Right Time

World®’16

TechVision:TestDataonDemand:DeliveringtheRightData,totheRightPlace,attheRightTimeArifMuhammad– DirectorProductManagement– CATechnologiesAbhijitMugali – SrPrincipalProductOwner,CATestDataManager– CATechnologies

DO5X42S

DEVOPS

2 ©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD

©2016CA.Allrightsreserved.Alltrademarksreferencedhereinbelongtotheirrespectivecompanies.

Thecontentprovidedinthis CAWorld2016presentationisintendedforinformationalpurposesonlyanddoesnotformanytypeofwarranty. The informationprovidedbyaCApartnerand/orCAcustomerhasnotbeenreviewedforaccuracybyCA.

ForInformationalPurposesOnlyTermsofthisPresentation


Abstract

Inordertodeliverqualitysoftwareontime,testersrequireondemandaccesstotheexactdataneededtosatisfyagivensetoftestcases.However,theyarefrequentlyleftwaitingidlyforanupstreamteamwhoareoverwhelmedwithrequeststoprovisiontheirdata.Dataisfurtherunavailableinparallel,andrareorinterestingdataisusedupbyanotherteam.Often,thedataeventuallyprovisionedisalargecopyofproductiondata,sothattestersspendevenmoretimelookingforthedatatheyneedforagiventest,orhavetocreateitmanuallywhenitdoesn’texist.

Thissessionwillofferadifferentapproachtodataprovisioning,whichremovesthedependencyoftestersonanupstreamteam.Instead,“goldcopy”datathatcontainsallthepossiblecombinationsneededfortestingiscreatedandisstoredasre-usableassetsinacentralrepository.Fromthere,thedatasetsaremadeaccessibletotestersondemand,usingaself-serviceweb-portal.Dynamicformbuildingallowsdatarequestformstobeconstructedusingtheexactcriterianeededforspecifictestcases,usingthelanguageoftherequirementsandteststhemselves.Thedataisthendeliveredautomaticallyandisclonedasitisprovisioned.There-usabledatasetscanthereforebeusedinparallel,deliveringqualitysoftwareearlier,andatlesscost.

ArifMohammedCATechnologiesDirectorProductManagement

AbhijitMugaliCATechnologiesSr.PrincipalProductOwner


Agenda

DATAPROVISIONING:WHATDOESITMEANFORTESTERS?

SELF-SERVICE,ONDEMANDPROVISIONING

CATDMTESTDATAASADATADELIVERYPLATFORM

SETTINGUPTHESERVICES

COMPLETE,END-TO-ENDTESTDATAMANAGEMENT

Q&A

1

2

3

4

5

6


DataProvisioning:WhatDoesItMeanForTesters?


TestingBecomestheBlockerforAgileAdoption

TestersfeelhelplessastheycannotfindTestDataforthetestingscenariostheyneedtocoverintheshort

timeframetheyhave.


TestDataMgmt

Team

TestDataProvisioningProcessHowisitdonetoday?

Upstreamdependency

Manualprovisioning

Dataconstraints

DataRequests

Testers

6-8weeksturnaroundtime

CompositeAppsDataisspreadacrossMultipleDBs


DataProvisioned

NoMetrics,NoVisibilityandNoCertainty

DataRequest


Whatiftherewastechnologytoprovideself-servicedataaccesstotesters,withautomaticdelivery?

Demo

World®’16©2016CA.ALLRIGHTSRESERVED.@CAWORLD#CAWORLD9


Gartner’sITServiceManagementMaturityModel

Source:http://performancematters.blogspot.com/2005_11_01_archive.html


WhatTestingTeamsWant?

§ Awayforthemtofind/minetheBusinessObjectacrossenvironmentsforthescenariostheyareabouttoTest.

§ Abilitytocopy/clonefoundBusinessObjectsintotheirtargetenvironments

§ DecoupleTestDataFromTestCasessoitcanbeexecutedinanyenvironmentwheredatamatchingcriteriaexists

§ OptimallyAllocate/Reservedataforspecifictestcasesbeforetestcaseexecution


TranslatingTheseWantstoaService?

§ Lookforrepeatedrequestsofsametypeofdatawithvariations

§ Onceyouknowwhatisbeingrequestedagainandagaindecidefastestwaytoautomatethedatarequest

§ Choosefrom– Fullysyntheticallygenerated– LettheTesterfinddatafromotherenvironmentsandcloneittotarget– Createacentralcopyofre-usabledatasetpatternsintheformofGold

copyandkeepaugmentingtheGoldcopywithmorepatterns


GetYourDatatoServe– GoldenCopy§ LogicalConceptofgroupofinterrelateddatabasesacrossmultipledata

sources

§ ContainssetofbasedatapatternsinterestingfortheTesters

§ Dataneedstobereferentiallycorrectacrosssourcessothatanendtoendbusinessscenarioacrossappcomponentistestable

§ GenerallythegoldcopygetreplicatedtoDevandTestenvironments

§ NewDataintotheGoldcopycancomefromproductionrefreshorSyntheticDataGeneration


Step1– AcquiretheData– Traditionalway

COPYPRODUCTIONDATA SUBSET MASK

Productiondatadoesnotprovidethedatatestersneedtoruntheirtests Dataismaskedandinsertedintodifferent

datastoresfordev/test


Step2– FindCoverageGaps

Productiondatais:§ Drawnfrompastscenarioswithusers

behavingasexpected§ Itthereforedoesnotcontainbaddata,

outliersorfuturescenarios

Testersneed:§ Datatosatisfythefullrangeoftestcases§ Thisincludesnegativetesting,datafor

boundarytesting,etc.

Ourauditshavefoundthatittypicallycoversjust10-20%oftests


Step3– FilltheGapsUsingSyntheticDataGeneration

§ Profiledata,modelit,andaccuratelymeasureitscoverage

§ Generaterichsyntheticdatawhichprovides100%coverage

§ Covereveryoutlier,unexpectedresult,boundaryconditionandnegativepath

NotReadyforTesting!

CATestDataManager+Requireddatacharacteristics

Provisionfitforpurpose dataanytimeandeverytimeProvisiondatawithorwithoutaccesstoproductionsystems

ReadyforTesting!


GoldCopyisNotaCopyofProductionData

Productiondatais:§ Drawnfrompast,scenarioswithusers

behavingasexpected§ Itthereforedoesnotcontainbaddata,

outliersorfuturescenarios

Testersneed:§ Datatosatisfythefullrangeoftestcases§ Thisincludesnegativetesting,datafor

boundarytesting,etc.

Ourauditshavefoundthatittypicallycoversjust10-20%oftests


TheNew“GoldStandard”:1. Providesastandardsetofdatatotestwith

2. Datais“production-like”,up-to-dateandcontainsallpreviousdata

3. Itoversallpossibleteststhatneedtoberun,includingfutureandnegativescenarios

4. Containsenoughdatatotestwithrepeatedly

5. Isup-to-date,Containsnosensitivedata

6. Newdatacanbeeasilyandcontinuouslyadded,withcloning,reservationandlocking

7. Datacanbeclonedwithouttheneedtoduplicate

□□□□□□□


CreateSelf-ServedTestDataServices

§ Basedonspecificrequirementdecideon– SyntheticDataGenerationservicefromscratch– moretimeconsumingto

setup– Clone

§ Lettesterfindspecificdatapatternfromtheoneenvironment/GoldCopyandmakeacloneintodifferentorsameenvironmentwithcertainvariations

– Hybridservice– setofbusinessobjectsareclonedwhiletransactionaldataisGenerated

§ AllowTestertoReservetheDatafortheirspecifictestcasestoavoidoversteppingusingTestDataCriteria-TestCaseMapping


Complete,End-to-EndTestDataManagement


EnterpriseWideTDMCOE

TestDataWarehouse

App2App1

EnterpriseTestDataManagementasCOEforAgileTeamsCA’sVision

OpsControlled Dev Controlled

Profile,Subset&Mask

VisualizeandAnalyzeData

Quality

Copy,Clone,GenerateData

toTarget

FindDataacrossenvironments

Configure

Create

ManageTestDataEngineer

Scrumteams

DeliverDatainformatuser

needs

LargenumberofDB&DataSourcesSupported

AcrossMainframeAndDistributed

ProductionDB

App2App1

TestDataEngineer

DataServicesCatalog

AgileCentral ServiceVirtualization


Questions?


MustSeeDemos

DataComplianceandTesting

CATDMTheatre5

DeliverTestDataFaster

CATDMTheatre5

UseModelBasedTesting

CAARDTheatre5

AchieveMaximumTestCoverage

CAARDTheatre5


Stayconnectedatcommunities.ca.com

Thankyou.


DevOps– ContinuousDelivery

FormoreinformationonDevOps– ContinuousDelivery,pleasevisit:http://cainc.to/PiTFpu

TechVision: Test Data on Demand: Delivering the Right Data, to the Right Place, at the Right Time

Technology

Transcript of TechVision: Test Data on Demand: Delivering the Right Data, to the Right Place, at the Right Time